Sim eval — ep-0950_sr-0.527 (Phase B confirm (50 ep/task, 3 seeds))
- phase:
Phase B confirm (50 ep/task, 3 seeds) - mean_success_rate:
0.506±0.007571877794400371 - n_test: 500
- n_test_per_task: 50
- test_start_seed: 1000
- num_exp: 3
Per-task SR
| Task | SR |
|---|---|
KITCHEN_SCENE4_put_the_black_bowl_in_the_bottom_drawer_of_th |
94.7% |
KITCHEN_SCENE3_turn_on_the_stove_and_put_the_moka_pot_on_it |
88.7% |
KITCHEN_SCENE6_put_the_yellow_and_white_mug_in_the_microwave |
74.0% |
STUDY_SCENE1_pick_up_the_book_and_place_it_in_the_back_compa |
66.0% |
LIVING_ROOM_SCENE2_put_both_the_cream_cheese_box_and_the_but |
56.7% |
LIVING_ROOM_SCENE5_put_the_white_mug_on_the_left_plate_and_p |
48.0% |
KITCHEN_SCENE8_put_both_moka_pots_on_the_stove |
43.3% |
LIVING_ROOM_SCENE6_put_the_white_mug_on_the_plate_and_put_th |
34.0% |
LIVING_ROOM_SCENE2_put_both_the_alphabet_soup_and_the_tomato |
0.7% |
LIVING_ROOM_SCENE1_put_both_the_alphabet_soup_and_the_cream_ |
0.0% |