Robotics
KempnerInstitute commited on
Commit
8eca7e6
·
verified ·
1 Parent(s): 1ee434e

Upload folder using huggingface_hub

Browse files
This view is limited to 50 files because it contains too many changes.   See raw diff
Files changed (50) hide show
  1. .gitattributes +28 -0
  2. README.md +39 -3
  3. data/agibot_beta_temp/cleaned_metadata.csv +3 -0
  4. data/agibot_beta_temp/merged_metadata.csv +3 -0
  5. data/droid/cleaned_metadata.csv +3 -0
  6. data/droid/merged_metadata.csv +3 -0
  7. data/ego4d/cleaned_metadata.csv +3 -0
  8. data/ego4d/merged_metadata.csv +3 -0
  9. data/epic_kitchens/cleaned_metadata.csv +0 -0
  10. data/epic_kitchens/merged_metadata.csv +3 -0
  11. data/openx_embodiment/bridge/cleaned_metadata.csv +0 -0
  12. data/openx_embodiment/bridge/merged_metadata.csv +3 -0
  13. data/openx_embodiment/language_table/cleaned_metadata.csv +3 -0
  14. data/openx_embodiment/language_table/merged_metadata.csv +3 -0
  15. data/ours_test/metadata.csv +101 -0
  16. data/ours_test/test_images/0001.jpg +0 -0
  17. data/ours_test/test_images/0002.png +3 -0
  18. data/ours_test/test_images/0003.png +3 -0
  19. data/ours_test/test_images/0004.jpg +0 -0
  20. data/ours_test/test_images/0005.jpg +0 -0
  21. data/ours_test/test_images/0006.jpg +0 -0
  22. data/ours_test/test_images/0007.jpg +0 -0
  23. data/ours_test/test_images/0008.jpg +0 -0
  24. data/ours_test/test_images/0009.jpg +0 -0
  25. data/ours_test/test_images/0010.jpg +0 -0
  26. data/ours_test/test_images/0011.jpg +0 -0
  27. data/ours_test/test_images/0012.jpg +0 -0
  28. data/ours_test/test_images/0013.jpg +0 -0
  29. data/ours_test/test_images/0014.JPG +0 -0
  30. data/ours_test/test_images/0015.JPG +0 -0
  31. data/ours_test/test_images/0016.jpg +0 -0
  32. data/ours_test/test_images/0017.jpg +0 -0
  33. data/ours_test/test_images/0018.png +3 -0
  34. data/ours_test/test_images/0019.png +3 -0
  35. data/ours_test/test_images/0020.jpg +0 -0
  36. data/ours_test/test_images/0021.jpg +0 -0
  37. data/ours_test/test_images/0022.jpg +0 -0
  38. data/ours_test/test_images/0023.jpg +0 -0
  39. data/ours_test/test_images/0024.jpg +0 -0
  40. data/ours_test/test_images/0025.jpg +0 -0
  41. data/ours_test/test_images/0026.jpg +0 -0
  42. data/ours_test/test_images/0027.jpg +0 -0
  43. data/ours_test/test_images/0028.jpg +0 -0
  44. data/ours_test/test_images/0029.jpg +0 -0
  45. data/ours_test/test_images/0030.jpg +0 -0
  46. data/ours_test/test_images/0031.jpg +0 -0
  47. data/ours_test/test_images/0032.jpg +0 -0
  48. data/ours_test/test_images/0033.jpg +0 -0
  49. data/ours_test/test_images/0034.jpg +0 -0
  50. data/ours_test/test_images/0035.jpg +0 -0
.gitattributes CHANGED
@@ -33,3 +33,31 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ data/agibot_beta_temp/cleaned_metadata.csv filter=lfs diff=lfs merge=lfs -text
37
+ data/agibot_beta_temp/merged_metadata.csv filter=lfs diff=lfs merge=lfs -text
38
+ data/droid/cleaned_metadata.csv filter=lfs diff=lfs merge=lfs -text
39
+ data/droid/merged_metadata.csv filter=lfs diff=lfs merge=lfs -text
40
+ data/ego4d/cleaned_metadata.csv filter=lfs diff=lfs merge=lfs -text
41
+ data/ego4d/merged_metadata.csv filter=lfs diff=lfs merge=lfs -text
42
+ data/epic_kitchens/merged_metadata.csv filter=lfs diff=lfs merge=lfs -text
43
+ data/openx_embodiment/bridge/merged_metadata.csv filter=lfs diff=lfs merge=lfs -text
44
+ data/openx_embodiment/language_table/cleaned_metadata.csv filter=lfs diff=lfs merge=lfs -text
45
+ data/openx_embodiment/language_table/merged_metadata.csv filter=lfs diff=lfs merge=lfs -text
46
+ data/ours_test/test_images/0002.png filter=lfs diff=lfs merge=lfs -text
47
+ data/ours_test/test_images/0003.png filter=lfs diff=lfs merge=lfs -text
48
+ data/ours_test/test_images/0018.png filter=lfs diff=lfs merge=lfs -text
49
+ data/ours_test/test_images/0019.png filter=lfs diff=lfs merge=lfs -text
50
+ data/ours_test/test_images/0046.jpg filter=lfs diff=lfs merge=lfs -text
51
+ data/ours_test/test_images/0051.jpg filter=lfs diff=lfs merge=lfs -text
52
+ data/ours_test/test_images/0072.JPG filter=lfs diff=lfs merge=lfs -text
53
+ data/ours_test/test_images/0096.JPG filter=lfs diff=lfs merge=lfs -text
54
+ data/ours_test/test_images/0112.JPG filter=lfs diff=lfs merge=lfs -text
55
+ data/ours_test/test_images/0141.JPG filter=lfs diff=lfs merge=lfs -text
56
+ data/ours_test/test_images/0161.JPG filter=lfs diff=lfs merge=lfs -text
57
+ data/ours_test/test_images/0165.JPG filter=lfs diff=lfs merge=lfs -text
58
+ data/pandas/cleaned_metadata.csv filter=lfs diff=lfs merge=lfs -text
59
+ data/pandas/cleaned_metadata_with_youtube_key.csv filter=lfs diff=lfs merge=lfs -text
60
+ data/pandas/merged_metadata.csv filter=lfs diff=lfs merge=lfs -text
61
+ data/pandas/merged_metadata_with_youtube_key.csv filter=lfs diff=lfs merge=lfs -text
62
+ data/something_something_v2/cleaned_metadata.csv filter=lfs diff=lfs merge=lfs -text
63
+ data/something_something_v2/merged_metadata.csv filter=lfs diff=lfs merge=lfs -text
README.md CHANGED
@@ -1,3 +1,39 @@
1
- ---
2
- license: apache-2.0
3
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ This folder contains the trained large video planner checkpoints (14B parameter) and all the metadata for eight dataset sources: Agibot-world, droid, bridge, language-tables, Pandas(filtered), SomethingSomethingV2, ego4d, epic_kitchens. We release a `merged_metadata.csv` and a `cleaned_metadata.csv` for each.
2
+ We also released our test set in `data/ours_test/` with the images and text instructions gathered from third-parties.
3
+
4
+ ## Trained Checkpoints for our Large Video Planner
5
+ `checkpoints/lvp_14B.ckpt` is the trained weights for the transformer backbone.
6
+
7
+ ## Dataset format
8
+ We train on a mixture of datasets, so we define a unified dataset format for consistency and ease of management.
9
+
10
+ Each dataset includes a global metadata file, typically named `metadata_merged.csv`, which contains key information for each video clip.
11
+
12
+ The file is named as metdata_**merged**.csv because each video clip may have multiple recaptions. Instead of saving the captions for each video into a list within a single csv row, we just create another row on the `metadata_merged.csv`. So `metadata_merged.csv` may contain multiple rows referring to the same video with different captions. For some dataset, we also provide a `cleaned_metadata.csv`, which contains a deduplicated version of the metadata (one entry per video) but excludes the additional recaptions.
13
+
14
+ Important fields of the global metadata includes:
15
+ 1. `video_path`: Relative path (from the metadata file) to the video clip.
16
+ 2. `trim_start`, and `trim_end` (optional): Specifies the trimmed segment of the clip. If absent, the full video is used.
17
+ 3. `gemini_caption`: Action-focused caption generated by Gemini Flash 2.0.
18
+ 4. `original_caption`: Original caption from the source dataset; used when no Gemini caption is available.
19
+ 5. `prompt_embed_path`: Path to precomputed T5 prompt embeddings (not released due to large size).
20
+ 6. `stable_brightess` (optional): 1.0 if brightness is stable, 0.0 otherwise. We recommend removing videos with `stable_brightess == 0.0`
21
+ 7. `stable_background` (optional): Either 1.0 or 0.0. Recommend to remove videos with `stable_background == 0.0`, this indicates the video has large average optical flow magnitudes, which very likely contains large background motions.
22
+ 8. `detected_hand_in_first_frame` (optional): 1.0 if a human hand is detected in the first frame, 0.0 otherwise. Videos with 0.0 often cause embodiment ambiguity and should be filtered out.
23
+ 9. There are some other fields which can help you understand more about this clips. `n_frames`, `n_fps`, `height`, `width`, ... etc.
24
+
25
+
26
+
27
+
28
+ ## Downloading the dataset
29
+ We provide dataset-specific download scripts for AgiBot World, DROID, Ego4D, EpicKitchens, and Something-Something in their respective dataset.py files within the `datasets/` folder of the relased code.
30
+
31
+ For downloading the filtered Pandas subset, we provide the unique `youtube_key_segment` for each video_clip, and the `trim_start`, and `trim_end` for each clip. To download these subset, please download the official metadata from [Pandas-70M](https://snap-research.github.io/Panda-70M/), then using the `youtube_key_segment` to find the URL of the video clips and then download with your own online video downloader.
32
+
33
+
34
+
35
+ For Bridge, please download from (Bridge)[https://rail-berkeley.github.io/bridgedata/].
36
+
37
+ For Language Table, please download from (Language Table)[https://github.com/google-research/language-table].
38
+
39
+
data/agibot_beta_temp/cleaned_metadata.csv ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3e7f425f58518218ec48d3fc326cbfadce36519173696b293b4406b03e1cc42d
3
+ size 198542119
data/agibot_beta_temp/merged_metadata.csv ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9aa035e4c86d5826c00d970bb1ab8e29d603a07d19c78d6bc0533622c9d05d8c
3
+ size 1007836169
data/droid/cleaned_metadata.csv ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:16056f12240ca704127b0f5537385f4285ff138b6320bb2aea5d1dea234b6e5b
3
+ size 62527026
data/droid/merged_metadata.csv ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2275eeff28e795499bafd4482a5505fe3520ab470972465662963669fa482286
3
+ size 633802873
data/ego4d/cleaned_metadata.csv ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e77e847444a764f7a734dd65363acdb659eed6273d84334ea337a35e24c44a24
3
+ size 12996168
data/ego4d/merged_metadata.csv ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:06a9168e15679664e79c59adfad5b00c8b9aa9d245d335c8773ff0b0882cf751
3
+ size 175207692
data/epic_kitchens/cleaned_metadata.csv ADDED
The diff for this file is too large to render. See raw diff
 
data/epic_kitchens/merged_metadata.csv ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7881afffda2e87111f36df7b353537ea89497a6f113b1d8b8982078fae8d963e
3
+ size 82276011
data/openx_embodiment/bridge/cleaned_metadata.csv ADDED
The diff for this file is too large to render. See raw diff
 
data/openx_embodiment/bridge/merged_metadata.csv ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:47eb8c5fcc2c0e1df2fce417e7325e9d2450783807056680c9e54215e927f23d
3
+ size 58338378
data/openx_embodiment/language_table/cleaned_metadata.csv ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:01de130779cc2bd683592e97120199e1b5d26caa7742d103cc0e45860a3dce3c
3
+ size 25910700
data/openx_embodiment/language_table/merged_metadata.csv ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:bcd4956b06107fd429db761d0ad81e8ddc44516172d096ebc2fe5422611db344
3
+ size 115551089
data/ours_test/metadata.csv ADDED
@@ -0,0 +1,101 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ image_path,original_caption,height,width,fps,n_frames,gemini_caption
2
+ test_images/0001.jpg,Break off a piece of tape,480,832,16,1,"A person holds the red tape dispenser with their left hand on a wooden desk and pulls a piece of clear tape with their right hand, tearing it against the serrated edge."
3
+ test_images/0003.png,Insert the straw into the cup,480,832,16,1,A person's left hand inserts a green straw into the cross-shaped opening on the lid of a clear plastic cup that is sitting on a wooden desk next to a keyboard.
4
+ test_images/0004.jpg,Lift up the chair from the back,480,832,16,1,A left hand reaches for the curved backrest of a pale-green wooden chair that is sitting on a patterned rug and lifts it.
5
+ test_images/0005.jpg,Move the knife out of the glass box,480,832,16,1,A hand removes the large kitchen knife with a grey and white handle from the square glass container on the wooden cutting board.
6
+ test_images/0006.jpg,Move the mannequin's arm so that its arms form a V shape,480,832,16,1,"A hand reaches for the wooden mannequin standing on a white shelf and lifts its right arm, positioning both arms upward in a V-shape."
7
+ test_images/0008.jpg,Open the juice bottle by twisting the cap counter clockwise,480,832,16,1,"A person holds a blue carton of Jumex Mango Nectar steady with their left hand while their right hand is positioned above the white cap, preparing to twist it counter-clockwise to open it."
8
+ test_images/0011.jpg,Pick up the glasses with left hand,480,832,16,1,The left hand moves inward to pick up the black-rimmed glasses from the white table.
9
+ test_images/0012.jpg,Pickup the fork,480,832,16,1,A right hand reaches to pick up a silver fork that is lying on a white desk next to a grey Logitech keyboard.
10
+ test_images/0017.jpg,Stir the tomatoes in the pot,480,832,16,1,A metal ladle stirs chopped tomatoes simmering in a steaming stainless steel pot on a white electric stovetop.
11
+ test_images/0018.png,Throw the cup into the recycling bin,480,832,16,1,"A hand holding a clear plastic cup drops it into the left opening of a white dual waste bin, which is labeled 'RECYCLE & NON-COMPOST'."
12
+ test_images/0020.jpg,Unplug the upper charging cable,480,832,16,1,A person's left hand unplugs the upper of two black cables from the side of a laptop.
13
+ test_images/0021.jpg,Use the pen to press the button F,480,832,16,1,A right hand uses a blue pen to press the 'F' key on the black keyboard of a laptop.
14
+ test_images/0022.jpg,Use the scissors to pick up the plastic bag,480,832,16,1,"A left hand holding a pair of grey and orange scissors moves toward a translucent plastic bag on a white table, preparing to pick it up."
15
+ test_images/0025.jpg,close_dishwasher,480,832,16,1,"A person's right hand lifts the open, downward-facing dishwasher door to close it."
16
+ test_images/0026.jpg,fetch_a_napkin,480,832,16,1,A left hand reaches toward a colorful Kleenex box on a wooden desk to pull out a tissue.
17
+ test_images/0027.jpg,flick_bathroom switch_off,480,832,16,1,"A right hand reaches for a dark, vintage-style toggle light switch on a white wall and flicks it down to the off position."
18
+ test_images/0028.jpg,flip_cup_upside_down,480,832,16,1,The left hand reaches for the green paper cup resting on the black leather couch and flips it upside down.
19
+ test_images/0029.jpg,fold_down_the_lamp_arm,480,832,16,1,A hand folds the upper arm of the white desk lamp down towards its base on the wooden desk.
20
+ test_images/0031.jpg,grab the pen from the desktop,480,832,16,1,"A person's left hand, wearing a grey and black plaid shirt, reaches over a white desk, positioning the thumb and index finger around a black pen that is lying on a paper with handwritten notes."
21
+ test_images/0033.jpg,grad the cup from the desktop,480,832,16,1,"A left hand reaches to grab a white paper cup filled with a clear liquid from a white desk, situated in front of a silver laptop."
22
+ test_images/0034.jpg,lift_up_the_door_handle,480,832,16,1,"A hand, extending from a blue sleeve, reaches for the brushed metal lever handle on the white paneled door and lifts it upward."
23
+ test_images/0035.jpg,move the mouse leftwards,480,832,16,1,"A right hand moves a white ergonomic mouse leftwards, sliding it from its position on top of the black keyboard."
24
+ test_images/0038.jpg,pick up the airpod box on the table,480,832,16,1,A left hand reaches onto the corner of a light-brown wooden table and picks up the white AirPods case.
25
+ test_images/0041.jpg,put the fork into the red cup,480,832,16,1,A hand holding a silver fork moves it from over the black mug and places it into the red cup decorated with a dog pattern.
26
+ test_images/0042.jpg,put the fork to the red cup,480,832,16,1,A hand moves a silver fork toward the red cup decorated with a repeating pattern of a cartoon French bulldog.
27
+ test_images/0043.jpg,put the spoon into the black cup,480,832,16,1,"A right hand holding a silver spoon places it into the black mug with the white ""The Robotics Institute"" logo."
28
+ test_images/0044.jpg,put the spoon into the red cup,480,832,16,1,"A person's right hand holding a silver spoon places it into the red cup decorated with French bulldog faces, which already contains a fork."
29
+ test_images/0046.jpg,squeeze_cup,480,832,16,1,A left hand reaches toward a green and white paper cup on a grey patterned carpet and squeezes it.
30
+ test_images/0048.jpg,turn the book to the next page,480,832,16,1,"The left hand holds an open book, its thumb placed on the bottom corner of the right-hand page, and turns the page to the left."
31
+ test_images/0049.jpg,unplug_the_cord,480,832,16,1,A hand reaches toward a white power strip on a grey carpeted floor to unplug a black electrical cord.
32
+ test_images/0050.jpg,zip_up_the_backpack,480,832,16,1,A person's hand reaches for the open zipper on the front of a black backpack resting on a wooden desk to zip it closed.
33
+ test_images/0051.jpg,cleaning_the_floor_with_paper_towel,480,832,16,1,"A hand wipes a light brown, wood-grain floor with a white paper towel."
34
+ test_images/0052.jpg,close_the_washing_machine_door,480,832,16,1,"A left hand pushes the open door of the silver, front-loading LG washing machine to close it."
35
+ test_images/0053.JPG,closing_the_mailbox,480,832,16,1,"A right hand reaches up to close the open metal door of a rusty, cylindrical mailbox mounted on a weathered wooden post next to brick stairs."
36
+ test_images/0054.JPG,closing_the_recycling_bin,480,832,16,1,A right hand pushes down the open blue lid of a recycling bin located on a street curb.
37
+ test_images/0055.JPG,cut_the_paper_0,480,832,16,1,"A right hand pushes down on the black handle of a paper cutter, slicing a sheet of white paper positioned on the gridded cutting board."
38
+ test_images/0057.JPG,drag_out_wood_chair_0,480,832,16,1,A left hand pulls a light-colored wooden chair away from a dark wooden table.
39
+ test_images/0060.JPG,erasing_the_triangle_from_the_board,480,832,16,1,A right hand reaches for a black eraser resting in a black holder attached to a whiteboard to erase the purple triangle drawn on the board.
40
+ test_images/0061.JPG,flicking_the_light_switch,480,832,16,1,"A right hand reaches toward a white, single-toggle light switch on a beige wall to flick it upward."
41
+ test_images/0062.JPG,flip_the_cover_of_the_book_closed,480,832,16,1,"A right hand flips the black cover of the open spiral notebook over the gridded page, closing the notebook on the round, mirrored table."
42
+ test_images/0063.JPG,flipping_the_red_lever_of_the_machine,480,832,16,1,A person's right hand flips up the red lever on the chrome spigot of the black hot water dispenser.
43
+ test_images/0066.JPG,grab_chair_0,480,832,16,1,A right hand reaches to grab the wooden chair with a cross-back design on the red brick patio.
44
+ test_images/0068.JPG,grab_gasoline_pump_0,480,832,16,1,A right hand reaches to grab the black gas nozzle from its holster on a white and yellow Shell gas pump.
45
+ test_images/0071.JPG,grab_phone_0,480,832,16,1,A person's right hand reaches toward the black handset of a purple public payphone on a sidewalk.
46
+ test_images/0073.JPG,grab_scooter_by_handle_0,480,832,16,1,A right hand reaches to grab the right handlebar of a green and white electric scooter parked on a city sidewalk.
47
+ test_images/0074.JPG,grab_soap_0,480,832,16,1,"A left hand reaches across a white sink to pick up a small, white, leaf-shaped bar of soap resting on the right side."
48
+ test_images/0075.JPG,grab_steel_wool_0,480,832,16,1,A person's right hand reaches over a stainless steel sink toward a blue plastic container holding a steel wool pad.
49
+ test_images/0079.jpg,grab_the_plunger,480,832,16,1,A person's left arm reaches down toward the black plunger in its holder on the hexagonal-tiled floor next to the toilet.
50
+ test_images/0080.jpg,grab_the_weight,480,832,16,1,"A person's left hand reaches to grab the top black, octagonal 'Iron Grip' weight plate from a stack on a yellow Hammer Strength machine."
51
+ test_images/0081.JPG,grab_winshield_cleaner_0,480,832,16,1,"A right hand reaches into a yellow bucket, which is on a green crate next to a white bollard, and grabs a black windshield squeegee."
52
+ test_images/0082.jpg,hold_electrical_vehicle_handle_0,480,832,16,1,"A right hand with dark nail polish on the thumb, extending from a white-cuffed sleeve, reaches to hold the black, textured right handlebar of an electric scooter."
53
+ test_images/0083.jpg,lift_lid_of_computer,480,832,16,1,"A left hand, extending from a grey sweatshirt sleeve, is positioned to lift the lid of a dark grey laptop resting on the canvas seat of a light wood-framed chair."
54
+ test_images/0084.jpg,lift_pan_lid,480,832,16,1,A right hand in a grey sleeve reaches to lift the stainless steel pan lid from the black electric stovetop.
55
+ test_images/0085.jpg,lift_spoon,480,832,16,1,A right hand lifts a silver spoon out of a white bowl of berries and nuts resting on the canvas seat of a wooden chair.
56
+ test_images/0087.JPG,move_pink_mouse_closer_0,480,832,16,1,A hand moves the pink mouse on the white desk closer to the foreground.
57
+ test_images/0088.jpg,open_book_cover,480,832,16,1,"A left hand lifts the yellow cover of the book ""David and Goliath"" by Malcolm Gladwell, which is resting on a fluffy white rug."
58
+ test_images/0089.jpg,open_bottle_cap_0,480,832,16,1,A right hand with dark nail polish reaches toward a white desk to unscrew the cap of an orange plastic bottle of Ito En Hojicha tea.
59
+ test_images/0090.jpg,open_cabinet_0,480,832,16,1,A left hand reaches up and pulls open the left-side white cabinet door located above a white range hood.
60
+ test_images/0092.JPG,open_compost_bin_0,480,832,16,1,A hand reaches from the bottom right of the frame to open the lid of the green compost bin located in the center.
61
+ test_images/0093.JPG,open_dryer_door_0,480,832,16,1,A left hand reaches for the handle on the left side of a white dryer door and pulls it open.
62
+ test_images/0094.jpg,open_earpods_0,480,832,16,1,"A right hand with black nail polish reaches toward a teal, BMO-themed earbud case on a wooden surface to open the lid."
63
+ test_images/0095.JPG,open_fridge_0,480,832,16,1,A hand pulls the black vertical handle on the left side of the white refrigerator's upper door to open it.
64
+ test_images/0096.JPG,open_gate_with_knob_0,480,832,16,1,A right hand reaches for the silver doorknob to open the black metal mesh security gate.
65
+ test_images/0099.jpg,open_oven,480,832,16,1,A person's left hand reaches toward the handle of a stainless steel oven to pull it open.
66
+ test_images/0101.JPG,open_trash_bin_0,480,832,16,1,A right hand reaches toward the white trash bin located under a wooden surface to open it.
67
+ test_images/0102.JPG,open_washingmachine_lid_0,480,832,16,1,"A person's left hand lifts the lid of a white, top-loading washing machine using the indented handle on the front."
68
+ test_images/0103.JPG,open_wood_door_0,480,832,16,1,A person's right hand pushes open the right side of a wooden gate equipped with a black metal latch.
69
+ test_images/0106.jpg,pick_notebook,480,832,16,1,"A left hand reaches toward a round, mirrored table and picks up a dark blue, spiral-bound notebook."
70
+ test_images/0107.JPG,pick_towel_0,480,832,16,1,A right hand reaches toward a crumpled grey towel lying on a light brown wooden surface next to a white pole.
71
+ test_images/0108.JPG,pick_up_cone_0,480,832,16,1,A right hand reaches towards an orange and white traffic cone sitting on a white line in an asphalt parking lot.
72
+ test_images/0109.jpg,pick_up_eraser_0,480,832,16,1,A right hand with dark nail polish reaches for a black rectangular eraser resting on a light-colored wooden surface.
73
+ test_images/0112.JPG,pick_up_the_blue_creamer,480,832,16,1,A right hand reaches across the black speckled countertop to pick up the blue creamer cup from a group of three.
74
+ test_images/0114.jpg,pick_up_the_green_booklet,480,832,16,1,"A person's left hand reaches across a wooden table to pick up the 'Berkeley ENGINEER' magazine, which has a green banner at the top and the title 'Light Science' on its cover."
75
+ test_images/0115.jpg,picking_up_the_skateboard,480,832,16,1,A right hand reaches down to pick up an upside-down electric skateboard from a wooden floor.
76
+ test_images/0116.jpg,place_hand_under_soap_dispenser,480,832,16,1,"A person places their left hand, palm facing up, under the nozzle of a black and clear Purell hand sanitizer dispenser mounted on a black stand."
77
+ test_images/0117.JPG,plugging_in_the_charger_to_the_outlet,480,832,16,1,"A right hand reaches for a white power adapter on a white desk, moving to plug it into the adjacent white power strip."
78
+ test_images/0119.JPG,press_gas_station_button_0,480,832,16,1,"A person's right index finger presses the circular button on the red ""Shell V-Power"" section of a gas pump."
79
+ test_images/0120.JPG,press_red_button_0,480,832,16,1,A person's right hand reaches towards the silver panel on a black emergency call box to press the red button.
80
+ test_images/0122.jpg,pressing_the_button_to_flush_the_toilet,480,832,16,1,"From an overhead view, a person's right hand reaches over the open lid of a white toilet to press the smaller button on the chrome dual-flush mechanism."
81
+ test_images/0123.JPG,pressing_the_number_four_on_the_pad,480,832,16,1,A right index finger presses the number 8 button on the lower physical keypad of a stainless steel elevator control panel.
82
+ test_images/0128.JPG,punching_the_glass,480,832,16,1,"From a first-person perspective, a right hand clenched into a fist moves to punch a glass safety railing overlooking a lower floor with a polished terrazzo surface."
83
+ test_images/0131.JPG,pushing_the_door_open,480,832,16,1,"A hand reaches up to press the horizontal metal panic bar, which has a green indicator light, on a white door."
84
+ test_images/0138.jpg,reach_into_bottle,480,832,16,1,A left hand reaches into the open blue and white bottle of vitamins sitting on a white plastic cart.
85
+ test_images/0140.jpg,scoop_paste_onto_finger,480,832,16,1,A person's right index finger moves towards an open container of yellow petroleum jelly on a white bathroom counter to scoop some of the paste.
86
+ test_images/0144.JPG,staple_papers_0,480,832,16,1,"A right hand presses a black manual stapler onto the corner of a document resting on a dark, speckled countertop."
87
+ test_images/0147.JPG,take_out_blue_book_0,480,832,16,1,"A right hand reaches towards a row of books on a wooden shelf to pull out the blue hardcover book titled ""molecular genetics of bacteria."""
88
+ test_images/0149.jpg,take_out_straw,480,832,16,1,A hand reaches towards a plastic cup on a white desk to pull the clear straw out of the lid.
89
+ test_images/0150.jpg,take_out_the_chair,480,832,16,1,"A hand reaches into a grey, felt-lined nook and pulls a three-legged wooden stool out from the corner."
90
+ test_images/0153.jpg,take_spray_bottle,480,832,16,1,"A person's left hand reaches for a spray bottle in a yellow caddy on a janitorial cart, which also holds a box of black gloves, a roll of paper towels, and other cleaning solutions."
91
+ test_images/0154.JPG,taking_milk_out_of_the_fridge,480,832,16,1,A right hand reaches into the top shelf of an open Frigidaire refrigerator and pulls out a half-full plastic jug of Berkeley milk.
92
+ test_images/0155.JPG,taking_the_lid_off_the_jar,480,832,16,1,"A right hand lifts the lid off a white ceramic jar labeled ""TREATS"" that is sitting on a dark blue countertop."
93
+ test_images/0156.JPG,tear_down_the_poster,480,832,16,1,"A right hand tears down a paper sign with a ""Berkeley Physics Lab"" logo from a glass door."
94
+ test_images/0160.JPG,turn_door_handle_0,480,832,16,1,"A person's right hand reaches towards the silver doorknob on a dirty, white paneled door, which is next to a textured white wall with a double light switch."
95
+ test_images/0161.JPG,turn_fan_dial_0,480,832,16,1,"A person's right hand rests on the black dial of a silver Patton floor fan, which is positioned on a counter in front of shelves stocked with cases of instant noodles."
96
+ test_images/0162.jpg,turn_knob_of_washing_machine,480,832,16,1,"A person's left hand, wearing a grey sweatshirt, turns the large silver control knob on the front of a grey LG washing machine."
97
+ test_images/0164.jpg,turn_the_lock,480,832,16,1,A person's right hand turns the thumb-turn of the brass deadbolt lock on a white door.
98
+ test_images/0165.JPG,turn_wheel_on_pipe_0,480,832,16,1,"A person's hand reaches to turn the black, circular wheel on a large, red industrial valve located in front of a chain-link fence."
99
+ test_images/0170.JPG,twist_off_closure_0,480,832,16,1,"A person's right hand moves to twist the white, ridged cap off a white and red bottle that is lying on the gray fabric of an office chair."
100
+ test_images/0172.JPG,unscrewing_the_lid_on_the_jar,480,832,16,1,"A right hand unscrews the white lid from a large, translucent plastic jar that is sitting on a dark, speckled countertop."
101
+ test_images/0174.JPG,use_key_to_open_door_0,480,832,16,1,"A right hand holding a key with a black head moves it towards the silver deadbolt lock on a white door, positioned above a silver doorknob."
data/ours_test/test_images/0001.jpg ADDED
data/ours_test/test_images/0002.png ADDED

Git LFS Details

  • SHA256: 3dcb690acca3f82f7cf202ebd48cdce9a1e7d8661e8090452167e44bec7a108d
  • Pointer size: 131 Bytes
  • Size of remote file: 719 kB
data/ours_test/test_images/0003.png ADDED

Git LFS Details

  • SHA256: bd1800a952f68419ee254a1aff9ca894709db11e3841707dbc0e9d00fb83a501
  • Pointer size: 131 Bytes
  • Size of remote file: 612 kB
data/ours_test/test_images/0004.jpg ADDED
data/ours_test/test_images/0005.jpg ADDED
data/ours_test/test_images/0006.jpg ADDED
data/ours_test/test_images/0007.jpg ADDED
data/ours_test/test_images/0008.jpg ADDED
data/ours_test/test_images/0009.jpg ADDED
data/ours_test/test_images/0010.jpg ADDED
data/ours_test/test_images/0011.jpg ADDED
data/ours_test/test_images/0012.jpg ADDED
data/ours_test/test_images/0013.jpg ADDED
data/ours_test/test_images/0014.JPG ADDED
data/ours_test/test_images/0015.JPG ADDED
data/ours_test/test_images/0016.jpg ADDED
data/ours_test/test_images/0017.jpg ADDED
data/ours_test/test_images/0018.png ADDED

Git LFS Details

  • SHA256: 12358d1c8d73e01dd83398a0feac50b896637441581810055336dfa7731573e4
  • Pointer size: 131 Bytes
  • Size of remote file: 528 kB
data/ours_test/test_images/0019.png ADDED

Git LFS Details

  • SHA256: b23838c2735ca87fcfe8ed0698d70e4bd61f1fc476018372b084f5c960ee8d19
  • Pointer size: 131 Bytes
  • Size of remote file: 549 kB
data/ours_test/test_images/0020.jpg ADDED
data/ours_test/test_images/0021.jpg ADDED
data/ours_test/test_images/0022.jpg ADDED
data/ours_test/test_images/0023.jpg ADDED
data/ours_test/test_images/0024.jpg ADDED
data/ours_test/test_images/0025.jpg ADDED
data/ours_test/test_images/0026.jpg ADDED
data/ours_test/test_images/0027.jpg ADDED
data/ours_test/test_images/0028.jpg ADDED
data/ours_test/test_images/0029.jpg ADDED
data/ours_test/test_images/0030.jpg ADDED
data/ours_test/test_images/0031.jpg ADDED
data/ours_test/test_images/0032.jpg ADDED
data/ours_test/test_images/0033.jpg ADDED
data/ours_test/test_images/0034.jpg ADDED
data/ours_test/test_images/0035.jpg ADDED