Open the book and turn off the desk lamp

Instruction: Please help me with this task in the scene: open the book and turn off the desk lamp.

{"steps": 9, "actions": ["Move(forward, 1.0)", "Manipulate(Book, open)", "Move(backward, 0.5)", "Rotate(right, 90)", "Rotate(right, 90)", "Move(forward, 0.5)", "Move(forward, 0.25)", "ChangeState(DeskLamp, off)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "Book", "field": "isOpen", "value": true}, {"type": "object_state", "object_type": "DeskLamp", "field": "isToggled", "value": false}]


Instruction: I'm checking a couple interactions, so please open the book gently so it stays open and then in the simulation, break the window using the provided interaction.

{"steps": 6, "actions": ["Move(forward, 0.25)", "Move(left, 0.5)", "Move(forward, 1.0)", "Manipulate(Book, open)", "Manipulate(Window, break)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "Book", "field": "isOpen", "value": true}, {"type": "object_state", "object_type": "Window", "field": "isBroken", "value": true}]


Pick up the phone and place it on the bed

Instruction: I want this put away neatly, so please pick up the phone carefully and hold it securely and then place it on the bed and make sure it sits flat and won't slide.

{"steps": 8, "actions": ["Move(left, 0.5)", "Move(forward, 1.0)", "Move(forward, 0.25)", "Pick(CellPhone)", "Rotate(right, 90)", "Move(forward, 0.25)", "Place(Bed)", "EndTask(success)"]}

[{"type": "object_in_receptacle", "object_type": "CellPhone", "receptacle_type": "Bed", "value": true}]


Pick up the pen and place it on the shelf

Instruction: I want this put away neatly, so please pick up the pen carefully and hold it securely and then place it on the shelf and make sure it sits flat and won't slide.

{"steps": 14, "actions": ["Rotate(right, 90)", "Move(forward, 0.25)", "Pick(Pen)", "Rotate(left, 90)", "Rotate(left, 90)", "Move(forward, 1.0)", "Move(forward, 0.25)", "Rotate(right, 90)", "Move(forward, 0.25)", "Move(left, 0.5)", "Move(left, 0.25)", "Move(forward, 0.5)", "Place(Shelf)", "EndTask(success)"]}

[{"type": "object_in_receptacle", "object_type": "Pen", "receptacle_type": "Shelf", "value": true}]


Pick up the credit card and place it on the side table

Instruction: I want this put away neatly, so please pick up the credit card carefully and hold it securely and then place it on the side table and make sure it sits flat and won't slide.

{"steps": 12, "actions": ["Rotate(right, 90)", "Move(forward, 0.5)", "Move(forward, 0.25)", "Pick(CreditCard)", "Rotate(right, 90)", "Rotate(right, 90)", "Move(forward, 0.5)", "Move(forward, 0.25)", "Move(right, 1.0)", "Move(forward, 0.25)", "Place(SideTable)", "EndTask(success)"]}

[{"type": "object_in_receptacle", "object_type": "CreditCard", "receptacle_type": "SideTable", "value": true}]


Turn off the desk lamp and pick up the book and place it on the bed

Instruction: It's time to power down, so please turn off the desk lamp and confirm it's really off and then pick up the book and place it on the bed.

{"steps": 11, "actions": ["Move(forward, 0.5)", "Move(forward, 0.25)", "ChangeState(DeskLamp, off)", "Pick(Book)", "Rotate(right, 90)", "Rotate(right, 90)", "Move(forward, 1.0)", "Move(forward, 0.5)", "Move(forward, 0.25)", "Place(Bed)", "EndTask(success)"]}

[{"type": "object_in_receptacle", "object_type": "Book", "receptacle_type": "Bed", "value": true}, {"type": "object_state", "object_type": "DeskLamp", "field": "isToggled", "value": false}]


Open the book and pick up the cell phone and turn it on

Instruction: I want to read and check my phone, so please open the book gently so it stays open and then pick up the cell phone carefully and hold it securely and turn on the cell phone and wait until you can tell it's powered on.

{"steps": 8, "actions": ["Rotate(left, 90)", "Move(forward, 0.5)", "Move(forward, 0.25)", "Manipulate(Book, open)", "Move(backward, 0.25)", "Pick(CellPhone)", "ChangeState(CellPhone, on)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "Book", "field": "isOpen", "value": true}, {"type": "object_state", "object_type": "CellPhone", "field": "isToggled", "value": true}, {"type": "object_state", "object_type": "CellPhone", "field": "isPickedUp", "value": true}]


Instruction: Could you help me out by picking up the cellphone, turning it on carefully, and holding it securely?

{"steps": 10, "actions": ["Move(forward, 0.25)", "Move(right, 0.25)", "Move(forward, 0.5)", "Move(forward, 0.25)", "Move(right, 0.25)", "Move(forward, 0.5)", "Rotate(left, 90)", "Pick(CellPhone)", "ChangeState(CellPhone, on)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "CellPhone", "field": "isPickedUp", "value": true}, {"type": "object_state", "object_type": "CellPhone", "field": "isToggled", "value": true}]


Instruction: I need it running, so please turn on the laptop and wait until you can tell it's powered on.

{"steps": 6, "actions": ["Move(right, 0.5)", "Move(forward, 1.0)", "Move(forward, 0.5)", "Rotate(left, 90)", "ChangeState(Laptop, on)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "Laptop", "field": "isToggled", "value": true}]


Instruction: It's not needed right now, so please turn off the desk lamp on the desk and confirm it's really off.

{"steps": 9, "actions": ["Move(forward, 0.5)", "Move(forward, 0.25)", "Move(right, 0.5)", "Move(right, 0.25)", "Move(forward, 1.0)", "Move(forward, 1.0)", "Move(right, 0.5)", "ChangeState(DeskLamp, off)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "DeskLamp", "field": "isToggled", "value": false}]


Instruction: I'm putting things where they belong, so please pick up the book, carry it over, and set it down on the bed neatly.

{"steps": 17, "actions": ["Move(forward, 0.25)", "Move(right, 0.5)", "Move(forward, 1.0)", "Move(forward, 1.0)", "Move(forward, 0.5)", "Move(right, 0.25)", "Move(forward, 0.5)", "Move(forward, 0.25)", "Rotate(right, 90)", "Move(left, 0.5)", "Move(forward, 0.25)", "Pick(Book)", "Rotate(right, 90)", "Rotate(right, 90)", "Move(forward, 0.5)", "Place(Bed)", "EndTask(success)"]}

[{"type": "object_in_receptacle", "object_type": "Book", "receptacle_type": "Bed", "value": true}, {"type": "object_in_receptacle", "object_type": "Book", "receptacle_type": "Desk", "value": false}]


Instruction: I need it running, so please turn on the laptop and wait until you can tell it's powered on.

{"steps": 9, "actions": ["Move(right, 0.5)", "Move(forward, 1.0)", "Move(forward, 1.0)", "Move(forward, 0.5)", "Move(forward, 0.25)", "Move(right, 0.5)", "Rotate(right, 90)", "ChangeState(Laptop, on)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "Laptop", "field": "isToggled", "value": true}]


Instruction: Could you help me out by picking up the pencil carefully and holding it securely?

{"steps": 13, "actions": ["Move(forward, 0.25)", "Move(right, 0.5)", "Move(forward, 0.25)", "Move(right, 0.25)", "Move(forward, 1.0)", "Move(forward, 1.0)", "Move(forward, 0.5)", "Move(forward, 0.25)", "Rotate(right, 90)", "Move(left, 0.5)", "Move(forward, 0.25)", "Pick(Pencil)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "Pencil", "field": "isPickedUp", "value": true}]


Instruction: Could you help me out by picking up the pen carefully and holding it securely?

{"steps": 13, "actions": ["Move(forward, 0.25)", "Move(right, 0.5)", "Move(forward, 0.25)", "Move(right, 0.25)", "Move(forward, 1.0)", "Move(forward, 1.0)", "Move(forward, 0.5)", "Move(forward, 0.25)", "Rotate(right, 90)", "Move(left, 0.5)", "Move(forward, 0.25)", "Pick(Pen)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "Pen", "field": "isPickedUp", "value": true}]


Instruction: I'm putting things where they belong, so please pick up the alarm clock, carry it over, and set it down on the desk neatly.

{"steps": 15, "actions": ["Rotate(right, 90)", "Move(forward, 0.5)", "Pick(AlarmClock)", "Move(left, 0.25)", "Move(forward, 0.25)", "Rotate(left, 90)", "Move(forward, 1.0)", "Move(forward, 1.0)", "Move(forward, 1.0)", "Move(forward, 0.5)", "Rotate(right, 90)", "Move(forward, 0.25)", "Move(left, 0.25)", "Place(Desk)", "EndTask(success)"]}

[{"type": "object_in_receptacle", "object_type": "AlarmClock", "receptacle_type": "Desk", "value": true}, {"type": "object_in_receptacle", "object_type": "AlarmClock", "receptacle_type": "SideTable", "value": false}]


Instruction: I need it running, so please turn on the TV and wait until you can tell it's powered on.

{"steps": 9, "actions": ["Rotate(right, 90)", "Move(left, 0.25)", "Move(forward, 0.25)", "Move(left, 0.5)", "Move(forward, 0.5)", "Move(left, 0.25)", "Move(forward, 0.25)", "ChangeState(Television, on)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "Television", "field": "isToggled", "value": true}]


Instruction: I want to tidy up, so please pick up the remote control carefully and hold it securely and then put it into the drawer.

{"steps": 10, "actions": ["Rotate(right, 90)", "Move(left, 0.25)", "Move(forward, 0.5)", "Move(left, 0.5)", "Move(left, 0.25)", "Move(forward, 0.25)", "Manipulate(Drawer, open)", "Pick(RemoteControl)", "Place(Drawer)", "EndTask(success)"]}

[{"type": "object_in_receptacle", "object_type": "RemoteControl", "receptacle_type": "Drawer", "value": true}]


Instruction: I want to tidy up, so please pick up the pencil carefully and hold it securely and then put it into the drawer.

{"steps": 17, "actions": ["Move(forward, 0.5)", "Move(forward, 0.25)", "Move(right, 0.5)", "Move(forward, 1.0)", "Move(forward, 1.0)", "Move(forward, 0.5)", "Move(forward, 0.25)", "Move(right, 0.25)", "Rotate(right, 90)", "Pick(Pencil)", "Rotate(right, 90)", "Move(forward, 1.0)", "Rotate(left, 90)", "Move(forward, 0.25)", "Manipulate(Drawer, open)", "Place(Drawer)", "EndTask(success)"]}

[{"type": "object_in_receptacle", "object_type": "Pencil", "receptacle_type": "Drawer", "value": true}]


Instruction: I want to tidy up, so please pick up the pen carefully and hold it securely and then put it into the drawer.

{"steps": 17, "actions": ["Move(forward, 0.5)", "Move(forward, 0.25)", "Move(right, 0.5)", "Move(forward, 1.0)", "Move(forward, 1.0)", "Move(forward, 0.5)", "Move(forward, 0.25)", "Move(right, 0.25)", "Rotate(right, 90)", "Pick(Pen)", "Rotate(right, 90)", "Move(forward, 1.0)", "Rotate(left, 90)", "Move(forward, 0.25)", "Manipulate(Drawer, open)", "Place(Drawer)", "EndTask(success)"]}

[{"type": "object_in_receptacle", "object_type": "Pen", "receptacle_type": "Drawer", "value": true}]


Instruction: I'm putting things where they belong, so please pick up the phone, carry it over, and set it down on the bed neatly.

{"steps": 7, "actions": ["Rotate(right, 90)", "Move(forward, 1.0)", "Pick(CellPhone)", "Rotate(right, 90)", "Move(forward, 1.0)", "Place(Bed)", "EndTask(success)"]}

[{"type": "object_in_receptacle", "object_type": "CellPhone", "receptacle_type": "Bed", "value": true}, {"type": "object_in_receptacle", "object_type": "CellPhone", "receptacle_type": "Desk", "value": false}]


Instruction: I need to tidy up, so please tidy up the bed by clearing any clutter and wiping it until it looks clean and then turn on the laptop and wait until you can tell it's powered on.

{"steps": 10, "actions": ["Rotate(right, 90)", "Rotate(right, 90)", "Move(forward, 0.5)", "Move(forward, 0.25)", "Move(left, 0.25)", "Move(forward, 0.25)", "Manipulate(Bed, clean)", "Move(forward, 0.25)", "ChangeState(Laptop, on)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "Laptop", "field": "isToggled", "value": true}, {"type": "object_state", "object_type": "Bed", "field": "isDirty", "value": false}]


Clean the bed and turn off the desk lamp

Instruction: I'm cleaning up before I relax, so please tidy up the bed and wiping it until it looks clean, then turn off the desk lamp and confirm it's really off.

{"steps": 10, "actions": ["Rotate(right, 90)", "Move(right, 0.5)", "Move(right, 0.25)", "Move(forward, 1.0)", "Move(forward, 0.25)", "Rotate(right, 90)", "ChangeState(DeskLamp, off)", "Move(forward, 0.25)", "Manipulate(Bed, clean)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "Bed", "field": "isDirty", "value": false}, {"type": "object_state", "object_type": "DeskLamp", "field": "isToggled", "value": false}]


Move the book to the bed and clean the bed

Instruction: I'm tidying up, so please pick up the book and place it on the nightstand, then tidy up the bed and wiping it until it looks clean.

{"steps": 8, "actions": ["Rotate(right, 90)", "Move(forward, 0.5)", "Pick(Book)", "Rotate(right, 90)", "Move(forward, 1.0)", "Manipulate(Bed, clean)", "Place(Bed)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "Bed", "field": "isDirty", "value": false}, {"type": "object_in_receptacle", "object_type": "Book", "receptacle_type": "Bed", "value": true}]


Fill the mug with water and clean the bed

Instruction: I'm taking care of a couple things, so please fill the mug with water until it shows it's filled, then keep it steady and then tidy up the bed and wiping it until it looks clean.

{"steps": 11, "actions": ["Rotate(right, 90)", "Rotate(right, 90)", "Move(forward, 0.5)", "Move(forward, 0.25)", "Manipulate(Bed, clean)", "Rotate(left, 90)", "Move(left, 0.5)", "Move(left, 0.25)", "Move(forward, 0.25)", "Manipulate(Mug,water, fillwithliquid)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "Bed", "field": "isDirty", "value": false}, {"type": "object_state", "object_type": "Mug", "field": "isFilledWithLiquid", "value": true, "liquid": "water"}]


Fill the mug with coffee and clean the bed

Instruction: I'm doing some chores, so please put the laptop on the desk, then tidy up the bed and wiping it until it looks clean.

{"steps": 11, "actions": ["Rotate(right, 90)", "Rotate(right, 90)", "Move(forward, 0.5)", "Move(forward, 0.25)", "Manipulate(Bed, clean)", "Rotate(left, 90)", "Move(left, 0.5)", "Move(left, 0.25)", "Move(forward, 0.25)", "Manipulate(Mug, coffee, fillwithliquid)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "Bed", "field": "isDirty", "value": false}, {"type": "object_state", "object_type": "Mug", "field": "isFilledWithLiquid", "value": true, "liquid": "coffee"}]


Instruction: I'm putting things where they belong, so please pick up the cell phone, carry it over, and set it down on the bed neatly.

{"steps": 8, "actions": ["Rotate(right, 90)", "Move(forward, 0.25)", "Pick(CellPhone)", "Rotate(right, 90)", "Move(forward, 0.5)", "Move(forward, 0.25)", "Place(Bed)", "EndTask(success)"]}

[{"type": "object_in_receptacle", "object_type": "CellPhone", "receptacle_type": "Bed", "value": true}, {"type": "object_in_receptacle", "object_type": "CellPhone", "receptacle_type": "Desk", "value": false}]


Instruction: Please help me with this task in the scene: toggle the laptop on and clean the bed.

{"steps": 12, "actions": ["Rotate(right, 90)", "Rotate(right, 90)", "Move(forward, 0.5)", "ChangeState(Laptop, on)", "Rotate(right, 90)", "Move(right, 0.5)", "Move(forward, 1.0)", "Move(forward, 0.5)", "Move(forward, 0.25)", "Rotate(left, 90)", "Manipulate(Bed, clean)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "Bed", "field": "isDirty", "value": false}, {"type": "object_state", "object_type": "Laptop", "field": "isToggled", "value": true}]


Instruction: I'm putting things where they belong, so please pick up the alarm clock, carry it over, and set it down on the desk neatly.

{"steps": 7, "actions": ["Move(forward, 0.5)", "Pick(AlarmClock)", "Rotate(right, 90)", "Rotate(right, 90)", "Move(forward, 1.0)", "Place(Desk)", "EndTask(success)"]}

[{"type": "object_in_receptacle", "object_type": "AlarmClock", "receptacle_type": "Desk", "value": true}, {"type": "object_in_receptacle", "object_type": "AlarmClock", "receptacle_type": "SideTable", "value": false}]


Instruction: I'm putting things where they belong, so please pick up the book, carry it over, and set it down on the desk neatly.

{"steps": 7, "actions": ["Move(forward, 0.5)", "Pick(Book)", "Rotate(right, 90)", "Rotate(right, 90)", "Move(forward, 1.0)", "Place(Desk)", "EndTask(success)"]}

[{"type": "object_in_receptacle", "object_type": "Book", "receptacle_type": "Desk", "value": true}, {"type": "object_in_receptacle", "object_type": "Book", "receptacle_type": "SideTable", "value": false}]


Instruction: I'm putting things where they belong, so please pick up the alarm clock, carry it over, and set it down on the desk neatly.

{"steps": 15, "actions": ["Rotate(right, 90)", "Rotate(right, 90)", "Move(left, 1.0)", "Move(forward, 1.0)", "Move(forward, 0.5)", "Move(forward, 0.25)", "Pick(AlarmClock)", "Rotate(left, 90)", "Rotate(left, 90)", "Move(forward, 1.0)", "Move(forward, 0.5)", "Rotate(right, 90)", "Move(left, 0.5)", "Place(Desk)", "EndTask(success)"]}

[{"type": "object_in_receptacle", "object_type": "AlarmClock", "receptacle_type": "Desk", "value": true}, {"type": "object_in_receptacle", "object_type": "AlarmClock", "receptacle_type": "Dresser", "value": false}]


Break the mirror and fill the mug with water

Instruction: I'm running a quick test, so please in the simulation, break the mirror using the provided interaction and then fill the mug with water until it shows it's filled, then keep it steady.

{"steps": 7, "actions": ["Move(right, 0.5)", "Move(forward, 0.25)", "Manipulate(Mirror, break)", "Rotate(right, 90)", "Move(forward, 0.5)", "Manipulate(Mug,water, fillwithliquid)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "Mirror", "field": "isBroken", "value": true}, {"type": "object_state", "object_type": "Mug", "state": "isFilledWithLiquid", "value": true, "liquid": "Water"}]


Make the mirror dirty and fill the mug with water

Instruction: I'm preparing the scene, so please in the simulation, make the mirror dirty so its state changes to 'dirty' and then fill the mug with water until it shows it's filled, then keep it steady.

{"steps": 7, "actions": ["Move(right, 0.5)", "Move(forward, 0.25)", "DirtyObject(Mirror)", "Rotate(right, 90)", "Move(forward, 0.5)", "Manipulate(Mug, fillwithliquid)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "Mirror", "field": "isDirty", "value": true}, {"type": "object_state", "object_type": "Mug", "state": "isFilledWithLiquid", "value": true, "liquid": "Water"}]


Turn on the laptop and turn on the cell phone

Instruction: I need some help with my electronics. Please turn on the laptop first. Once it's fully booted, power on the cell phone and make sure they both stay on.

{"steps": 10, "actions": ["Rotate(right, 90)", "Move(forward, 1.0)", "ChangeState(Laptop, on)", "Rotate(right, 90)", "Move(forward, 1.0)", "Move(forward, 0.25)", "Move(left, 0.5)", "Move(forward, 0.5)", "ChangeState(CellPhone, on)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "Laptop", "field": "isToggled", "value": true}, {"type": "object_state", "object_type": "CellPhone", "field": "isToggled", "value": true}]


Break the laptop and pick up the cell phone

Instruction: I'm running a quick test, so please in the simulation, break the laptop using the provided interaction and then pick up the cell phone carefully and hold it securely.

{"steps": 10, "actions": ["Rotate(right, 90)", "Move(forward, 1.0)", "Manipulate(Laptop, break)", "Rotate(right, 90)", "Move(forward, 1.0)", "Move(forward, 0.25)", "Move(left, 0.5)", "Move(forward, 0.5)", "Pick(CellPhone)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "Laptop", "field": "isBroken", "value": true}, {"type": "object_state", "object_type": "CellPhone", "field": "isPickedUp", "value": true}]


Pick up the laptop and break the cell phone

Instruction: I'm testing interactions, so please pick up the laptop carefully and hold it securely and then in the simulation, break the cell phone using the provided interaction.

{"steps": 10, "actions": ["Rotate(right, 90)", "Move(forward, 1.0)", "Pick(Laptop)", "Rotate(right, 90)", "Move(forward, 1.0)", "Move(forward, 0.25)", "Move(left, 0.5)", "Move(forward, 0.5)", "Manipulate(CellPhone, break)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "Laptop", "field": "isPickedUp", "value": true}, {"type": "object_state", "object_type": "CellPhone", "field": "isBroken", "value": true}]


Break the laptop and break the cell phone

Instruction: I'm testing the scene interactions, so please in the simulation, break the laptop using the provided interaction and then in the simulation, break the cell phone using the provided interaction.

{"steps": 10, "actions": ["Rotate(right, 90)", "Move(forward, 1.0)", "Manipulate(Laptop, break)", "Rotate(right, 90)", "Move(forward, 1.0)", "Move(forward, 0.25)", "Move(left, 0.5)", "Move(forward, 0.5)", "Manipulate(CellPhone, break)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "Laptop", "field": "isBroken", "value": true}, {"type": "object_state", "object_type": "CellPhone", "field": "isBroken", "value": true}]


Instruction: I'm putting things where they belong, so please pick up the tabletop decor, carry it over, and set it down on the dresser neatly.

{"steps": 9, "actions": ["Rotate(right, 90)", "Move(forward, 1.0)", "Pick(TableTopDecor)", "Rotate(right, 90)", "Move(forward, 1.0)", "Move(forward, 0.5)", "Move(forward, 0.25)", "Place(Dresser)", "EndTask(success)"]}

[{"type": "object_in_receptacle", "object_type": "TableTopDecor", "receptacle_type": "Dresser", "value": true}, {"type": "object_in_receptacle", "object_type": "TableTopDecor", "receptacle_type": "Desk", "value": false}]


Instruction: I'm putting things where they belong, so please pick up the keychain, carry it over, and set it down on the dresser neatly.

{"steps": 11, "actions": ["Rotate(right, 90)", "Move(forward, 1.0)", "Pick(KeyChain)", "Rotate(right, 90)", "Move(forward, 1.0)", "Move(forward, 0.5)", "Move(forward, 0.25)", "Move(left, 0.5)", "Move(forward, 0.25)", "Place(Dresser)", "EndTask(success)"]}

[{"type": "object_in_receptacle", "object_type": "KeyChain", "receptacle_type": "Dresser", "value": true}, {"type": "object_in_receptacle", "object_type": "KeyChain", "receptacle_type": "Desk", "value": false}]


Instruction: I'm putting things where they belong, so please pick up the credit card, carry it over, and set it down on the dresser neatly.

{"steps": 11, "actions": ["Rotate(right, 90)", "Move(forward, 1.0)", "Pick(CreditCard)", "Rotate(right, 90)", "Move(forward, 1.0)", "Move(forward, 0.5)", "Move(forward, 0.25)", "Move(left, 0.5)", "Move(forward, 0.25)", "Place(Dresser)", "EndTask(success)"]}

[{"type": "object_in_receptacle", "object_type": "CreditCard", "receptacle_type": "Dresser", "value": true}, {"type": "object_in_receptacle", "object_type": "CreditCard", "receptacle_type": "Desk", "value": false}]


Turn on the cell phone and open the book

Instruction: To start the research, please power on the cell phone and open the book for me.

{"steps": 7, "actions": ["Move(left, 0.5)", "Move(left, 0.25)", "Move(forward, 0.5)", "Move(forward, 0.25)", "ChangeState(CellPhone, on)", "Manipulate(Book, open)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "CellPhone", "field": "isToggled", "value": true}, {"type": "object_state", "object_type": "Book", "field": "isOpen", "value": true}]

["Bed", "Book", "CellPhone", "TeddyBear"]


Instruction: I’m leaving the room now, so please turn off the light switch.

{"steps": 6, "actions": ["Rotate(left, 90)", "Move(forward, 1.0)", "Move(forward, 1.0)", "Move(forward, 0.5)", "ChangeState(LightSwitch, off)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "LightSwitch", "state": "isToggled", "value": false}]


Instruction: I want the space neat, so please tidy up the bed and wiping it until it looks clean.

{"steps": 7, "actions": ["Rotate(right, 90)", "Rotate(right, 90)", "Move(left, 0.5)", "Move(forward, 0.5)", "Move(forward, 0.25)", "Manipulate(Bed, clean)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "Bed", "field": "isDirty", "value": false}]


Instruction: It looks like it needs liquid, so please fill the houseplant with water until it shows it's filled, then keep it steady.

{"steps": 7, "actions": ["Rotate(right, 90)", "Move(forward, 0.5)", "Move(right, 0.5)", "Move(forward, 0.5)", "Move(forward, 0.25)", "Manipulate(HousePlant,water, fillwithliquid)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "HousePlant", "state": "isFilledWithLiquid", "value": true, "liquid": "water"}]


Instruction: It looks like it needs liquid, so please fill the bowl with water until it shows it's filled, then keep it steady.

{"steps": 7, "actions": ["Rotate(right, 90)", "Move(forward, 0.5)", "Move(right, 0.5)", "Move(forward, 0.5)", "Move(forward, 0.25)", "Manipulate(Bowl,water, fillwithliquid)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "Bowl", "state": "isFilledWithLiquid", "value": true, "liquid": "water"}]


Instruction: I'm preparing the scene, so please in the simulation, make the bowl dirty so its state changes to 'dirty'.

{"steps": 7, "actions": ["Rotate(right, 90)", "Rotate(right, 90)", "Rotate(left, 90)", "Move(right, 0.25)", "Move(forward, 1.0)", "DirtyObject(Bowl)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "Bowl", "state": "isDirty", "value": true}]


Pick the baseballbat and break the mirror

Instruction: Please help me with this task in the scene: pick the baseballbat and break the mirror.

{"steps": 13, "actions": ["Move(left, 0.25)", "Move(forward, 0.25)", "Pick(BaseballBat)", "Rotate(right, 90)", "Rotate(right, 90)", "Move(forward, 0.5)", "Move(forward, 0.25)", "Rotate(left, 90)", "Move(forward, 0.5)", "Rotate(left, 90)", "Move(forward, 0.25)", "Manipulate(Mirror, break)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "Mirror", "state": "isBroken", "value": true}, {"type": "object_state", "object_type": "BaseballBat", "state": "isPickedUp", "value": true}]


Instruction: I need to start my remote meeting, so please toggle the laptop on.

{"steps": 5, "actions": ["Rotate(left, 90)", "Move(left, 1.0)", "Move(forward, 0.25)", "ChangeState(Laptop, on)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "Laptop", "state": "isToggled", "value": true}]


Instruction: It's time for my daily exercise; can you pick up the dumbbell from the floor.

{"steps": 6, "actions": ["Rotate(right, 90)", "Rotate(right, 90)", "Move(forward, 1.0)", "Move(forward, 0.25)", "Pick(Dumbbell)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "Dumbbell", "state": "isPickedUp", "value": true}]


Turn off the desk lamp and light switch.

Instruction: It's time to power down, so please turn off the desk lamp and confirm it's really off and then turn off the light switch and confirm it's really off.

{"steps": 7, "actions": ["ChangeState(DeskLamp, off)", "Rotate(left, 90)", "Move(forward, 0.5)", "Move(forward, 0.25)", "Rotate(right, 90)", "ChangeState(LightSwitch, off)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "DeskLamp", "state": "isToggled", "value": false}, {"type": "object_state", "object_type": "LightSwitch", "state": "isToggled", "value": false}]


Instruction: Please help me with this task in the scene: break the vase.

{"steps": 7, "actions": ["Rotate(right, 90)", "Move(forward, 1.0)", "Move(forward, 1.0)", "Move(forward, 0.5)", "Move(forward, 0.25)", "Manipulate(Vase, break)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "Vase", "state": "isBroken", "value": true}]


Instruction: I want it switched on, so please toggle the TV to the on position and check the room responds.

{"steps": 11, "actions": ["Rotate(right, 90)", "Move(forward, 1.0)", "Move(forward, 1.0)", "Move(forward, 0.25)", "Rotate(right, 90)", "Move(forward, 0.5)", "Move(left, 0.25)", "Move(forward, 0.5)", "Move(forward, 0.25)", "ChangeState(Television, on)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "Television", "state": "isToggled", "value": true}]


Toggle off the light switch near the door

Instruction: I'm leaving the room, please toggle off the light switch near the door to save power.

{"steps": 14, "actions": ["Rotate(right, 90)", "Move(forward, 1.0)", "Move(forward, 0.5)", "Rotate(left, 90)", "Move(forward, 0.25)", "Move(right, 0.5)", "Move(right, 0.25)", "Move(forward, 1.0)", "Move(right, 0.5)", "Move(forward, 1.0)", "Move(forward, 1.0)", "Move(forward, 0.5)", "ChangeState(LightSwitch, off)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "LightSwitch", "state": "isToggled", "value": false}]


Instruction: These old plaster models are taking up too much space. break one of the statues so we can fit it in the trash bin.

{"steps": 7, "actions": ["Move(forward, 1.0)", "Move(forward, 1.0)", "Move(forward, 0.25)", "Rotate(left, 90)", "Move(forward, 0.25)", "Manipulate(Statue, break)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "Statue", "state": "isBroken", "value": true}]


Instruction: I want to play some games. Please switch on the laptop and the television.

{"steps": 6, "actions": ["Rotate(right, 90)", "Rotate(right, 90)", "Move(right, 0.5)", "ChangeState(Laptop, on)", "ChangeState(Television, on)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "Laptop", "state": "isToggled", "value": true}, {"type": "object_state", "object_type": "Television", "state": "isToggled", "value": true}]


Instruction: I want things put away, so please close the box on the table all the way until it looks fully shut.

{"steps": 5, "actions": ["Rotate(right, 90)", "Rotate(right, 90)", "Move(forward, 1.0)", "Manipulate(Box, close)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "Box", "state": "isOpen", "value": false}]


Instruction: I'm going to take a nap on the couch, so switch off the desk lamp right next to it.

{"steps": 8, "actions": ["Move(left, 1.0)", "Move(left, 0.25)", "Move(forward, 1.0)", "Move(forward, 0.25)", "Move(left, 0.5)", "Move(forward, 0.5)", "ChangeState(DeskLamp, off)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "DeskLamp", "state": "isToggled", "value": false}]


Instruction: I'm putting things where they belong, so please pick up the box, carry it over, and set it down on the dining table neatly.

{"steps": 16, "actions": ["Rotate(left, 90)", "Move(forward, 0.5)", "Move(forward, 0.25)", "Move(right, 0.25)", "Move(forward, 0.25)", "Pick(Box)", "Rotate(right, 90)", "Move(right, 1.0)", "Move(right, 0.5)", "Move(right, 0.25)", "Move(forward, 1.0)", "Move(forward, 1.0)", "Move(forward, 1.0)", "Move(forward, 0.5)", "Place(DiningTable)", "EndTask(success)"]}

[{"type": "object_in_receptacle", "object_type": "Box", "receptacle_type": "DiningTable", "value": true}, {"type": "object_in_receptacle", "object_type": "Box", "receptacle_type": "CoffeeTable", "value": false}]


Instruction: We need to test the alarm system; break any window to see if the sensors trigger correctly.

{"steps": 19, "actions": ["Rotate(left, 90)", "Move(forward, 1.0)", "Move(left, 0.5)", "Move(forward, 0.5)", "Move(forward, 0.25)", "Move(right, 0.25)", "Move(forward, 0.5)", "Move(right, 0.25)", "Move(forward, 0.25)", "Move(right, 0.25)", "Move(forward, 0.25)", "Move(right, 0.25)", "Move(forward, 0.25)", "Move(right, 0.25)", "Move(forward, 0.25)", "Move(right, 0.25)", "Move(forward, 0.25)", "Manipulate(Window, break)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "Window", "state": "isBroken", "value": true}]


Instruction: We are under a cyberattack. Power on the TV for news and break the laptop immediately so no one can access the data.

{"steps": 12, "actions": ["Rotate(right, 90)", "Rotate(right, 90)", "Move(forward, 0.5)", "Move(forward, 0.25)", "ChangeState(Television, on)", "Rotate(right, 90)", "Rotate(right, 90)", "Move(forward, 1.0)", "Move(forward, 1.0)", "Move(forward, 0.25)", "Manipulate(Laptop, break)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "Television", "state": "isToggled", "value": true}, {"type": "object_state", "object_type": "Laptop", "state": "isBroken", "value": true}]


Instruction: I'm settling in on the sofa, so please turn on the laptop on the sofa and wait until you can tell it's powered on.

{"steps": 7, "actions": ["Rotate(right, 90)", "Move(left, 0.5)", "Move(left, 0.25)", "Move(forward, 1.0)", "Move(forward, 0.5)", "ChangeState(Laptop, on)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "Laptop", "state": "isToggled", "value": true}]


Instruction: My hands are full of stationery; please open one of the drawers so i can put these away.

{"steps": 6, "actions": ["Move(left, 1.0)", "Move(forward, 0.5)", "Move(forward, 0.25)", "Manipulate(Drawer, open)", "Move(backward, 0.5)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "Drawer", "state": "isOpen", "value": true}]


Instruction: The pot looks a bit dusty, so clean the pot and slice the tomato for the soup.

{"steps": 6, "actions": ["Manipulate(Pot, clean)", "Rotate(right, 90)", "Move(forward, 0.5)", "Move(forward, 0.25)", "Manipulate(Tomato, slice)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "Pot", "state": "isDirty", "value": false}, {"type": "object_state", "object_type": "Tomato", "state": "isSliced", "value": true}]


Fill the pot with liquid and slice the tomato

Instruction: I’m planning to make soup, so please fill the pot with water and slice the tomato for me.

{"steps": 8, "actions": ["Rotate(right, 90)", "Move(forward, 0.5)", "Move(forward, 0.25)", "Manipulate(Pot, water, fillwithliquid)", "Rotate(right, 90)", "Move(forward, 1.0)", "Manipulate(Tomato, slice)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "Pot", "state": "isFilledWithLiquid", "value": true, "liquid": "water"}, {"type": "object_state", "object_type": "Tomato", "state": "isSliced", "value": true}]


Instruction: I'm done with it, so please empty the wine bottle in the simulation and then break it using the provided interaction.

{"steps": 6, "actions": ["Rotate(right, 90)", "Move(forward, 1.0)", "Move(right, 1.0)", "Manipulate(WineBottle, emptyliquidfrom)", "Manipulate(WineBottle, break)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "WineBottle", "state": "isFilledWithLiquid", "value": false}, {"type": "object_state", "object_type": "WineBottle", "state": "isBroken", "value": true}]


Instruction: I’m ready to start the stew, so please pick up the pot for me.

{"steps": 5, "actions": ["Rotate(right, 90)", "Move(forward, 0.5)", "Move(forward, 0.25)", "Pick(Pot)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "Pot", "state": "isPickedUp", "value": true}]

I’m ready to start the stew, so please pick up the pot and slice the tomato for me.


Instruction: I need to fry an egg; could you turn on one of the stove knobs for me?

{"steps": 6, "actions": ["Rotate(right, 90)", "Move(forward, 0.5)", "Move(forward, 0.25)", "Move(left, 0.25)", "ChangeState(StoveKnob, on)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "StoveKnob", "state": "isToggled", "value": true}]


Instruction: I'm hungry, so please slice the bread as instructed and keep the pieces together and then pick up one piece.

{"steps": 5, "actions": ["Move(forward, 0.5)", "Move(forward, 0.25)", "Manipulate(Bread, slice)", "Pick(Bread)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "Bread", "state": "isSliced", "value": true}, {"type": "object_state", "object_type": "BreadSliced", "state": "isPickedUp", "value": true}]


Instruction: I'm putting things where they belong, so please pick up the tomato, carry it over, and set it down on the dining table neatly.

{"steps": 12, "actions": ["Rotate(left, 90)", "Pick(Tomato)", "Move(forward, 0.5)", "Move(forward, 0.25)", "Move(left, 0.25)", "Move(forward, 1.0)", "Move(forward, 1.0)", "Rotate(right, 90)", "Move(forward, 0.25)", "Move(left, 0.25)", "Place(DiningTable)", "EndTask(success)"]}

[{"type": "object_in_receptacle", "object_type": "Tomato", "receptacle_type": "DiningTable", "value": true}, {"type": "object_in_receptacle", "object_type": "Tomato", "receptacle_type": "CounterTop", "value": false}]


Instruction: The soup is a bit bland, please move the salt shaker to the dining table.

{"steps": 17, "actions": ["Rotate(right, 90)", "Move(forward, 1.0)", "Pick(SaltShaker)", "Rotate(right, 90)", "Rotate(right, 90)", "Move(forward, 0.5)", "Move(left, 0.25)", "Move(forward, 1.0)", "Move(left, 0.25)", "Move(forward, 1.0)", "Move(forward, 1.0)", "Move(forward, 0.25)", "Rotate(right, 90)", "Move(forward, 0.5)", "Move(left, 0.25)", "Place(DiningTable)", "EndTask(success)"]}

[{"type": "object_in_receptacle", "object_type": "SaltShaker", "receptacle_type": "DiningTable", "value": true}, {"type": "object_in_receptacle", "object_type": "SaltShaker", "receptacle_type": "CounterTop", "value": false}]


Instruction: Dinner is served, so please move the ladle to the dining table.

{"steps": 21, "actions": ["Rotate(right, 90)", "Move(left, 0.5)", "Move(left, 0.25)", "Move(forward, 0.5)", "Move(forward, 0.25)", "Pick(Ladle)", "Rotate(right, 90)", "Move(forward, 0.5)", "Rotate(right, 90)", "Move(forward, 0.25)", "Move(left, 1.0)", "Move(forward, 1.0)", "Move(forward, 1.0)", "Move(forward, 1.0)", "Move(forward, 0.25)", "Rotate(right, 90)", "Move(forward, 0.5)", "Move(left, 0.5)", "Move(forward, 0.25)", "Place(DiningTable)", "EndTask(success)"]}

[{"type": "object_in_receptacle", "object_type": "Ladle", "receptacle_type": "DiningTable", "value": true}, {"type": "object_in_receptacle", "object_type": "Ladle", "receptacle_type": "CounterTop", "value": false}]


Instruction: I’m making a fresh summer salad; please slice the lettuce and the apple for me.

{"steps": 9, "actions": ["Rotate(right, 90)", "Rotate(right, 90)", "Move(forward, 0.5)", "Move(forward, 0.25)", "Manipulate(Lettuce, slice)", "Move(forward, 0.25)", "Move(left, 0.5)", "Manipulate(Apple, slice)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "Lettuce", "state": "isSliced", "value": true}, {"type": "object_state", "object_type": "Apple", "state": "isSliced", "value": true}]


Instruction: My lunch has gotten cold, so please turn on the microwave to warm it up.

{"steps": 5, "actions": ["Rotate(right, 90)", "Move(forward, 1.0)", "Move(forward, 0.25)", "ChangeState(Microwave, on)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "Microwave", "state": "isToggled", "value": true}]


Instruction: I'm getting ready to eat, so please slice the apple as instructed and keep the pieces together and then slice the bread as instructed and keep the pieces together.

{"steps": 7, "actions": ["Rotate(right, 90)", "Move(left, 0.5)", "Move(left, 0.25)", "Move(forward, 0.25)", "Manipulate(Apple, slice)", "Manipulate(Bread, slice)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "Apple", "state": "isSliced", "value": true}, {"type": "object_state", "object_type": "Bread", "state": "isSliced", "value": true}]


Instruction: I need it running, so please turn on the faucet.

{"steps": 6, "actions": ["Rotate(right, 90)", "Rotate(right, 90)", "Move(forward, 1.0)", "Move(forward, 0.5)", "ChangeState(Faucet, on)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "Faucet", "state": "isToggled", "value": true}]


Instruction: I'm about to cook, so please slice the lettuce as instructed and keep the pieces together.

{"steps": 6, "actions": ["Rotate(right, 90)", "Move(right, 1.0)", "Move(right, 0.25)", "Move(forward, 0.5)", "Manipulate(Lettuce, slice)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "Lettuce", "state": "isSliced", "value": true}]


Instruction: Please help me with this task in the scene: break the plate.

{"steps": 7, "actions": ["Rotate(right, 90)", "Rotate(right, 90)", "Move(forward, 1.0)", "Move(forward, 0.5)", "Move(right, 0.5)", "Manipulate(Plate, break)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "Plate", "state": "isBroken", "value": true}]


Instruction: We need to make soup for dinner; can you fill the pot with water?

{"steps": 6, "actions": ["Rotate(right, 90)", "Rotate(right, 90)", "Move(forward, 1.0)", "Move(forward, 0.5)", "Manipulate(Pot, water, fillwithliquid)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "Pot", "state": "isFilledWithLiquid", "value": true, "liquid": "water"}]


Instruction: I'm starting breakfast now, so please turn on the coffee machine for me.

{"steps": 8, "actions": ["Rotate(right, 90)", "Rotate(right, 90)", "Move(right, 0.5)", "Move(forward, 0.25)", "Move(right, 0.25)", "Move(forward, 0.25)", "ChangeState(CoffeeMachine, on)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "CoffeeMachine", "state": "isToggled", "value": true}]


Instruction: I’m feeling a bit sleepy; could you fill the cup with coffee to help me wake up?

{"steps": 7, "actions": ["Rotate(right, 90)", "Rotate(right, 90)", "Move(forward, 1.0)", "Move(forward, 1.0)", "Rotate(left, 90)", "Manipulate(Cup, coffee, fillwithliquid)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "Cup", "state": "isFilledWithLiquid", "value": true, "liquid": "coffee"}]


Instruction: I want to exercise now, could you please hand me the tennis racket on the floor near the table?

{"steps": 3, "actions": ["Rotate(right, 90)", "Pick(TennisRacket)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "TennisRacket", "field": "isPickedUp", "value": true}]


Pick up the baseball bat at the door and smash the mirror

Instruction: I want to swing the baseball bat a few times for a workout and then smash the mirror. I think that's cool.

{"steps": 7, "actions": ["Move(forward, 0.25)", "Tilt(down, 30)", "Pick(BaseballBat)", "Rotate(right, 90)", "Move(forward, 0.5)", "Manipulate(Mirror, break)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "BaseballBat", "field": "isPickedUp", "value": true}, {"type": "object_state", "object_type": "Mirror", "field": "isBroken", "value": true}]


Pick up the baseball bat and smash the laptop

Instruction: Please pick up the baseball bat in the corner, walk to the window, and smash the laptop on the table

{"steps": 10, "actions": ["Move(forward, 1.0)", "Rotate(right, 90)", "Move(forward, 0.25)", "Pick(BaseballBat)", "Rotate(right, 90)", "Move(forward, 1.0)", "Move(forward, 1.0)", "Move(forward, 0.25)", "Manipulate(Laptop, break)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "BaseballBat", "field": "isPickedUp", "value": true}, {"type": "object_state", "object_type": "Laptop", "field": "isBroken", "value": true}]


Pick up the baseball bat and smash the CellPhone

Instruction: I want to practice dribbling in the room. Please pick up the basketball next to my shoes and then smash the phone screen. That could be my workout.

{"steps": 10, "actions": ["Rotate(right, 90)", "Rotate(right, 90)", "Move(forward, 0.5)", "Move(forward, 0.25)", "Tilt(down, 30)", "Pick(BasketBall)", "Move(forward, 0.5)", "Move(forward, 0.25)", "Manipulate(CellPhone, break)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "BasketBall", "field": "isPickedUp", "value": true}, {"type": "object_state", "object_type": "CellPhone", "field": "isBroken", "value": true}]


Instruction: I see a tennis racket—could you pick it up for me? I'm ready to get a good workout in

{"steps": 9, "actions": ["Rotate(right, 90)", "Move(forward, 1.0)", "Move(forward, 1.0)", "Rotate(right, 90)", "Move(forward, 1.0)", "Move(forward, 0.5)", "Move(forward, 0.25)", "Pick(TennisRacket)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "TennisRacket", "field": "isPickedUp", "value": true}]


Pick up the tennis racket and clean the bed

Instruction: I want to combine my workout with some housework—could you pick up the tennis racket in the corner and give the bed a quick clean?

{"steps": 11, "actions": ["Rotate(left, 90)", "Move(forward, 0.5)", "Move(forward, 0.25)", "Rotate(left, 90)", "Move(forward, 1.0)", "Rotate(right, 90)", "Move(forward, 0.5)", "Move(forward, 0.25)", "Pick(TennisRacket)", "Manipulate(Bed, clean)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "TennisRacket", "field": "isPickedUp", "value": true}, {"type": "object_state", "object_type": "Bed", "field": "isDirty", "value": false}]


Pick up the basketball and smash the mirror

Instruction: I was just trying to work out, but I picked up the basketball and accidentally smashed the mirror—I really didn’t mean to.

{"steps": 7, "actions": ["Tilt(down, 30)", "Move(forward, 0.5)", "Move(forward, 0.25)", "Pick(BasketBall)", "Rotate(left, 90)", "Manipulate(Mirror, break)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "BasketBall", "field": "isPickedUp", "value": true}, {"type": "object_state", "object_type": "Mirror", "field": "isBroken", "value": true}]


Instruction: It’s such a lovely day; I feel like tidying up my bed. First I’ll pick up my pillow, then give the whole bed a good clean—so I can sleep nice and sound tonight.

{"steps": 7, "actions": ["Move(forward, 0.5)", "Pick(Pillow)", "Move(right, 0.25)", "Move(forward, 0.25)", "Move(right, 0.5)", "Manipulate(Bed, clean)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "Pillow", "field": "isPickedUp", "value": true}, {"type": "object_state", "object_type": "Bed", "field": "isDirty", "value": false}]


Instruction: I’d like to take a nap. Could you tidy the bed and turn off the wall light for me? Thank you.

{"steps": 9, "actions": ["Move(forward, 0.5)", "Move(forward, 0.25)", "Rotate(right, 90)", "Move(forward, 0.5)", "Move(forward, 0.25)", "Manipulate(Bed, clean)", "Move(forward, 0.25)", "ChangeState(LightSwitch, off)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "Bed", "field": "isDirty", "value": false}, {"type": "object_state", "object_type": "LightSwitch", "field": "isToggled", "value": false}]


Instruction: I have something really important tomorrow, and I’m afraid I won’t hear the alarm—could you please move the alarm clock from the desk to my side table? Thanks!

{"steps": 12, "actions": ["Move(left, 1.0)", "Move(forward, 0.5)", "Move(left, 0.25)", "Move(forward, 0.5)", "Pick(AlarmClock)", "Move(backward, 1.0)", "Rotate(right, 90)", "Move(forward, 1.0)", "Tilt(down, 30)", "Move(forward, 0.5)", "Place(SideTable)", "EndTask(success)"]}

[{"type": "object_in_receptacle", "object_type": "AlarmClock", "receptacle_type": "SideTable", "value": true}, {"type": "object_in_receptacle", "object_type": "AlarmClock", "receptacle_type": "Desk", "value": false}]


Instruction: I'm so tired tonight that I want to go to bed early—could you please turn off the lights in the room? The switch is right by the door. Thanks!

{"steps": 12, "actions": ["Rotate(left, 90)", "Move(forward, 1.0)", "Move(forward, 0.5)", "Move(forward, 0.25)", "Rotate(right, 90)", "Move(forward, 0.5)", "Move(forward, 0.25)", "Rotate(left, 90)", "Move(forward, 0.5)", "Tilt(down, 30)", "ChangeState(LightSwitch, off)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "LightSwitch", "field": "isToggled", "value": false}]


Instruction: I'd like to read the book on my desk before bed. Please tidy up the bed first, then bring the book and place it on the bed for me.

{"steps": 12, "actions": ["Move(forward, 0.25)", "Manipulate(Bed, clean)", "Rotate(left, 90)", "Move(forward, 1.0)", "Move(forward, 1.0)", "Rotate(right, 90)", "Move(forward, 0.5)", "Pick(Book)", "Rotate(right, 90)", "Move(forward, 0.5)", "Place(Bed)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "Bed", "field": "isDirty", "value": false}, {"type": "object_in_receptacle", "object_type": "Book", "receptacle_type": "Bed"}]


Instruction: I'm getting a little sleepy and want to go play on my phone in bed. I need to take my phone from the table to the bed

{"steps": 11, "actions": ["Move(forward, 0.5)", "Pick(CellPhone)", "Rotate(left, 90)", "Rotate(left, 90)", "Move(forward, 0.5)", "Rotate(right, 90)", "Move(forward, 0.5)", "Move(forward, 0.25)", "Rotate(right, 90)", "Place(Bed)", "EndTask(success)"]}

[{"type": "object_in_receptacle", "object_type": "CellPhone", "receptacle_type": "Desk", "value": false}, {"type": "object_in_receptacle", "object_type": "CellPhone", "receptacle_type": "Bed", "value": true}]


Instruction: My pencil is broken and I want to throw it away. Could you help me drop it from the table into the trash can?

{"steps": 9, "actions": ["Move(forward, 0.5)", "Move(forward, 0.25)", "Pick(Pencil)", "Rotate(left, 90)", "Move(forward, 0.5)", "Rotate(right, 90)", "Tilt(down, 30)", "Place(GarbageCan)", "EndTask(success)"]}

[{"type": "object_in_receptacle", "object_type": "Pencil", "receptacle_type": "GarbageCan", "value": true}, {"type": "object_in_receptacle", "object_type": "Pencil", "receptacle_type": "Desk", "value": false}]


Instruction: I feel like taking a break; I'm going to pick up my phone from the desk and turn on the TV so I can really unwind

{"steps": 7, "actions": ["Move(forward, 0.5)", "Pick(CellPhone)", "Rotate(right, 90)", "Move(forward, 0.5)", "Move(forward, 0.25)", "ChangeState(Television, on)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "CellPhone", "field": "isPickedUp", "value": true}, {"type": "object_state", "object_type": "Television", "field": "isToggled", "value": true}]


Instruction: I have ended my friendship and I no longer want the pencil or the fountain pen he gave me. I have already thrown the pencil into the trash. Could you throw the fountain pen into the trash for me?

{"steps": 6, "actions": ["Move(left, 0.5)", "Pick(Pen)", "Move(right, 0.5)", "Move(forward, 0.25)", "Place(GarbageCan)", "EndTask(success)"]}

[{"type": "object_in_receptacle", "object_type": "Pen", "receptacle_type": "GarbageCan", "value": true}, {"type": "object_in_receptacle", "object_type": "Pen", "receptacle_type": "Desk", "value": false}]


Straighten up the bed and then turn on the laptop.

Instruction: I just woke up and I’m ready to get to work—you help me tidy the bed first, then use the laptop to start working and get it booted up.

{"steps": 13, "actions": ["Move(left, 0.5)", "Move(left, 0.25)", "Move(forward, 0.5)", "Move(left, 0.25)", "Move(forward, 1.0)", "Move(forward, 1.0)", "Rotate(right, 90)", "Manipulate(Bed, clean)", "Rotate(right, 90)", "Move(right, 1.0)", "Move(right, 0.25)", "ChangeState(Laptop, on)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "Bed", "field": "isDirty", "value": false}, {"type": "object_state", "object_type": "Laptop", "field": "isToggled", "value": true}]


Please pour a glass of water, turn on the cell phone

Instruction: I want to take a proper break. Please pour me a glass of water and then turn on my phone—I’d like to play a game for a bit.

{"steps": 7, "actions": ["Rotate(right, 90)", "Move(forward, 0.25)", "Move(left, 0.25)", "Move(forward, 0.25)", "Manipulate(Mug, water, fillwithliquid)", "ChangeState(CellPhone, on)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "CellPhone", "field": "isToggled", "value": true}, {"type": "object_state", "object_type": "Mug", "field": "isFilledWithLiquid", "value": true, "liquid": "water"}]


Instruction: I’d like to start working from the bedside—could you please pick up the laptop?

{"steps": 3, "actions": ["Move(forward, 0.25)", "Pick(Laptop)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "Laptop", "state": "isPickedUp", "value": true}]

I’d like to start working from the bedside—could you please move the laptop from the desk to the bed?


Pour water into the cup and then pick up the baseball.

Instruction: I'm about to start working out—please fill the cup with water and then pick up the baseball bat

{"steps": 4, "actions": ["Move(forward, 1.0)", "Manipulate(Mug, water, fillwithliquid)", "Pick(BaseballBat)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "Mug", "state": "isFilledWithLiquid", "value": true, "liquid": "water"}, {"type": "object_state", "object_type": "BaseballBat", "state": "isPickedUp", "value": true}]


Please pick up the pen and open the book.

Instruction: I want to study now. Please pick up the fountain pen on the table in front of you, then go to the other desk, open the book, and study hard

{"steps": 10, "actions": ["Move(forward, 0.5)", "Move(forward, 0.25)", "Pick(Pen)", "Rotate(right, 90)", "Move(forward, 1.0)", "Move(forward, 0.5)", "Move(forward, 0.25)", "Move(left, 0.5)", "Manipulate(Book, open)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "Pen", "field": "isPickedUp", "value": true}, {"type": "object_state", "object_type": "Book", "field": "isOpen", "value": true}]


Instruction: I'm going to start working seriously. Please turn on the desktop computer and the laptop on the desk, then fill the bowl with water

{"steps": 9, "actions": ["Rotate(right, 90)", "Move(forward, 1.0)", "ChangeState(Desktop, on)", "Rotate(left, 90)", "Move(forward, 0.25)", "ChangeState(Laptop, on)", "Rotate(right, 90)", "Manipulate(Bowl, water, fillwithliquid)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "Desktop", "field": "isToggled", "value": true}, {"type": "object_state", "object_type": "Laptop", "field": "isToggled", "value": true}, {"type": "object_state", "object_type": "Bowl", "field": "isFilledWithLiquid", "value": true, "liquid": "water"}]


Instruction: The sunlight is lovely now—there's no need for extra light. Please switch off the bedside lamp

{"steps": 8, "actions": ["Rotate(left, 90)", "Move(forward, 1.0)", "Move(forward, 0.25)", "Rotate(right, 90)", "Move(forward, 1.0)", "Move(forward, 0.25)", "ChangeState(DeskLamp, off)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "DeskLamp", "field": "isToggled", "value": false}]


Instruction: I'm ready to work now; the desk lamp is already on. Please turn off the wall light and open the book

{"steps": 9, "actions": ["ChangeState(LightSwitch, off)", "Rotate(right, 90)", "Rotate(right, 90)", "Move(forward, 0.5)", "Move(forward, 0.25)", "Move(right, 0.5)", "Move(forward, 0.5)", "Manipulate(Book, open)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "LightSwitch", "state": "isToggled", "value": false}, {"type": "object_state", "object_type": "Book", "state": "isOpen", "value": true}]


Put the book on the table onto the armchair

Instruction: I'd like to take a break and read for a while in the armchair—could you place the book on it?

{"steps": 6, "actions": ["Tilt(down, 30)", "Pick(Book)", "Rotate(left, 90)", "Move(forward, 0.25)", "Place(ArmChair)", "EndTask(success)"]}

[{"type": "object_in_receptacle", "object_type": "Book", "receptacle_type": "Desk", "value": false}, {"type": "object_in_receptacle", "object_type": "Book", "receptacle_type": "ArmChair"}]


Put the book on the table onto the bed, then open it

Instruction: I'd like to read in bed for a while—please take the book from the desk, put it on the bed, and open it

{"steps": 14, "actions": ["Rotate(left, 90)", "Move(forward, 0.5)", "Move(forward, 0.25)", "Rotate(right, 90)", "Move(forward, 0.5)", "Pick(Book)", "Rotate(left, 90)", "Move(forward, 1.0)", "Move(forward, 0.5)", "Rotate(left, 90)", "Move(forward, 0.5)", "Place(Bed)", "Manipulate(Book, open)", "EndTask(success)"]}

[{"type": "object_in_receptacle", "object_type": "Book", "receptacle_type": "Dresser", "value": false}, {"type": "object_in_receptacle", "object_type": "Book", "receptacle_type": "Bed"}, {"type": "object_state", "object_type": "Book", "field": "isOpen", "value": true}]


Pick up the trophy and place it on the dresser, then grab the basketball

Instruction: I'm the MVP of the basketball league. Next to a painting on my bookshelf is my MVP trophy; I want to place it on the dresser , then pick up the basketball and work out

{"steps": 19, "actions": ["Pick(Statue)", "Rotate(left, 90)", "Rotate(left, 90)", "Move(forward, 1.0)", "Rotate(right, 90)", "Move(forward, 0.5)", "Place(Dresser)", "Rotate(right, 90)", "Move(forward, 0.25)", "Rotate(right, 90)", "Move(forward, 1.0)", "Rotate(left, 90)", "Move(forward, 0.25)", "Rotate(right, 90)", "Move(forward, 1.0)", "Move(forward, 0.25)", "Tilt(down, 30)", "Pick(BasketBall)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "BasketBall", "field": "isPickedUp", "value": true}, {"type": "object_in_receptacle", "object_type": "Statue", "receptacle_type": "Dresser"}]


Instruction: I'd like to work and watch TV at the same time. Could you please help me turn on the laptop and then the TV?

{"steps": 10, "actions": ["Move(forward, 1.0)", "Move(forward, 0.5)", "ChangeState(Laptop, on)", "Rotate(right, 90)", "Rotate(right, 90)", "Move(forward, 1.0)", "Move(forward, 1.0)", "Move(forward, 0.25)", "ChangeState(Television, on)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "Laptop", "field": "isToggled", "value": true}, {"type": "object_state", "object_type": "Television", "field": "isToggled", "value": true}]


Take the tissues and place them on the table

Instruction: I'd like to work at the other desk. Please take the tissues next to the laptop and put them beside the newspaper on that desk

{"steps": 17, "actions": ["Move(forward, 1.0)", "Rotate(left, 90)", "Move(forward, 0.5)", "Move(forward, 0.25)", "Rotate(right, 90)", "Move(forward, 0.5)", "Pick(TissueBox)", "Rotate(right, 90)", "Rotate(right, 90)", "Move(forward, 1.0)", "Move(forward, 1.0)", "Rotate(right, 90)", "Move(forward, 1.0)", "Rotate(left, 90)", "Move(forward, 0.5)", "Place(SideTable)", "EndTask(success)"]}

[{"type": "object_in_receptacle", "object_type": "TissueBox", "receptacle_type": "Cabinet", "value": false}, {"type": "object_in_receptacle", "object_type": "TissueBox", "receptacle_type": "SideTable"}]


Please close the paper box and pick it up.

Instruction: I'm cleaning the room right now—please close the paper box and then pick it up for me

{"steps": 6, "actions": ["Move(forward, 0.5)", "Tilt(down, 30)", "Move(forward, 0.25)", "Manipulate(Box, close)", "Pick(Box)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "Box", "field": "isOpen", "value": false}, {"type": "object_state", "object_type": "Box", "field": "isPickedUp", "value": true}]


Please turn off the light and turn on the TV.

Instruction: It's a bright and sunny afternoon; I'm going to turn off the wall light in the room, switch on the TV, and enjoy the rest of the day

{"steps": 12, "actions": ["Move(forward, 1.0)", "Move(forward, 1.0)", "Move(forward, 1.0)", "ChangeState(LightSwitch, off)", "Rotate(left, 90)", "Rotate(left, 90)", "Move(forward, 1.0)", "Move(forward, 1.0)", "Move(forward, 1.0)", "Move(forward, 1.0)", "ChangeState(Television, on)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "LightSwitch", "field": "isToggled", "value": false}, {"type": "object_state", "object_type": "Television", "field": "isToggled", "value": true}]


Instruction: I'm taking out the trash, and the glove sculpture in front of me is something I really dislike—please pick it up, walk over to the trash can, and smash it

{"steps": 19, "actions": ["Move(left, 0.5)", "Move(left, 0.25)", "Move(forward, 0.5)", "Move(forward, 0.25)", "Pick(Statue)", "Rotate(right, 90)", "Move(forward, 0.5)", "Move(right, 0.5)", "Move(forward, 0.25)", "Move(right, 1.0)", "Move(right, 1.0)", "Move(right, 0.5)", "Move(forward, 1.0)", "Move(right, 0.25)", "Move(forward, 0.5)", "Tilt(down, 30)", "Move(forward, 0.25)", "Manipulate(Statue, break)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "Statue", "field": "isBroken", "value": true}]


Please turn off the light and turn on the floor lamp.

Instruction: The sunlight is so bright today—there's no need for the big wall light in the room, so go ahead and switch it off. Just turn on the floor lamp next to the sofa instead.

{"steps": 13, "actions": ["Move(forward, 1.0)", "Move(forward, 1.0)", "Move(forward, 1.0)", "ChangeState(LightSwitch, off)", "Rotate(left, 90)", "Move(forward, 1.0)", "Move(forward, 0.5)", "Rotate(left, 90)", "Move(forward, 0.25)", "Rotate(right, 90)", "Move(forward, 0.5)", "ChangeState(FloorLamp, on)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "LightSwitch", "state": "isToggled", "value": false}, {"type": "object_state", "object_type": "FloorLamp", "state": "isToggled", "value": true}]


Please pick up the key chain and turn off the light.

Instruction: I'm going to walk out for a bit—I don't want to leave the door open, so please pick up the key chain and turn off the light

{"steps": 19, "actions": ["Tilt(down, 30)", "Move(forward, 0.5)", "Move(forward, 0.25)", "Rotate(left, 90)", "Move(forward, 0.5)", "Move(forward, 0.25)", "Pick(KeyChain)", "Rotate(right, 90)", "Move(forward, 1.0)", "Move(forward, 1.0)", "Move(forward, 0.25)", "Rotate(right, 90)", "Move(forward, 0.5)", "Move(forward, 0.25)", "Rotate(left, 90)", "Move(forward, 0.25)", "Tilt(up, 30)", "ChangeState(LightSwitch, off)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "LightSwitch", "state": "isToggled", "value": false}, {"type": "object_state", "object_type": "KeyChain", "state": "isPickedUp", "value": true}]


Please fill the house plant with water and pick up the plate.

Instruction: I'm the housekeeper of this room, and I need to keep it tidy every day. I'd like to water the flowers by the door first, then clear the plates from the dinner table.

{"steps": 10, "actions": ["Move(forward, 1.0)", "Rotate(left, 90)", "Move(forward, 0.25)", "Manipulate(HousePlant, water, fillwithliquid)", "Rotate(left, 90)", "Move(forward, 0.5)", "Rotate(right, 90)", "Move(forward, 0.25)", "Pick(Plate)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "HousePlant", "state": "isFilledWithLiquid", "value": true, "liquid": "water"}, {"type": "object_state", "object_type": "Plate", "state": "isPickedUp", "value": true}]


Please pick up the bowl and put it on the dining table.

Instruction: I'm about to have dinner—please take the bowl from the coffee table and bring it to the dinner table

{"steps": 14, "actions": ["Move(left, 0.5)", "Move(left, 0.25)", "Move(forward, 0.5)", "Tilt(down, 30)", "Pick(Bowl)", "Move(backward, 0.5)", "Move(backward, 0.25)", "Rotate(right, 90)", "Move(forward, 1.0)", "Move(forward, 1.0)", "Move(left, 1.0)", "Move(left, 0.25)", "Place(DiningTable)", "EndTask(success)"]}

[{"type": "object_in_receptacle", "object_type": "Bowl", "receptacle_type": "CoffeeTable", "value": false}, {"type": "object_in_receptacle", "object_type": "Bowl", "receptacle_type": "DiningTable"}]


Please pick up the newspaper and place it on the coffee table

Instruction: I've finished dinner and feel like reading the paper—please pick up the newspaper and set it right next to the bowl on the coffee table so I can relax and read.

{"steps": 14, "actions": ["Move(forward, 0.25)", "Pick(Newspaper)", "Rotate(left, 90)", "Rotate(left, 90)", "Move(forward, 0.5)", "Rotate(right, 90)", "Move(forward, 1.0)", "Move(forward, 1.0)", "Move(forward, 1.0)", "Rotate(right, 90)", "Move(forward, 1.0)", "Tilt(down, 30)", "Place(CoffeeTable)", "EndTask(success)"]}

[{"type": "object_in_receptacle", "object_type": "Newspaper", "receptacle_type": "DiningTable", "value": false}, {"type": "object_in_receptacle", "object_type": "Newspaper", "receptacle_type": "CoffeeTable"}]


Please pick up the watch and place it in the box

Instruction: I'm going to the gym and need to put my watch in the box—please pick up the watch and place it in the box

{"steps": 13, "actions": ["Move(forward, 0.5)", "Move(forward, 0.25)", "Pick(Watch)", "Rotate(left, 90)", "Rotate(left, 90)", "Move(forward, 0.5)", "Move(forward, 0.25)", "Rotate(left, 90)", "Move(forward, 0.5)", "Rotate(right, 90)", "Move(forward, 0.5)", "Place(Box)", "EndTask(success)"]}

[{"type": "object_in_receptacle", "object_type": "Watch", "receptacle_type": "TVStand", "value": false}, {"type": "object_in_receptacle", "object_type": "Watch", "receptacle_type": "Box", "value": true}]


Please pick up the book and place it on the sofa

Instruction: Please pick up the book on the TV stand—the one that seems to be about deep learning—and place it on the sofa; I'll come study it in a bit

{"steps": 12, "actions": ["Move(forward, 0.5)", "Move(forward, 0.25)", "Rotate(left, 90)", "Move(forward, 0.5)", "Rotate(right, 90)", "Move(forward, 0.25)", "Pick(Book)", "Rotate(left, 90)", "Rotate(left, 90)", "Move(forward, 1.0)", "Place(Sofa)", "EndTask(success)"]}

[{"type": "object_in_receptacle", "object_type": "Book", "receptacle_type": "TVStand", "value": false}, {"type": "object_in_receptacle", "object_type": "Book", "receptacle_type": "Sofa"}]


Please water the potted plant on the table and then turn off the wall light

Instruction: I'm getting ready to head out—please water the potted plant on the table and then turn off the wall light.

{"steps": 11, "actions": ["Move(forward, 0.5)", "Manipulate(HousePlant, water, fillwithliquid)", "Rotate(right, 90)", "Rotate(right, 90)", "Move(forward, 0.5)", "Move(forward, 0.25)", "Rotate(left, 90)", "Move(forward, 1.0)", "Move(forward, 0.25)", "ChangeState(LightSwitch, off)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "HousePlant", "field": "isFilledWithLiquid", "value": true, "liquid": "water"}, {"type": "object_state", "object_type": "LightSwitch", "field": "isToggled", "value": false}]


Instruction: I’ve finished dinner and feel like scrolling on my phone for a bit—please pick it up from the dining table, turn it on, and set it on the coffee table so I can relax with a full stomach

{"steps": 14, "actions": ["Move(forward, 1.0)", "ChangeState(CellPhone, on)", "Pick(CellPhone)", "Rotate(left, 90)", "Move(forward, 1.0)", "Move(forward, 1.0)", "Move(forward, 1.0)", "Move(forward, 0.25)", "Rotate(left, 90)", "Move(forward, 0.25)", "Tilt(down, 30)", "Move(forward, 0.5)", "Place(CoffeeTable)", "EndTask(success)"]}

[{"type": "object_in_receptacle", "object_type": "CellPhone", "receptacle_type": "DiningTable", "value": false}, {"type": "object_state", "object_type": "CellPhone", "field": "isToggled", "value": true}, {"type": "object_in_receptacle", "object_type": "CellPhone", "receptacle_type": "CoffeeTable"}]


Instruction: I'm cleaning the bathroom right now—I'm throwing the rag into the trash can

{"steps": 7, "actions": ["Move(forward, 0.5)", "Pick(Cloth)", "Rotate(right, 90)", "Move(forward, 1.0)", "Move(forward, 0.25)", "Place(GarbageCan)", "EndTask(success)"]}

[{"type": "object_in_receptacle", "object_type": "Cloth", "receptacle_type": "SideTable", "value": false}, {"type": "object_in_receptacle", "object_type": "Cloth", "receptacle_type": "GarbageCan", "value": true}]


Lift the lid and put the toilet paper into the toilet paper hanger

Instruction: I need to use the toilet—please lift the lid and then put the toilet paper into the toilet paper hanger.

{"steps": 11, "actions": ["Move(forward, 1.0)", "Rotate(left, 90)", "Move(forward, 0.5)", "Rotate(right, 90)", "Move(forward, 0.25)", "Tilt(down, 30)", "Move(forward, 0.5)", "Manipulate(Toilet, open)", "Pick(ToiletPaper)", "Place(ToiletPaperHanger)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "Toilet", "field": "isOpen", "value": true}, {"type": "object_in_receptacle", "object_type": "ToiletPaper", "receptacle_type": "ToiletPaperHanger", "value": true}]


Pick up the scrub brush and put it in the bathtub basin

Instruction: I need to clean the bathtub—please pick up the scrub brush and then put it in the bathtub basin.

{"steps": 14, "actions": ["Move(forward, 1.0)", "Move(forward, 0.25)", "Move(left, 1.0)", "Move(left, 0.25)", "Move(forward, 0.25)", "Tilt(down, 30)", "Move(forward, 0.25)", "Pick(ScrubBrush)", "Rotate(right, 90)", "Move(right, 0.25)", "Move(forward, 1.0)", "Move(forward, 0.5)", "Place(BathtubBasin)", "EndTask(success)"]}

[{"type": "object_in_receptacle", "object_type": "ScrubBrush", "receptacle_type": "BathtubBasin", "value": true}]


Pick up the soap bottle, turn on the faucet, and put it back on the side table

Instruction: I want to wash my face—please pick up the soap bottle, turn on the faucet, and then put it back on the side table where it was.

{"steps": 11, "actions": ["Move(forward, 0.25)", "Pick(SoapBottle)", "Move(forward, 0.25)", "Rotate(right, 90)", "Move(forward, 0.25)", "Rotate(left, 90)", "Move(forward, 0.25)", "ChangeState(Faucet, on)", "Rotate(right, 90)", "Place(SideTable)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "Faucet", "field": "isToggled", "value": true}, {"type": "object_in_receptacle", "object_type": "SoapBottle", "receptacle_type": "SideTable"}]


Open the shower door, step inside, turn on the faucet

Instruction: I'm going to take a shower—open the shower door, step inside, turn on the faucet

{"steps": 10, "actions": ["Move(forward, 0.25)", "Manipulate(ShowerDoor, open)", "Rotate(right, 90)", "Move(forward, 0.25)", "Rotate(left, 90)", "Move(forward, 1.0)", "Rotate(left, 90)", "Tilt(down, 30)", "ChangeState(ShowerHead, on)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "ShowerDoor", "state": "isOpen", "value": true}, {"type": "object_state", "object_type": "ShowerHead", "state": "isToggled", "value": true}]


Open the bathroom door, turn off the lights

Instruction: I've finished my shower. Please open the bathroom door for me and turn off the lights. Thank you.

{"steps": 11, "actions": ["Move(forward, 0.25)", "Manipulate(ShowerDoor, open)", "Rotate(right, 90)", "Move(forward, 0.25)", "Rotate(right, 90)", "Rotate(right, 90)", "Move(left, 0.5)", "Move(forward, 0.5)", "Move(forward, 0.25)", "ChangeState(LightSwitch, off)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "ShowerDoor", "field": "isOpen", "value": true}, {"type": "object_state", "object_type": "LightSwitch", "field": "isToggled", "value": false}]


Pick up the soap bar, put it on the bathtub, and then open the shower curtain

Instruction: I'm about to hop in the shower. Could you grab the soap, drop it on the tub, and pull the curtain open for me?

{"steps": 11, "actions": ["Move(forward, 0.5)", "Move(right, 0.5)", "Move(right, 0.25)", "Pick(SoapBar)", "Rotate(right, 90)", "Tilt(down, 30)", "Move(forward, 0.25)", "Place(Bathtub)", "Move(right, 0.5)", "Manipulate(ShowerCurtain, open)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "ShowerCurtain", "state": "isOpen", "value": true}, {"type": "object_in_receptacle", "object_type": "SoapBar", "receptacle_type": "Bathtub", "value": true}, {"type": "object_in_receptacle", "object_type": "SoapBar", "receptacle_type": "CounterTop", "value": false}]

["Bathtub", "CounterTop", "ShowerCurtain", "SoapBar"]


Instruction: I want to wash my face—could you turn on the faucet for me?

{"steps": 4, "actions": ["Move(left, 0.25)", "Move(forward, 0.5)", "ChangeState(Faucet, on)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "Faucet", "field": "isToggled", "value": true}]


Instruction: I need to use the toilet. Could you lift the seat for me? Thanks.

{"steps": 4, "actions": ["Move(right, 0.5)", "Move(right, 0.25)", "Manipulate(Toilet, open)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "Toilet", "field": "isOpen", "value": true}]


Take the paper towel from the sink counter and place it on the toilet

Instruction: I'm in a hurry to use the toilet—could you take the paper towel from the sink counter and place it on the toilet for me?

{"steps": 11, "actions": ["Move(forward, 0.5)", "Move(right, 0.5)", "Pick(TissueBox)", "Rotate(left, 90)", "Move(forward, 0.5)", "Move(forward, 0.25)", "Rotate(left, 90)", "Tilt(down, 30)", "Move(right, 0.5)", "Place(Toilet)", "EndTask(success)"]}

[{"type": "object_in_receptacle", "object_type": "TissueBox", "receptacle_type": "CounterTop", "value": false}, {"type": "object_in_receptacle", "object_type": "TissueBox", "receptacle_type": "Toilet"}]


Instruction: I am done with the big clean. My rag is really dirty. Could you toss it in the trash for me?

{"steps": 6, "actions": ["Pick(Cloth)", "Rotate(left, 90)", "Tilt(down, 30)", "Move(forward, 0.5)", "Place(GarbageCan)", "EndTask(success)"]}

[{"type": "object_in_receptacle", "object_type": "Cloth", "receptacle_type": "GarbageCan"}]


Instruction: I want to clean the shower glass—please grab the spray bottle

{"steps": 3, "actions": ["Move(forward, 0.25)", "Pick(SprayBottle)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "SprayBottle", "field": "isPickedUp", "value": true}]


Take a hand towel from the hand-towel holder so I can dry them

Instruction: I've finished using the toilet and washing my hands—please take a hand towel from the hand-towel holder so I can dry them.

{"steps": 3, "actions": ["Move(forward, 0.5)", "Pick(HandTowel)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "HandTowel", "field": "isPickedUp", "value": true}]


Pick up the candle next to the bathtub and place it on the countertop to keep water from splashing onto it

Instruction: I'm about to start my bath—please pick up the candle next to the bathtub and place it on the countertop to keep water from splashing onto it.

{"steps": 13, "actions": ["Rotate(left, 90)", "Move(right, 1.0)", "Move(forward, 0.25)", "Tilt(down, 30)", "Pick(Candle)", "Rotate(right, 90)", "Rotate(right, 90)", "Move(forward, 1.0)", "Rotate(left, 90)", "Move(forward, 0.5)", "Move(forward, 0.25)", "Place(CounterTop)", "EndTask(success)"]}

[{"type": "object_in_receptacle", "object_type": "Candle", "receptacle_type": "CounterTop"}, {"type": "object_in_receptacle", "object_type": "Candle", "receptacle_type": "Bathtub", "value": false}]


Instruction: I feel like going out for a walk—could you hand me my boots?

{"steps": 10, "actions": ["Move(right, 0.25)", "Move(forward, 1.0)", "Move(forward, 1.0)", "Move(forward, 0.25)", "Rotate(left, 90)", "Move(forward, 0.5)", "Tilt(down, 30)", "Move(forward, 0.5)", "Pick(Boots)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "Boots", "field": "isPickedUp", "value": true}]


Fill the kettle with coffee, then pick it up and place it on the coffee table

Instruction: I'm about to start work—please fill the kettle with coffee, then pick it up and place it on the coffee table.

{"steps": 4, "actions": ["Manipulate(WateringCan, coffee, fillwithliquid)", "Pick(WateringCan)", "Place(CoffeeTable)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "WateringCan", "state": "isFilledWithLiquid", "value": true, "liquid": "coffee"}, {"type": "object_in_receptacle", "object_type": "WateringCan", "receptacle_type": "TvStand", "value": false}, {"type": "object_in_receptacle", "object_type": "WateringCan", "receptacle_type": "CoffeeTable"}]


Instruction: I need to go to the bank—could you find my credit card and hand it to me if you locate it?

{"steps": 5, "actions": ["Move(left, 1.0)", "Move(forward, 0.5)", "Tilt(down, 30)", "Pick(CreditCard)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "CreditCard", "field": "isPickedUp", "value": true}]


Put my watch on the dresser and grab my laptop so I don’t forget it.

Instruction: I am heading out to work. Could you do me a quick favor? Grab my watch from the side table and set it on the dresser, then bring my laptop along so I do not forget it.

{"steps": 12, "actions": ["Move(right, 0.5)", "Move(forward, 0.5)", "Move(forward, 0.25)", "Pick(Watch)", "Rotate(left, 90)", "Move(left, 0.5)", "Move(forward, 1.0)", "Move(forward, 1.0)", "Rotate(right, 90)", "Place(Dresser)", "Pick(Laptop)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "Laptop", "state": "isPickedUp", "value": true}, {"type": "object_in_receptacle", "object_type": "Watch", "receptacle_type": "Dresser", "value": true}, {"type": "object_in_receptacle", "object_type": "Watch", "receptacle_type": "SideTable", "value": false}]

["Dresser", "Laptop", "SideTable", "Watch"]


Find my vase and place it on the side table next to the sofa

Instruction: Please find the vase that’s sitting on the shelf, pick it up, and set it on the side table next to the sofa.

{"steps": 15, "actions": ["Tilt(down, 30)", "Move(forward, 0.5)", "Move(forward, 0.25)", "Move(right, 0.25)", "Pick(Vase)", "Rotate(right, 90)", "Rotate(right, 90)", "Move(forward, 1.0)", "Move(forward, 1.0)", "Move(forward, 1.0)", "Move(forward, 0.25)", "Rotate(left, 90)", "Move(forward, 0.5)", "Place(SideTable)", "EndTask(success)"]}

[{"type": "object_in_receptacle", "object_type": "Vase", "receptacle_type": "Shelf", "value": true}, {"type": "object_in_receptacle", "object_type": "Vase", "receptacle_type": "SideTable", "value": true}]


Grab the kettle, fill it up, and set it on the coffee table, then switch the TV on for me

Instruction: I’m ready to really kick back and watch some TV. Could you grab the kettle, fill it up, and set it on the coffee table? Then switch the TV on for me. Nothing beats sipping water while I binge my show.

{"steps": 20, "actions": ["Move(forward, 0.5)", "Move(forward, 0.25)", "Move(right, 0.25)", "Tilt(down, 30)", "Move(forward, 0.25)", "Move(right, 0.25)", "Manipulate(WateringCan, water, fillwithliquid)", "Pick(WateringCan)", "Rotate(left, 90)", "Rotate(left, 90)", "Move(forward, 1.0)", "Place(CoffeeTable)", "Rotate(right, 90)", "Move(forward, 1.0)", "Move(forward, 0.5)", "Move(forward, 0.25)", "Move(right, 1.0)", "Move(right, 0.25)", "ChangeState(Television, on)", "EndTask(success)"]}

[{"type": "object_in_receptacle", "object_type": "WateringCan", "receptacle_type": "Shelf", "value": false}, {"type": "object_in_receptacle", "object_type": "WateringCan", "receptacle_type": "CoffeeTable", "value": true}, {"type": "object_state", "object_type": "WateringCan", "state": "isFilledWithLiquid", "value": true, "liquid": "water"}, {"type": "object_state", "object_type": "Television", "state": "isToggled", "value": true}]

["CoffeeTable", "Shelf", "Television", "WateringCan"]


Put the watch and credit card from the table into the box

Instruction: I'm tidying up—please put the watch and credit card from the table into the box

{"steps": 9, "actions": ["Tilt(down, 30)", "Move(forward, 0.5)", "Move(forward, 0.25)", "Move(left, 0.25)", "Pick(Watch)", "Place(Box)", "Pick(CreditCard)", "Place(Box)", "EndTask(success)"]}

[{"type": "object_in_receptacle", "object_type": "Watch", "receptacle_type": "SideTable", "value": false}, {"type": "object_in_receptacle", "object_type": "CreditCard", "receptacle_type": "SideTable", "value": false}, {"type": "object_in_receptacle", "object_type": "CreditCard", "receptacle_type": "Box", "value": true}, {"type": "object_in_receptacle", "object_type": "Watch", "receptacle_type": "Box", "value": true}]

["Box", "CreditCard", "SideTable", "SideTable ", "Watch"]


Pick up the remote and place it on the coffee table

Instruction: I'm looking for the remote. Could you look for the remote on the dresser and then set it on the coffee table in front of the TV?

{"steps": 10, "actions": ["Move(forward, 0.25)", "Tilt(down, 30)", "Move(forward, 0.25)", "Pick(RemoteControl)", "Rotate(right, 90)", "Move(right, 0.25)", "Move(forward, 0.5)", "Move(forward, 0.25)", "Place(CoffeeTable)", "EndTask(success)"]}

[{"type": "object_in_receptacle", "object_type": "RemoteControl", "receptacle_type": "Sofa", "value": false}, {"type": "object_in_receptacle", "object_type": "RemoteControl", "receptacle_type": "CoffeeTable", "value": true}]


Turn off all the lights in the room, including the one over the dining table and the floor lamp

Instruction: I'm getting sleepy—could you turn off all the lights in the room for me, including the one over the dining table and the floor lamp? Thanks.

{"steps": 14, "actions": ["Move(forward, 0.5)", "Rotate(right, 90)", "Move(left, 0.5)", "Move(forward, 0.5)", "Move(forward, 0.25)", "ChangeState(LightSwitch, off)", "Rotate(left, 90)", "Rotate(left, 90)", "Move(forward, 1.0)", "Move(left, 1.0)", "Move(forward, 1.0)", "Move(forward, 0.5)", "ChangeState(FloorLamp, off)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "LightSwitch", "field": "isToggled", "value": false}, {"type": "object_state", "object_type": "FloorLamp", "field": "isToggled", "value": false}]


Grab my phone from the dining table and bring it to the sofa

Instruction: Could you grab my phone from the dining table and bring it to the sofa? I'm feeling a bit lazy and don't want to get up.

{"steps": 21, "actions": ["Move(forward, 0.5)", "Move(left, 0.5)", "Pick(CellPhone)", "Rotate(right, 90)", "Rotate(right, 90)", "Move(left, 0.25)", "Move(forward, 1.0)", "Move(forward, 0.5)", "Move(left, 0.25)", "Move(forward, 1.0)", "Move(forward, 0.5)", "Rotate(right, 90)", "Move(forward, 0.25)", "Move(left, 0.5)", "Move(forward, 1.0)", "Rotate(right, 90)", "Move(left, 0.5)", "Tilt(down, 30)", "Move(forward, 0.5)", "Place(Sofa)", "EndTask(success)"]}

[{"type": "object_in_receptacle", "object_type": "CellPhone", "receptacle_type": "DiningTable", "value": false}, {"type": "object_in_receptacle", "object_type": "CellPhone", "receptacle_type": "Sofa", "value": true}]


Please fill the watering can with wine and place it on the dining table.

Instruction: I'd like some wine with dinner—please find my watering can, fill it with wine, and place it on the dining table

{"steps": 18, "actions": ["Move(forward, 1.0)", "Move(left, 0.25)", "Move(forward, 0.25)", "Manipulate(WateringCan, wine, fillwithliquid)", "Pick(WateringCan)", "Rotate(right, 90)", "Rotate(right, 90)", "Move(forward, 1.0)", "Move(forward, 1.0)", "Move(forward, 0.25)", "Rotate(left, 90)", "Move(forward, 1.0)", "Move(forward, 1.0)", "Move(forward, 1.0)", "Move(forward, 1.0)", "Move(forward, 0.25)", "Place(DiningTable)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "WateringCan", "field": "isFilledWithLiquid", "value": true, "liquid": "wine"}, {"type": "object_in_receptacle", "object_type": "WateringCan", "receptacle_type": "DiningTable", "value": true}]


Instruction: Could you please take my phone from the dining table and put it on the chair? I left it there after finishing my meal.

{"steps": 13, "actions": ["Move(left, 0.5)", "Move(forward, 0.5)", "Pick(CellPhone)", "Rotate(right, 90)", "Rotate(right, 90)", "Move(left, 1.0)", "Move(forward, 1.0)", "Move(forward, 1.0)", "Move(forward, 1.0)", "Tilt(down, 30)", "Move(forward, 0.25)", "Place(Chair)", "EndTask(success)"]}

[{"type": "object_in_receptacle", "object_type": "CellPhone", "receptacle_type": "DiningTable", "value": false}, {"type": "object_in_receptacle", "object_type": "CellPhone", "receptacle_type": "Chair", "value": true}]


Instruction: Can you find my dish sponge? If you do, please put it in the sink—I want to wash the dishes

{"steps": 6, "actions": ["Move(forward, 0.5)", "Pick(DishSponge)", "Rotate(right, 90)", "Move(right, 0.5)", "Place(Sink)", "EndTask(success)"]}

[{"type": "object_in_receptacle", "object_type": "DishSponge", "receptacle_type": "SideTable", "value": false}, {"type": "object_in_receptacle", "object_type": "DishSponge", "receptacle_type": "Sink", "value": true}]


Turn on the faucet and get the cloth dirty.

Instruction: I’m doing a deep clean. First, turn on the faucet; second, get the cloth dirty.

{"steps": 8, "actions": ["Move(forward, 0.5)", "ChangeState(Faucet, on)", "Rotate(right, 90)", "Move(forward, 0.25)", "Tilt(down, 30)", "Move(forward, 0.25)", "DirtyObject(Cloth)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "Faucet", "field": "isToggled", "value": true}, {"type": "object_state", "object_type": "Cloth", "state": "isDirty", "value": true}]


Please open the shower door, turn on the faucet, and hand me the towel for drying off

Instruction: I'm going to step into the shower—please open the door for me, turn on the faucet, and hand me the towel for drying off

{"steps": 20, "actions": ["Move(forward, 1.0)", "Move(forward, 0.25)", "Move(right, 0.5)", "Manipulate(ShowerDoor, open)", "Move(right, 0.25)", "Move(forward, 0.25)", "Move(right, 0.25)", "Move(forward, 0.5)", "Rotate(left, 90)", "Move(right, 0.25)", "Move(forward, 0.25)", "Move(right, 0.25)", "Move(forward, 0.5)", "Move(forward, 0.25)", "ChangeState(ShowerHead, on)", "Rotate(right, 90)", "Rotate(right, 90)", "Move(forward, 0.5)", "Pick(Towel)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "ShowerDoor", "field": "isOpen", "value": true}, {"type": "object_state", "object_type": "ShowerHead", "field": "isToggled", "value": true}, {"type": "object_state", "object_type": "Towel", "field": "isPickedUp", "value": true}]


Take the cloth from counter top and put it on the side table

Instruction: I left the cloth on the counter. Could you take it to the side table by the sink?

{"steps": 9, "actions": ["Move(forward, 1.0)", "Move(forward, 0.25)", "Pick(Cloth)", "Rotate(right, 90)", "Move(right, 1.0)", "Move(right, 1.0)", "Move(forward, 1.0)", "Place(SideTable)", "EndTask(success)"]}

[{"type": "object_in_receptacle", "object_type": "Cloth", "receptacle_type": "CounterTop", "value": false}, {"type": "object_in_receptacle", "object_type": "Cloth", "receptacle_type": "SideTable", "value": true}]


Take the tissues from counter top and put them on the side table

Instruction: We are out of paper by the sink. Could you move the tissues from the counter top to the side table next to it?

{"steps": 14, "actions": ["Move(forward, 1.0)", "Move(forward, 0.5)", "Move(left, 0.5)", "Move(left, 0.25)", "Pick(TissueBox)", "Rotate(right, 90)", "Move(right, 1.0)", "Move(forward, 0.5)", "Move(forward, 0.25)", "Move(right, 1.0)", "Move(forward, 1.0)", "Move(right, 0.25)", "Place(SideTable)", "EndTask(success)"]}

[{"type": "object_in_receptacle", "object_type": "TissueBox", "receptacle_type": "CounterTop", "value": false}, {"type": "object_in_receptacle", "object_type": "TissueBox", "receptacle_type": "SideTable", "value": true}]


Instruction: I don't need this rag anymore. Please throw it in the trash.

{"steps": 12, "actions": ["Move(left, 0.5)", "Move(left, 0.25)", "Pick(Cloth)", "Move(left, 0.5)", "Move(forward, 0.25)", "Tilt(down, 30)", "Rotate(left, 90)", "Move(forward, 0.25)", "Rotate(right, 90)", "Move(forward, 0.25)", "Place(GarbageCan)", "EndTask(success)"]}

[{"type": "object_in_receptacle", "object_type": "Cloth", "receptacle_type": "CounterTop", "value": false}, {"type": "object_in_receptacle", "object_type": "Cloth", "receptacle_type": "GarbageCan"}]


Instruction: I want to wash my hands—could you put the soap into the sink for me?

{"steps": 5, "actions": ["Move(right, 0.5)", "Move(forward, 0.25)", "Pick(SoapBar)", "Place(SinkBasin)", "EndTask(success)"]}

[{"type": "object_in_receptacle", "object_type": "SoapBar", "receptacle_type": "SinkBasin", "value": true}]


Instruction: My soap bottle is empty. Could you throw it in the trash for me?

{"steps": 17, "actions": ["Move(right, 0.5)", "Move(forward, 0.25)", "Pick(SoapBottle)", "Rotate(left, 90)", "Move(left, 0.5)", "Move(left, 0.25)", "Move(forward, 0.5)", "Move(left, 0.25)", "Move(forward, 0.5)", "Move(forward, 0.25)", "Move(left, 0.25)", "Move(forward, 0.5)", "Move(forward, 0.25)", "Tilt(down, 30)", "Move(forward, 0.25)", "Place(GarbageCan)", "EndTask(success)"]}

[{"type": "object_in_receptacle", "object_type": "SoapBottle", "receptacle_type": "GarbageCan", "value": true}]


Please throw the soap in the trash can next to the toilet

Instruction: This bar of soap is no longer useful. I'd like to throw it in the trash can next to the toilet

{"steps": 12, "actions": ["Move(forward, 0.5)", "Pick(SoapBar)", "Move(left, 0.5)", "Move(forward, 1.0)", "Move(forward, 0.25)", "Rotate(left, 90)", "Move(right, 0.5)", "Move(right, 0.25)", "Tilt(down, 30)", "Move(forward, 0.5)", "Place(GarbageCan)", "EndTask(success)"]}

[{"type": "object_in_receptacle", "object_type": "SoapBar", "receptacle_type": "GarbageCan"}]


Please go over and take the toilet paper from the trash can next to the toilet and place it on the counter top

Instruction: I accidentally threw the paper into the trash can next to the toilet. Could you go over, take it out, and place it on the counter top for me?

{"steps": 14, "actions": ["Move(left, 0.25)", "Move(forward, 1.0)", "Move(forward, 1.0)", "Move(forward, 0.25)", "Rotate(left, 90)", "Move(forward, 0.25)", "Tilt(down, 30)", "Move(forward, 0.5)", "Pick(ToiletPaper)", "Rotate(left, 90)", "Rotate(left, 90)", "Move(forward, 0.5)", "Place(CounterTop)", "EndTask(success)"]}

[{"type": "object_in_receptacle", "object_type": "ToiletPaper", "receptacle_type": "GarbageCan", "value": false}, {"type": "object_in_receptacle", "object_type": "ToiletPaper", "receptacle_type": "CounterTop", "value": true}]


Please get the faucet running and grab the toilet paper

Instruction: Can you get the faucet running and grab the toilet paper? I'll use it when I'm done washing up.

{"steps": 8, "actions": ["Move(left, 0.25)", "Move(forward, 1.0)", "Move(forward, 0.5)", "Tilt(down, 30)", "Rotate(right, 90)", "ChangeState(Faucet, on)", "Pick(ToiletPaper)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "Faucet", "field": "isToggled", "value": true}, {"type": "object_state", "object_type": "ToiletPaper", "field": "isPickedUp", "value": true}]


Please get the candle from under the counter and place it up top, then light it

Instruction: I'm feeling chilly after my shower. First, could you get the candle from under the counter and place it up top? Then, go ahead and light it, please

{"steps": 11, "actions": ["Move(forward, 1.0)", "Move(forward, 0.5)", "Move(forward, 0.25)", "Move(right, 0.25)", "Rotate(right, 90)", "Tilt(down, 30)", "Pick(Candle)", "Move(forward, 0.25)", "Place(CounterTop)", "ChangeState(Candle, on)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "Candle", "field": "isToggled", "value": true}, {"type": "object_in_receptacle", "object_type": "Candle", "receptacle_type": "CounterTop"}]


Instruction: I think I left my towel in the shower. Would you mind opening the shower door and stepping in to get it for me?

{"steps": 13, "actions": ["Move(forward, 1.0)", "Rotate(right, 90)", "Manipulate(ShowerDoor, open)", "Move(left, 0.25)", "Move(forward, 1.0)", "Move(forward, 0.25)", "Rotate(right, 90)", "Move(forward, 1.0)", "Move(forward, 0.5)", "Move(forward, 0.25)", "Move(left, 0.25)", "Pick(Cloth)", "EndTask(success)"]}

[{"type": "object_state", "object_type": "Cloth", "field": "isPickedUp", "value": true}, {"type": "object_state", "object_type": "ShowerDoor", "field": "isOpen", "value": true}]


Instruction: I'm getting ready to draw. Could you take a pencil from the drawer next to the bed and place the pencil on the desk for me?

{"steps": 17, "actions": ["Move(forward, 1.0)", "Move(forward, 1.0)", "Rotate(right, 90)", "Move(forward, 1.0)", "Move(forward, 0.25)", "Pick(Pencil)", "Rotate(left, 90)", "Rotate(left, 90)", "Move(forward, 1.0)", "Move(forward, 0.25)", "Move(left, 0.25)", "Move(forward, 1.0)", "Move(forward, 0.25)", "Rotate(right, 90)", "Move(forward, 0.5)", "Place(Desk)", "EndTask(success)"]}

[{"type": "object_in_receptacle", "object_type": "Pencil", "receptacle_type": "Desk"}, {"type": "object_in_receptacle", "object_type": "Pencil", "receptacle_type": "Drawer", "value": false}]


Instruction: Walk forward to reach the city lamp.

{"steps": 11, "actions": ["Rotate(right, 90)", "Rotate(right, 90)", "Rotate(right, 90)", "Rotate(right, 90)", "Rotate(right, 90)", "Rotate(right, 90)", "Rotate(right, 90)", "Rotate(right, 90)", "Move(forward, 10.0)", "Move(forward, 3.0)", "EndTask(success)"]}


Instruction: Walk to the position of the large tree ahead on the right. You can turn and move in any direction.

{"steps": 10, "actions": ["Move(forward, 10.0)", "Move(forward, 10.0)", "Rotate(right, 90)", "Rotate(right, 90)", "Move(forward, 10.0)", "Move(forward, 10.0)", "Rotate(left, 90)", "Rotate(left, 90)", "Rotate(left, 90)", "EndTask(success)"]}


Instruction: Walk to the newsstand next to the bus stop. You can turn and move in any direction.

{"steps": 8, "actions": ["Move(forward, 10.0)", "Move(forward, 10.0)", "Move(forward, 10.0)", "Rotate(right, 90)", "Rotate(right, 90)", "Move(forward, 10.0)", "Move(forward, 3.0)", "EndTask(success)"]}


Instruction: Walk to the streetlight in front of the gas station. You can turn and move in any direction.

{"steps": 6, "actions": ["Move(forward, 10.0)", "Move(forward, 10.0)", "Move(forward, 10.0)", "Rotate(right, 90)", "Move(forward, 10.0)", "EndTask(success)"]}


Instruction: Walk to the church. You can turn and move in any direction.

{"steps": 13, "actions": ["Move(forward, 10.0)", "Rotate(right, 90)", "Rotate(right, 90)", "Rotate(right, 90)", "Move(forward, 10.0)", "Move(forward, 10.0)", "Rotate(left, 90)", "Rotate(left, 90)", "Rotate(left, 90)", "Move(forward, 10.0)", "Move(forward, 10.0)", "Move(forward, 10.0)", "EndTask(success)"]}


Instruction: Walk to the yellow shop next to the trash can. You can turn and move in any direction.

{"steps": 9, "actions": ["Move(forward, 10.0)", "Rotate(left, 90)", "Rotate(left, 90)", "Move(forward, 3.0)", "Rotate(left, 90)", "Move(forward, 10.0)", "Move(forward, 3.0)", "Move(forward, 3.0)", "EndTask(success)"]}


Instruction: Walk to the position marked by the red line in the screenshot.You can turn and move in any direction.

{"steps": 9, "actions": ["Rotate(left, 90)", "Rotate(left, 90)", "Move(forward, 10.0)", "Move(forward, 10.0)", "Move(forward, 10.0)", "Rotate(left, 90)", "Move(forward, 3.0)", "Move(forward, 3.0)", "EndTask(success)"]}

{"type": "distance_to_waypoint", "target_location": [168.8, 229.7, 1.1], "threshold_label": "medium"}


Instruction: Walk to the position marked by the red line in the screenshot.You can turn and move in any direction.

{"steps": 7, "actions": ["Move(forward, 10.0)", "Move(forward, 10.0)", "Move(forward, 10.0)", "Rotate(left, 90)", "Move(forward, 10.0)", "Move(forward, 10.0)", "EndTask(success)"]}

{"type": "distance_to_waypoint", "target_location": [57.2, 250.2, 1.3], "threshold_label": "medium"}


Instruction: Walk to the position marked by the red line in the screenshot.You can turn and move in any direction.

{"steps": 15, "actions": ["Rotate(left, 90)", "Move(forward, 10.0)", "Move(forward, 10.0)", "Move(forward, 10.0)", "Rotate(left, 90)", "Move(forward, 10.0)", "Rotate(left, 90)", "Move(forward, 10.0)", "Rotate(left, 90)", "Move(forward, 10.0)", "Move(forward, 10.0)", "Move(forward, 10.0)", "Rotate(right, 90)", "Move(forward, 10.0)", "EndTask(success)"]}

{"type": "distance_to_waypoint", "target_location": [163.0, 113.2, 1.3], "threshold_label": "medium"}


Instruction: Walk to the position marked by the red line in the screenshot.You can turn and move in any direction.

{"steps": 10, "actions": ["Rotate(left, 90)", "Move(forward, 10.0)", "Rotate(left, 90)", "Rotate(left, 90)", "Move(forward, 10.0)", "Rotate(left, 90)", "Rotate(left, 90)", "Move(forward, 10.0)", "Move(forward, 10.0)", "EndTask(success)"]}

{"type": "distance_to_waypoint", "target_location": [37.3, -97.1, 1.0], "threshold_label": "medium"}


Instruction: Walk to the position marked by the red line in the screenshot.You can turn and move in any direction.

{"steps": 15, "actions": ["Rotate(left, 90)", "Rotate(left, 90)", "Move(forward, 10.0)", "Rotate(left, 90)", "Move(forward, 10.0)", "Move(forward, 3.0)", "Move(forward, 3.0)", "Rotate(right, 90)", "Rotate(right, 90)", "Rotate(right, 90)", "Move(forward, 10.0)", "Move(forward, 10.0)", "Move(forward, 3.0)", "Move(forward, 3.0)", "EndTask(success)"]}

{"type": "distance_to_waypoint", "target_location": [58.5, -38.1, 1.1], "threshold_label": "medium"}


Instruction: Walk to the house with the yellow roof.You can turn and move in any direction.

{"steps": 9, "actions": ["Rotate(left, 90)", "Move(forward, 10.0)", "Rotate(left, 90)", "Rotate(left, 90)", "Move(forward, 10.0)", "Move(forward, 10.0)", "Rotate(left, 90)", "Move(forward, 10.0)", "EndTask(success)"]}


Instruction: Walk to the ResidentialBuildingRoundCorners.You can turn and move in any direction.

{"steps": 11, "actions": ["Rotate(left, 90)", "Rotate(left, 90)", "Move(forward, 10.0)", "Rotate(left, 90)", "Move(forward, 10.0)", "Move(forward, 10.0)", "Move(forward, 10.0)", "Rotate(right, 90)", "Rotate(right, 90)", "Move(forward, 10.0)", "EndTask(success)"]}


Instruction: Walk to the HighwayLight_multiple.You can turn and move in any direction.

{"steps": 14, "actions": ["Rotate(left, 90)", "Rotate(left, 90)", "Rotate(left, 90)", "Move(forward, 10.0)", "Move(forward, 10.0)", "Move(forward, 10.0)", "Move(forward, 10.0)", "Move(forward, 10.0)", "Rotate(left, 90)", "Rotate(left, 90)", "Move(forward, 3.0)", "Move(forward, 3.0)", "Move(forward, 3.0)", "EndTask(success)"]}


Instruction: Walk to under the billboard.You can turn and move in any direction.

{"steps": 13, "actions": ["Move(forward, 10.0)", "Rotate(left, 90)", "Rotate(left, 90)", "Move(forward, 10.0)", "Move(forward, 10.0)", "Rotate(left, 90)", "Move(forward, 10.0)", "Move(forward, 3.0)", "Move(forward, 3.0)", "Rotate(right, 90)", "Move(forward, 3.0)", "Move(forward, 3.0)", "EndTask(success)"]}


Instruction: Walk to the streetlight near the stone.You can turn and move in any direction.

{"steps": 12, "actions": ["Rotate(right, 90)", "Rotate(right, 90)", "Move(forward, 10.0)", "Rotate(right, 90)", "Move(forward, 10.0)", "Move(forward, 10.0)", "Move(forward, 10.0)", "Move(forward, 10.0)", "Rotate(right, 90)", "Rotate(right, 90)", "Rotate(right, 90)", "EndTask(success)"]}


Instruction: Walk to the position marked by the red line in the screenshot.You can turn and move in any direction.

{"steps": 10, "actions": ["Move(forward, 10.0)", "Rotate(right, 90)", "Rotate(right, 90)", "Move(forward, 10.0)", "Move(forward, 10.0)", "Move(forward, 10.0)", "Rotate(right, 90)", "Move(forward, 10.0)", "Move(forward, 10.0)", "EndTask(success)"]}

{"type": "distance_to_waypoint", "target_location": [41.9, 32.9, 1.2], "threshold_label": "medium"}


Instruction: Walk to the position marked by the red line in the screenshot.You can turn and move in any direction.

{"steps": 11, "actions": ["Rotate(left, 90)", "Move(forward, 10.0)", "Rotate(left, 90)", "Rotate(left, 90)", "Move(forward, 10.0)", "Move(forward, 10.0)", "Rotate(left, 90)", "Rotate(left, 90)", "Rotate(left, 90)", "Move(forward, 10.0)", "EndTask(success)"]}

{"type": "distance_to_waypoint", "target_location": [162.3, 210.9, 1.3], "threshold_label": "medium"}


Instruction: Walk to the position marked by the red line in the screenshot.You can turn and move in any direction.

{"steps": 18, "actions": ["Move(forward, 10.0)", "Move(forward, 10.0)", "Move(forward, 10.0)", "Move(forward, 10.0)", "Move(forward, 10.0)", "Move(forward, 10.0)", "Move(forward, 10.0)", "Rotate(left, 90)", "Rotate(left, 90)", "Rotate(left, 90)", "Move(forward, 10.0)", "Move(forward, 10.0)", "Move(forward, 10.0)", "Rotate(right, 90)", "Rotate(right, 90)", "Rotate(right, 90)", "Move(forward, 10.0)", "EndTask(success)"]}

{"type": "distance_to_waypoint", "target_location": [116.7, -88.5, 9.1], "threshold_label": "medium"}


Instruction: Walk to the position marked by the red line in the screenshot.You can turn and move in any direction.

{"steps": 10, "actions": ["Move(forward, 10.0)", "Move(forward, 10.0)", "Rotate(left, 90)", "Move(forward, 10.0)", "Rotate(left, 90)", "Move(forward, 10.0)", "Move(forward, 10.0)", "Rotate(left, 90)", "Move(forward, 10.0)", "EndTask(success)"]}

{"type": "distance_to_waypoint", "target_location": [235.4, -123.9, 1.0], "threshold_label": "medium"}