<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the leftmost green block.
 {'width': 756, 'height': 660, 'annotations': [{'id': '8f941e4d-8e6b-4e37-a2cd-f19b3dbb1dda', 'keypoint': {'x': 293.1651, 'y': 452.9717}, 'name': 'orange'}, {'id': '77f8b560-0c54-4923-9c65-8e8973d5cc1a', 'keypoint': {'x': 293.9434, 'y': 485.6604}, 'name': 'orange'}, {'id': 'c2fdf254-12fa-43ee-9d04-fd097d6a2fad', 'keypoint': {'x': 332.8585, 'y': 517.5708}, 'name': 'orange'}, {'id': 'ee022624-310e-4867-8360-cd61e6a5e8ec', 'keypoint': {'x': 325.0755, 'y': 557.2642}, 'name': 'orange'}, {'id': 'ca78545f-1817-447e-a767-1ec186e40025', 'keypoint': {'x': 377.2217, 'y': 478.6557}, 'name': 'orange'}, {'id': 'b7ea891b-3095-4df8-a718-b193f34c851c', 'keypoint': {'x': 386.5613, 'y': 492.6651}, 'name': 'orange'}, {'id': '54b85951-9b7d-4a22-9b94-5856c8f4cc7e', 'keypoint': {'x': 441.8208, 'y': 499.6698}, 'name': 'orange'}, {'id': 'ef2b7cbb-fa7e-4894-8e50-310d43a6f3eb', 'keypoint': {'x': 385.0047, 'y': 448.3019}, 'name': 'yellow'}, {'id': 'ab7d8b08-4eaf-41cf-8066-9aba376bb546', 'keypoint': {'x': 392.0094, 'y': 523.0189}, 'name': 'yellow'}, {'id': 'aec059f3-2def-4458-9ea4-3fa5916cab5b', 'keypoint': {'x': 466.7264, 'y': 442.8538}, 'name': 'yellow'}, {'id': '39fe094d-1e93-4828-ae7e-d672aeb88451', 'keypoint': {'x': 356.2075, 'y': 533.9151}, 'name': 'blue'}, {'id': '57cde523-a685-45a5-b623-1b11057840da', 'keypoint': {'x': 487.7406, 'y': 488.7736}, 'name': 'blue'}, {'id': 'da91e8aa-4087-4689-95ed-80e4ba9a93ec', 'keypoint': {'x': 327.4104, 'y': 472.4292}, 'name': 'green'}, {'id': '9406ceb5-4de6-4915-8da6-132b32721d7c', 'keypoint': {'x': 508.7547, 'y': 511.3443}, 'name': 'green'}, {'id': 'fd93334e-be71-4598-9c11-abb001661b87', 'keypoint': {'x': 497.0802, 'y': 560.3774}, 'name': 'green'}, {'id': '3067ae97-d698-4a21-82f6-e94a33ed51a0', 'keypoint': {'x': 380.3349, 'y': 59.1509}, 'name': 'you'}]}
[/INST]
{'x': 508.7547, 'y': 511.3443}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:I would ask them to pick up the block in front of them to their top left corner.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'f8ce6e33-d9e3-4687-bd11-4537937b3def', 'keypoint': {'x': 313.4009, 'y': 480.9906}, 'name': 'orange'}, {'id': '8758b8e0-78a7-4dc2-b595-9eec97d96b3c', 'keypoint': {'x': 403.684, 'y': 466.9811}, 'name': 'orange'}, {'id': '9024f068-b46e-400e-bf69-507f61b74f80', 'keypoint': {'x': 459.7217, 'y': 452.9717}, 'name': 'orange'}, {'id': 'd7ab991d-e3d6-4ffb-8d9b-9cffbb59b9c6', 'keypoint': {'x': 502.5283, 'y': 457.6415}, 'name': 'orange'}, {'id': 'b9fb860f-d2d0-4b4a-a275-416d46a99ab8', 'keypoint': {'x': 335.9717, 'y': 450.6368}, 'name': 'yellow'}, {'id': '7404a736-3ae3-4f5c-b40d-5d5af4635c6e', 'keypoint': {'x': 316.5142, 'y': 531.5802}, 'name': 'yellow'}, {'id': 'd2d6b9ff-fd85-48a1-b368-3f0e1d09cc6f', 'keypoint': {'x': 396.6792, 'y': 491.8868}, 'name': 'yellow'}, {'id': 'c4551bcd-78f8-4205-b828-5b040d49a2c3', 'keypoint': {'x': 444.1557, 'y': 433.5142}, 'name': 'yellow'}, {'id': '3e0cda0a-1e8a-4472-8984-7886c02ff560', 'keypoint': {'x': 347.6462, 'y': 514.4575}, 'name': 'blue'}, {'id': '266b6be1-582b-4c4d-8544-906fad90cbd2', 'keypoint': {'x': 359.3208, 'y': 478.6557}, 'name': 'blue'}, {'id': 'c58796fa-46b4-4231-8589-0612cd52983f', 'keypoint': {'x': 395.9009, 'y': 444.4104}, 'name': 'blue'}, {'id': '3de58fe6-02ae-4e5a-b0eb-ac1e3ac60960', 'keypoint': {'x': 444.934, 'y': 537.8066}, 'name': 'blue'}, {'id': 'a1a96c53-7f31-4121-8983-bf6a69f42935', 'keypoint': {'x': 502.5283, 'y': 541.6981}, 'name': 'blue'}, {'id': 'c2e223f1-9da7-4029-ab29-81edb6be3c94', 'keypoint': {'x': 355.4292, 'y': 424.9528}, 'name': 'green'}, {'id': 'd1f0facf-050a-4a92-bfb1-1367c5fdd80d', 'keypoint': {'x': 466.7264, 'y': 500.4481}, 'name': 'green'}, {'id': '727b2df8-1454-47db-9eff-8cfe072974ff', 'keypoint': {'x': 383.4481, 'y': 44.3632}, 'name': 'you'}]}
[/INST]
{'x': 466.7264, 'y': 500.4481}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:There is a blue block on the table that is positioned to your farthest right. Please pick up the yellow block directly behind, and closest to, that blue block.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'b328ac50-08b5-486e-a937-2cddbc04245c', 'keypoint': {'x': 304.8396, 'y': 434.2925}, 'name': 'orange'}, {'id': '7344b92c-5683-4670-9cef-33f4cf8e73fb', 'keypoint': {'x': 472.9528, 'y': 456.0849}, 'name': 'orange'}, {'id': '6958bbb7-3dce-4cfe-9049-02ebc5a32e43', 'keypoint': {'x': 306.3962, 'y': 460.7547}, 'name': 'yellow'}, {'id': '62285f29-522c-498a-9fdb-53e2e8421ea5', 'keypoint': {'x': 330.5236, 'y': 470.8726}, 'name': 'yellow'}, {'id': 'e4402e53-f01e-4254-a874-f6ffd3892ac4', 'keypoint': {'x': 381.8915, 'y': 475.5425}, 'name': 'yellow'}, {'id': '3f3ed2fa-5312-418f-9f29-f4750a72aa9e', 'keypoint': {'x': 402.9057, 'y': 435.0708}, 'name': 'yellow'}, {'id': 'fe5ba8f4-1f70-4712-ae25-cd20ac9670d6', 'keypoint': {'x': 455.8302, 'y': 481.7689}, 'name': 'yellow'}, {'id': 'bd1c4ddb-8345-4f24-a982-f6cd6a64fb0e', 'keypoint': {'x': 501.75, 'y': 505.8962}, 'name': 'yellow'}, {'id': '8f332be0-ad10-4130-9840-f3a1dc330890', 'keypoint': {'x': 352.316, 'y': 513.6792}, 'name': 'yellow'}, {'id': '5b19adbf-7efb-4a90-9f40-4c0bf2f3b9a8', 'keypoint': {'x': 324.2972, 'y': 499.6698}, 'name': 'yellow'}, {'id': '7f124547-f8b7-4280-b7e1-401876ce2bdb', 'keypoint': {'x': 308.7311, 'y': 552.5943}, 'name': 'yellow'}, {'id': '8a1fd773-bc7c-4be5-a292-c1c53d5c77df', 'keypoint': {'x': 298.6132, 'y': 491.8868}, 'name': 'blue'}, {'id': '7be8714d-676d-466e-a70a-e9d078b757f4', 'keypoint': {'x': 381.1132, 'y': 497.3349}, 'name': 'blue'}, {'id': 'b1ee1721-f1d2-4cc8-828b-2e6c5072d54f', 'keypoint': {'x': 451.9387, 'y': 432.7358}, 'name': 'blue'}, {'id': '0416eebd-320e-40d8-bbc7-a94a6333aa95', 'keypoint': {'x': 497.0802, 'y': 485.6604}, 'name': 'green'}, {'id': '2240b1ab-d515-4872-a587-09ca836dc397', 'keypoint': {'x': 378.7783, 'y': 28.7972}, 'name': 'you'}]}
[/INST]
{'x': 324.2972, 'y': 499.6698}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the rightmost blue block.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'bb2508d6-bab0-4991-a4b4-1d60addd89be', 'keypoint': {'x': 374.8868, 'y': 32.6887}, 'name': 'you'}, {'id': '80bad823-3c11-42a6-9b7a-8f92fd053db8', 'keypoint': {'x': 286.1604, 'y': 504.3396}, 'name': 'yellow'}, {'id': '74407b61-1d02-4456-a3be-f44e6d0621e4', 'keypoint': {'x': 427.8113, 'y': 511.3443}, 'name': 'yellow'}, {'id': '0c043462-b620-40d3-b17e-3f17c7108248', 'keypoint': {'x': 391.2311, 'y': 435.8491}, 'name': 'green'}, {'id': 'e97e7600-f671-4774-970c-bcd8e011471e', 'keypoint': {'x': 311.066, 'y': 530.8019}, 'name': 'green'}, {'id': '3c45bdee-a533-4cab-8e50-cd3e05ea379a', 'keypoint': {'x': 375.6651, 'y': 518.3491}, 'name': 'green'}, {'id': 'fbe906a6-d83a-45bc-bd1e-7c81ccde8b9b', 'keypoint': {'x': 487.7406, 'y': 473.2075}, 'name': 'green'}, {'id': '662455ed-994d-45d8-b864-0f07ccfe9a1e', 'keypoint': {'x': 309.5094, 'y': 477.8774}, 'name': 'blue'}, {'id': '92e619ee-cc43-4ba6-80a0-fd23cabc2b68', 'keypoint': {'x': 378.7783, 'y': 457.6415}, 'name': 'blue'}, {'id': '4d3a4397-37ee-4ea1-bff7-ea4e40e4ba91', 'keypoint': {'x': 395.1226, 'y': 483.3255}, 'name': 'blue'}, {'id': '83c6c716-2497-4e0e-b233-3cda2dca8049', 'keypoint': {'x': 438.7075, 'y': 448.3019}, 'name': 'blue'}, {'id': '24b23d82-8281-40fd-9942-dc852b76fea4', 'keypoint': {'x': 444.1557, 'y': 484.1038}, 'name': 'blue'}, {'id': '541179c1-e37f-4069-99a8-fe58be3053c6', 'keypoint': {'x': 466.7264, 'y': 502.783}, 'name': 'blue'}, {'id': 'd950b6f4-0fff-425b-bf69-419e3a15863f', 'keypoint': {'x': 343.7547, 'y': 543.2547}, 'name': 'orange'}, {'id': 'a5aa3394-3b02-4613-8a7d-f14c9cd0fa58', 'keypoint': {'x': 431.7028, 'y': 539.3632}, 'name': 'orange'}, {'id': 'd7eb7db6-f69d-4df7-861e-0cc9dbebac2a', 'keypoint': {'x': 484.6274, 'y': 532.3585}, 'name': 'orange'}]}
[/INST]
{'x': 309.5094, 'y': 477.8774}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the blue block that is closest to me.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'c7519096-f7ea-4ab1-a66d-97c708976599', 'keypoint': {'x': 380.3349, 'y': 34.2453}, 'name': 'you'}, {'id': 'a95968ee-51d4-482d-ae79-97cb54663aed', 'keypoint': {'x': 293.9434, 'y': 479.434}, 'name': 'blue'}, {'id': '2fc8b299-6074-43eb-92bc-7ed758a958fc', 'keypoint': {'x': 329.7453, 'y': 557.2642}, 'name': 'blue'}, {'id': '11dbc045-de4f-4d3a-bc50-a1c2ba279bd1', 'keypoint': {'x': 366.3255, 'y': 546.3679}, 'name': 'blue'}, {'id': 'a3036737-fb7e-4f86-88c8-b0c5262cc051', 'keypoint': {'x': 379.5566, 'y': 438.9623}, 'name': 'blue'}, {'id': '5370f3f3-ef52-45e8-bdb4-3d96d54f4bd9', 'keypoint': {'x': 455.8302, 'y': 449.0802}, 'name': 'blue'}, {'id': '95294165-9354-41e7-b1be-60d4033bd1a3', 'keypoint': {'x': 318.0708, 'y': 454.5283}, 'name': 'yellow'}, {'id': '71f49ac9-1905-457f-9219-701cac7c42e4', 'keypoint': {'x': 318.0708, 'y': 530.8019}, 'name': 'yellow'}, {'id': '6f7a8a19-794f-4ffc-80ed-8a42dedeab0f', 'keypoint': {'x': 463.6132, 'y': 524.5755}, 'name': 'yellow'}, {'id': '407068c6-6a1e-4a14-b9d0-c0a4ed13b9c2', 'keypoint': {'x': 395.1226, 'y': 466.2028}, 'name': 'green'}, {'id': '847fb0f9-02cc-42df-8984-2b193e2abb8d', 'keypoint': {'x': 403.684, 'y': 523.0189}, 'name': 'green'}, {'id': 'bd78738b-2c39-4b85-9575-960d46ecd66c', 'keypoint': {'x': 325.0755, 'y': 500.4481}, 'name': 'orange'}, {'id': 'dafb3b80-4c1e-48a7-b398-e3aa56c5dcb0', 'keypoint': {'x': 374.1085, 'y': 515.2358}, 'name': 'orange'}, {'id': '36cf3f26-cac5-4368-91eb-ae8937ec2f91', 'keypoint': {'x': 412.2453, 'y': 427.2877}, 'name': 'orange'}, {'id': 'f93ff614-2250-4f4d-8ec0-ac87ec62305c', 'keypoint': {'x': 436.3726, 'y': 437.4057}, 'name': 'orange'}, {'id': '09066da9-88d0-4617-8cf6-7d604ee0190d', 'keypoint': {'x': 485.4057, 'y': 510.566}, 'name': 'orange'}]}
[/INST]
{'x': 329.7453, 'y': 557.2642}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the orange block that is furthest from you and closest to me. There are no other blocks near this orange block.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'b1c926ae-92c1-43eb-96c5-76c41d0fab06', 'keypoint': {'x': 342.1981, 'y': 437.4057}, 'name': 'orange'}, {'id': '76267614-95a1-4ded-b4e9-354b821f62bb', 'keypoint': {'x': 314.1792, 'y': 499.6698}, 'name': 'orange'}, {'id': '3124e476-176a-4feb-8844-b3973d48973a', 'keypoint': {'x': 453.4953, 'y': 463.8679}, 'name': 'orange'}, {'id': '728bd5ee-ece2-45c9-b0d7-b887692bb453', 'keypoint': {'x': 494.7453, 'y': 534.6934}, 'name': 'orange'}, {'id': 'da0db0c2-fdee-4550-8e2a-11f1aace764f', 'keypoint': {'x': 376.4434, 'y': 419.5047}, 'name': 'yellow'}, {'id': 'bd27285d-35e5-47b5-9b1e-78bd5fb9e595', 'keypoint': {'x': 360.0991, 'y': 516.7925}, 'name': 'yellow'}, {'id': '5a97f3d8-463a-4c65-9906-b8cf5f772e2b', 'keypoint': {'x': 408.3538, 'y': 526.9104}, 'name': 'yellow'}, {'id': 'f2444027-7c76-40d2-9424-36c46342a49f', 'keypoint': {'x': 318.8491, 'y': 449.0802}, 'name': 'blue'}, {'id': '0b2e3fca-8a71-43d2-a83e-f02af09891b9', 'keypoint': {'x': 315.7358, 'y': 480.2123}, 'name': 'blue'}, {'id': '897e21a9-0e62-42b6-8c01-826ad479026c', 'keypoint': {'x': 378.0, 'y': 490.3302}, 'name': 'blue'}, {'id': 'ca6c0f72-0600-47ba-b4b1-05dc3566a83e', 'keypoint': {'x': 402.1274, 'y': 463.0896}, 'name': 'blue'}, {'id': '41397d83-cab1-4c4d-ba20-b13a1c93f043', 'keypoint': {'x': 419.25, 'y': 432.7358}, 'name': 'blue'}, {'id': '051bcea0-383b-4791-9652-d84f6ef3fe4c', 'keypoint': {'x': 348.4245, 'y': 488.7736}, 'name': 'green'}, {'id': '58f619eb-2384-4a80-97f9-10520e5226a9', 'keypoint': {'x': 311.8443, 'y': 539.3632}, 'name': 'green'}, {'id': 'b9c15fa4-b07d-4f16-9f7d-f007698c83c2', 'keypoint': {'x': 417.6934, 'y': 502.0047}, 'name': 'green'}, {'id': '5c0026dd-e833-4ad7-8339-361bad3686e0', 'keypoint': {'x': 378.0, 'y': 59.1509}, 'name': 'you'}]}
[/INST]
{'x': 494.7453, 'y': 534.6934}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the green block in the middle of the five green block on your left.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'd5bcc2b7-ff91-4686-8f50-3a73ffd3beeb', 'keypoint': {'x': 315.7358, 'y': 436.6274}, 'name': 'orange'}, {'id': '70515b86-98d5-4fa6-9118-a0bc484f6de1', 'keypoint': {'x': 313.4009, 'y': 525.3538}, 'name': 'orange'}, {'id': 'b40b0958-f706-45d2-a042-0440eb67121b', 'keypoint': {'x': 380.3349, 'y': 515.2358}, 'name': 'orange'}, {'id': '9dcd051f-bad9-4f84-b653-9a30a72cb545', 'keypoint': {'x': 389.6745, 'y': 552.5943}, 'name': 'orange'}, {'id': 'b0de6329-f4d9-486c-91ed-478f28262cff', 'keypoint': {'x': 363.9906, 'y': 439.7406}, 'name': 'yellow'}, {'id': '10eeedc5-3253-42e7-b401-33c4616e4623', 'keypoint': {'x': 396.6792, 'y': 457.6415}, 'name': 'yellow'}, {'id': 'f7642eee-44ca-4f1d-abf7-a79d43dfbd0d', 'keypoint': {'x': 279.934, 'y': 535.4717}, 'name': 'yellow'}, {'id': 'd87c603d-89da-4c0b-8f61-a496275c1297', 'keypoint': {'x': 311.066, 'y': 487.217}, 'name': 'blue'}, {'id': '2065d0e4-12eb-4ab3-bde4-75688bb3c142', 'keypoint': {'x': 457.3868, 'y': 439.7406}, 'name': 'green'}, {'id': '1550df58-31c4-49f4-8e6d-7db4a57bc294', 'keypoint': {'x': 435.5943, 'y': 445.967}, 'name': 'green'}, {'id': '1990bf50-38ac-43ab-8581-26778cea80ed', 'keypoint': {'x': 458.9434, 'y': 484.8821}, 'name': 'green'}, {'id': '7895d85c-2e57-4008-b9d7-74625608a42c', 'keypoint': {'x': 485.4057, 'y': 515.2358}, 'name': 'green'}, {'id': '991a571a-03aa-4a00-b80d-3d6a0845625b', 'keypoint': {'x': 440.2642, 'y': 514.4575}, 'name': 'green'}, {'id': 'e0e1086c-3a47-42f6-98fd-868d0756cd4d', 'keypoint': {'x': 378.7783, 'y': 495.7783}, 'name': 'green'}, {'id': '3d132917-dd96-4ae9-9302-da261696357f', 'keypoint': {'x': 333.6368, 'y': 505.1179}, 'name': 'green'}, {'id': 'bc7494f1-b049-49c6-bf24-85e942969065', 'keypoint': {'x': 383.4481, 'y': 56.0377}, 'name': 'you'}]}
[/INST]
{'x': 458.9434, 'y': 484.8821}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Grab the 2nd right-most yellow block.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'fe98e63d-c226-4a25-9e75-229c9a2d828a', 'keypoint': {'x': 377.2217, 'y': 28.7972}, 'name': 'you'}, {'id': 'c1bfc6fe-9e0d-4152-aab0-9efe03f99ee1', 'keypoint': {'x': 291.6085, 'y': 487.217}, 'name': 'yellow'}, {'id': 'eefd5787-08ef-4674-ae9e-c41f4a2ed951', 'keypoint': {'x': 314.1792, 'y': 539.3632}, 'name': 'yellow'}, {'id': '8987e25d-8eb0-42ab-aa8e-390a7b046e44', 'keypoint': {'x': 363.2123, 'y': 544.8113}, 'name': 'yellow'}, {'id': 'af54d764-7469-4b39-95f2-d25ac6241715', 'keypoint': {'x': 434.0377, 'y': 542.4764}, 'name': 'yellow'}, {'id': 'c426887c-43fa-4c7c-9ee6-447f48c5b353', 'keypoint': {'x': 463.6132, 'y': 487.9953}, 'name': 'yellow'}, {'id': 'f1402f90-fe95-42c0-93e5-75019494beba', 'keypoint': {'x': 497.0802, 'y': 442.0755}, 'name': 'yellow'}, {'id': 'ad7b2c3b-efac-4705-8dc4-076514da9f0d', 'keypoint': {'x': 319.6274, 'y': 463.8679}, 'name': 'blue'}, {'id': '57fd11a6-5352-4e90-8c5e-78e080a9f63b', 'keypoint': {'x': 458.1651, 'y': 516.0142}, 'name': 'blue'}, {'id': 'bffb87fb-647e-44e1-b4c0-9bbf464cc4da', 'keypoint': {'x': 353.8726, 'y': 445.1887}, 'name': 'green'}, {'id': '2be751d9-34f0-411b-89e7-1b20b054e1ee', 'keypoint': {'x': 388.8962, 'y': 498.1132}, 'name': 'green'}, {'id': '98a41430-e570-492e-969a-8ea45857d765', 'keypoint': {'x': 395.9009, 'y': 533.1368}, 'name': 'green'}, {'id': '0327cb90-c17c-4528-8964-e978fbff5999', 'keypoint': {'x': 427.8113, 'y': 484.8821}, 'name': 'green'}, {'id': '97f91dd8-6cd8-4194-8783-e9e3b5786e68', 'keypoint': {'x': 456.6085, 'y': 452.9717}, 'name': 'green'}, {'id': '4ebadfd4-3497-4f80-a4e9-a870c25786bd', 'keypoint': {'x': 504.8632, 'y': 502.0047}, 'name': 'green'}, {'id': 'd2ff782c-210c-4090-91a5-de43a78f1232', 'keypoint': {'x': 331.3019, 'y': 512.9009}, 'name': 'orange'}]}
[/INST]
{'x': 463.6132, 'y': 487.9953}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the first orange block on your left, toward the top.
 {'width': 756, 'height': 660, 'annotations': [{'id': '82770085-a20a-439c-b1a3-f74a8d8ccdd0', 'keypoint': {'x': 322.0293, 'y': 491.9856}, 'name': 'yellow'}, {'id': '975d2226-07a6-46ec-8aa1-01f44597bb5c', 'keypoint': {'x': 376.8281, 'y': 495.1624}, 'name': 'yellow'}, {'id': '9fbd0dd7-292a-46c1-854e-eb8c4bb880cf', 'keypoint': {'x': 402.2421, 'y': 551.5495}, 'name': 'yellow'}, {'id': 'c2d61b6b-0585-4aef-acaf-17bad430192b', 'keypoint': {'x': 464.9827, 'y': 498.3391}, 'name': 'yellow'}, {'id': '13d3414e-a81b-4dad-8963-6fbda2f0d436', 'keypoint': {'x': 376.0339, 'y': 27.387}, 'name': 'you'}, {'id': '9b80a1cd-fed1-49e9-8b69-3cd0088f7fca', 'keypoint': {'x': 302.9689, 'y': 454.6589}, 'name': 'blue'}, {'id': '17a9c4b0-bf5e-45d6-aa6f-94959ad755e2', 'keypoint': {'x': 421.3025, 'y': 473.7194}, 'name': 'blue'}, {'id': '4a1722fe-5d45-4bf9-a201-c0b8f7acc36b', 'keypoint': {'x': 456.2467, 'y': 437.981}, 'name': 'blue'}, {'id': '680351fd-2723-4f71-9103-17bb8a809738', 'keypoint': {'x': 375.2398, 'y': 452.2763}, 'name': 'orange'}, {'id': '188969f7-812f-47f7-a97e-5a0652f78c64', 'keypoint': {'x': 354.5909, 'y': 508.6635}, 'name': 'orange'}, {'id': 'b57e9350-23ed-40aa-96cc-2811f14cd796', 'keypoint': {'x': 396.6828, 'y': 520.5763}, 'name': 'orange'}, {'id': 'f041b6b6-0396-42fe-99f5-46de7b8e6b23', 'keypoint': {'x': 431.6269, 'y': 507.8693}, 'name': 'orange'}, {'id': 'ae76b953-f8b0-4121-9569-ce66605c9783', 'keypoint': {'x': 433.2153, 'y': 530.9007}, 'name': 'orange'}, {'id': '256e0a15-32e1-445f-a768-a8adb00d4905', 'keypoint': {'x': 341.0898, 'y': 438.7752}, 'name': 'green'}, {'id': '51ccd976-c149-458d-960f-bd00da3a5131', 'keypoint': {'x': 318.0584, 'y': 536.46}, 'name': 'green'}, {'id': 'd3e5315b-8e96-4659-97cb-07d656a65db8', 'keypoint': {'x': 503.8978, 'y': 456.2473}, 'name': 'green'}]}
[/INST]
{'x': 433.2153, 'y': 530.9007}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the blue block at the top of the table located directly to the left of another blue block.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'c7519096-f7ea-4ab1-a66d-97c708976599', 'keypoint': {'x': 380.3349, 'y': 34.2453}, 'name': 'you'}, {'id': 'a95968ee-51d4-482d-ae79-97cb54663aed', 'keypoint': {'x': 293.9434, 'y': 479.434}, 'name': 'blue'}, {'id': '2fc8b299-6074-43eb-92bc-7ed758a958fc', 'keypoint': {'x': 329.7453, 'y': 557.2642}, 'name': 'blue'}, {'id': '11dbc045-de4f-4d3a-bc50-a1c2ba279bd1', 'keypoint': {'x': 366.3255, 'y': 546.3679}, 'name': 'blue'}, {'id': 'a3036737-fb7e-4f86-88c8-b0c5262cc051', 'keypoint': {'x': 379.5566, 'y': 438.9623}, 'name': 'blue'}, {'id': '5370f3f3-ef52-45e8-bdb4-3d96d54f4bd9', 'keypoint': {'x': 455.8302, 'y': 449.0802}, 'name': 'blue'}, {'id': '95294165-9354-41e7-b1be-60d4033bd1a3', 'keypoint': {'x': 318.0708, 'y': 454.5283}, 'name': 'yellow'}, {'id': '71f49ac9-1905-457f-9219-701cac7c42e4', 'keypoint': {'x': 318.0708, 'y': 530.8019}, 'name': 'yellow'}, {'id': '6f7a8a19-794f-4ffc-80ed-8a42dedeab0f', 'keypoint': {'x': 463.6132, 'y': 524.5755}, 'name': 'yellow'}, {'id': '407068c6-6a1e-4a14-b9d0-c0a4ed13b9c2', 'keypoint': {'x': 395.1226, 'y': 466.2028}, 'name': 'green'}, {'id': '847fb0f9-02cc-42df-8984-2b193e2abb8d', 'keypoint': {'x': 403.684, 'y': 523.0189}, 'name': 'green'}, {'id': 'bd78738b-2c39-4b85-9575-960d46ecd66c', 'keypoint': {'x': 325.0755, 'y': 500.4481}, 'name': 'orange'}, {'id': 'dafb3b80-4c1e-48a7-b398-e3aa56c5dcb0', 'keypoint': {'x': 374.1085, 'y': 515.2358}, 'name': 'orange'}, {'id': '36cf3f26-cac5-4368-91eb-ae8937ec2f91', 'keypoint': {'x': 412.2453, 'y': 427.2877}, 'name': 'orange'}, {'id': 'f93ff614-2250-4f4d-8ec0-ac87ec62305c', 'keypoint': {'x': 436.3726, 'y': 437.4057}, 'name': 'orange'}, {'id': '09066da9-88d0-4617-8cf6-7d604ee0190d', 'keypoint': {'x': 485.4057, 'y': 510.566}, 'name': 'orange'}]}
[/INST]
{'x': 366.3255, 'y': 546.3679}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:the 2nd yellow block from my right
 {'width': 756, 'height': 660, 'annotations': [{'id': 'fe98e63d-c226-4a25-9e75-229c9a2d828a', 'keypoint': {'x': 377.2217, 'y': 28.7972}, 'name': 'you'}, {'id': 'c1bfc6fe-9e0d-4152-aab0-9efe03f99ee1', 'keypoint': {'x': 291.6085, 'y': 487.217}, 'name': 'yellow'}, {'id': 'eefd5787-08ef-4674-ae9e-c41f4a2ed951', 'keypoint': {'x': 314.1792, 'y': 539.3632}, 'name': 'yellow'}, {'id': '8987e25d-8eb0-42ab-aa8e-390a7b046e44', 'keypoint': {'x': 363.2123, 'y': 544.8113}, 'name': 'yellow'}, {'id': 'af54d764-7469-4b39-95f2-d25ac6241715', 'keypoint': {'x': 434.0377, 'y': 542.4764}, 'name': 'yellow'}, {'id': 'c426887c-43fa-4c7c-9ee6-447f48c5b353', 'keypoint': {'x': 463.6132, 'y': 487.9953}, 'name': 'yellow'}, {'id': 'f1402f90-fe95-42c0-93e5-75019494beba', 'keypoint': {'x': 497.0802, 'y': 442.0755}, 'name': 'yellow'}, {'id': 'ad7b2c3b-efac-4705-8dc4-076514da9f0d', 'keypoint': {'x': 319.6274, 'y': 463.8679}, 'name': 'blue'}, {'id': '57fd11a6-5352-4e90-8c5e-78e080a9f63b', 'keypoint': {'x': 458.1651, 'y': 516.0142}, 'name': 'blue'}, {'id': 'bffb87fb-647e-44e1-b4c0-9bbf464cc4da', 'keypoint': {'x': 353.8726, 'y': 445.1887}, 'name': 'green'}, {'id': '2be751d9-34f0-411b-89e7-1b20b054e1ee', 'keypoint': {'x': 388.8962, 'y': 498.1132}, 'name': 'green'}, {'id': '98a41430-e570-492e-969a-8ea45857d765', 'keypoint': {'x': 395.9009, 'y': 533.1368}, 'name': 'green'}, {'id': '0327cb90-c17c-4528-8964-e978fbff5999', 'keypoint': {'x': 427.8113, 'y': 484.8821}, 'name': 'green'}, {'id': '97f91dd8-6cd8-4194-8783-e9e3b5786e68', 'keypoint': {'x': 456.6085, 'y': 452.9717}, 'name': 'green'}, {'id': '4ebadfd4-3497-4f80-a4e9-a870c25786bd', 'keypoint': {'x': 504.8632, 'y': 502.0047}, 'name': 'green'}, {'id': 'd2ff782c-210c-4090-91a5-de43a78f1232', 'keypoint': {'x': 331.3019, 'y': 512.9009}, 'name': 'orange'}]}
[/INST]
{'x': 463.6132, 'y': 487.9953}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the blue block that is right below the blue block furthest away from you.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'c7519096-f7ea-4ab1-a66d-97c708976599', 'keypoint': {'x': 380.3349, 'y': 34.2453}, 'name': 'you'}, {'id': 'a95968ee-51d4-482d-ae79-97cb54663aed', 'keypoint': {'x': 293.9434, 'y': 479.434}, 'name': 'blue'}, {'id': '2fc8b299-6074-43eb-92bc-7ed758a958fc', 'keypoint': {'x': 329.7453, 'y': 557.2642}, 'name': 'blue'}, {'id': '11dbc045-de4f-4d3a-bc50-a1c2ba279bd1', 'keypoint': {'x': 366.3255, 'y': 546.3679}, 'name': 'blue'}, {'id': 'a3036737-fb7e-4f86-88c8-b0c5262cc051', 'keypoint': {'x': 379.5566, 'y': 438.9623}, 'name': 'blue'}, {'id': '5370f3f3-ef52-45e8-bdb4-3d96d54f4bd9', 'keypoint': {'x': 455.8302, 'y': 449.0802}, 'name': 'blue'}, {'id': '95294165-9354-41e7-b1be-60d4033bd1a3', 'keypoint': {'x': 318.0708, 'y': 454.5283}, 'name': 'yellow'}, {'id': '71f49ac9-1905-457f-9219-701cac7c42e4', 'keypoint': {'x': 318.0708, 'y': 530.8019}, 'name': 'yellow'}, {'id': '6f7a8a19-794f-4ffc-80ed-8a42dedeab0f', 'keypoint': {'x': 463.6132, 'y': 524.5755}, 'name': 'yellow'}, {'id': '407068c6-6a1e-4a14-b9d0-c0a4ed13b9c2', 'keypoint': {'x': 395.1226, 'y': 466.2028}, 'name': 'green'}, {'id': '847fb0f9-02cc-42df-8984-2b193e2abb8d', 'keypoint': {'x': 403.684, 'y': 523.0189}, 'name': 'green'}, {'id': 'bd78738b-2c39-4b85-9575-960d46ecd66c', 'keypoint': {'x': 325.0755, 'y': 500.4481}, 'name': 'orange'}, {'id': 'dafb3b80-4c1e-48a7-b398-e3aa56c5dcb0', 'keypoint': {'x': 374.1085, 'y': 515.2358}, 'name': 'orange'}, {'id': '36cf3f26-cac5-4368-91eb-ae8937ec2f91', 'keypoint': {'x': 412.2453, 'y': 427.2877}, 'name': 'orange'}, {'id': 'f93ff614-2250-4f4d-8ec0-ac87ec62305c', 'keypoint': {'x': 436.3726, 'y': 437.4057}, 'name': 'orange'}, {'id': '09066da9-88d0-4617-8cf6-7d604ee0190d', 'keypoint': {'x': 485.4057, 'y': 510.566}, 'name': 'orange'}]}
[/INST]
{'x': 366.3255, 'y': 546.3679}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick the green block that's on the left side of the screen - it should be near the top left.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'f8ce6e33-d9e3-4687-bd11-4537937b3def', 'keypoint': {'x': 313.4009, 'y': 480.9906}, 'name': 'orange'}, {'id': '8758b8e0-78a7-4dc2-b595-9eec97d96b3c', 'keypoint': {'x': 403.684, 'y': 466.9811}, 'name': 'orange'}, {'id': '9024f068-b46e-400e-bf69-507f61b74f80', 'keypoint': {'x': 459.7217, 'y': 452.9717}, 'name': 'orange'}, {'id': 'd7ab991d-e3d6-4ffb-8d9b-9cffbb59b9c6', 'keypoint': {'x': 502.5283, 'y': 457.6415}, 'name': 'orange'}, {'id': 'b9fb860f-d2d0-4b4a-a275-416d46a99ab8', 'keypoint': {'x': 335.9717, 'y': 450.6368}, 'name': 'yellow'}, {'id': '7404a736-3ae3-4f5c-b40d-5d5af4635c6e', 'keypoint': {'x': 316.5142, 'y': 531.5802}, 'name': 'yellow'}, {'id': 'd2d6b9ff-fd85-48a1-b368-3f0e1d09cc6f', 'keypoint': {'x': 396.6792, 'y': 491.8868}, 'name': 'yellow'}, {'id': 'c4551bcd-78f8-4205-b828-5b040d49a2c3', 'keypoint': {'x': 444.1557, 'y': 433.5142}, 'name': 'yellow'}, {'id': '3e0cda0a-1e8a-4472-8984-7886c02ff560', 'keypoint': {'x': 347.6462, 'y': 514.4575}, 'name': 'blue'}, {'id': '266b6be1-582b-4c4d-8544-906fad90cbd2', 'keypoint': {'x': 359.3208, 'y': 478.6557}, 'name': 'blue'}, {'id': 'c58796fa-46b4-4231-8589-0612cd52983f', 'keypoint': {'x': 395.9009, 'y': 444.4104}, 'name': 'blue'}, {'id': '3de58fe6-02ae-4e5a-b0eb-ac1e3ac60960', 'keypoint': {'x': 444.934, 'y': 537.8066}, 'name': 'blue'}, {'id': 'a1a96c53-7f31-4121-8983-bf6a69f42935', 'keypoint': {'x': 502.5283, 'y': 541.6981}, 'name': 'blue'}, {'id': 'c2e223f1-9da7-4029-ab29-81edb6be3c94', 'keypoint': {'x': 355.4292, 'y': 424.9528}, 'name': 'green'}, {'id': 'd1f0facf-050a-4a92-bfb1-1367c5fdd80d', 'keypoint': {'x': 466.7264, 'y': 500.4481}, 'name': 'green'}, {'id': '727b2df8-1454-47db-9eff-8cfe072974ff', 'keypoint': {'x': 383.4481, 'y': 44.3632}, 'name': 'you'}]}
[/INST]
{'x': 466.7264, 'y': 500.4481}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:On your left side, pick up the yellow block that is 2 blocks behind the blue block.
 {'width': 756, 'height': 660, 'annotations': [{'id': '53de559f-b2c6-46fa-821d-b5a4e98c67c3', 'keypoint': {'x': 332.0802, 'y': 517.5708}, 'name': 'orange'}, {'id': 'fc5cb394-ebdb-4064-bb12-1d3da5b0cb6f', 'keypoint': {'x': 316.5142, 'y': 540.1415}, 'name': 'orange'}, {'id': 'b8f5d815-90be-4cd5-951f-44207215b594', 'keypoint': {'x': 373.3302, 'y': 537.8066}, 'name': 'orange'}, {'id': '78d88ded-7431-4260-a864-725e2dfef6ab', 'keypoint': {'x': 309.5094, 'y': 498.1132}, 'name': 'yellow'}, {'id': 'e5215008-d111-4bda-9009-f2c0795ffb0d', 'keypoint': {'x': 388.1179, 'y': 471.6509}, 'name': 'yellow'}, {'id': 'b722e1f1-0a3c-47ef-b35f-e9cd85898169', 'keypoint': {'x': 388.1179, 'y': 501.2264}, 'name': 'yellow'}, {'id': '27ad7266-481b-40f9-aaa0-e71dabd71d5e', 'keypoint': {'x': 413.8019, 'y': 491.1085}, 'name': 'yellow'}, {'id': '71348b8e-08dc-4a76-be0e-68cbee1a56a4', 'keypoint': {'x': 416.1368, 'y': 449.8585}, 'name': 'yellow'}, {'id': 'e46d6178-4284-433d-a81f-bc6cbbf7cdbb', 'keypoint': {'x': 490.8538, 'y': 505.1179}, 'name': 'yellow'}, {'id': '272a554a-aad9-4bba-9062-941465beeaa5', 'keypoint': {'x': 465.1698, 'y': 554.1509}, 'name': 'yellow'}, {'id': 'c6ba259b-1c8e-43d4-8bdb-404594f1c175', 'keypoint': {'x': 310.2877, 'y': 438.9623}, 'name': 'blue'}, {'id': '693a6454-810f-447d-8417-a2c57867901d', 'keypoint': {'x': 354.6509, 'y': 477.8774}, 'name': 'blue'}, {'id': '3ec7fa7f-c792-4c99-a423-b40bbb665d89', 'keypoint': {'x': 427.8113, 'y': 428.8443}, 'name': 'blue'}, {'id': '3e041f2e-9e78-4001-9fa0-3bebee80ac07', 'keypoint': {'x': 440.2642, 'y': 498.8915}, 'name': 'green'}, {'id': 'e2e59b33-f4e7-47cd-b137-77a874c834c7', 'keypoint': {'x': 450.3821, 'y': 519.1274}, 'name': 'green'}, {'id': '707959b0-76f6-463a-bc2f-62cd702815f4', 'keypoint': {'x': 377.2217, 'y': 55.2594}, 'name': 'you'}]}
[/INST]
{'x': 413.8019, 'y': 491.1085}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:The block is in the middle of the table. It is the 3rd block from the bottom.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'd5bcc2b7-ff91-4686-8f50-3a73ffd3beeb', 'keypoint': {'x': 315.7358, 'y': 436.6274}, 'name': 'orange'}, {'id': '70515b86-98d5-4fa6-9118-a0bc484f6de1', 'keypoint': {'x': 313.4009, 'y': 525.3538}, 'name': 'orange'}, {'id': 'b40b0958-f706-45d2-a042-0440eb67121b', 'keypoint': {'x': 380.3349, 'y': 515.2358}, 'name': 'orange'}, {'id': '9dcd051f-bad9-4f84-b653-9a30a72cb545', 'keypoint': {'x': 389.6745, 'y': 552.5943}, 'name': 'orange'}, {'id': 'b0de6329-f4d9-486c-91ed-478f28262cff', 'keypoint': {'x': 363.9906, 'y': 439.7406}, 'name': 'yellow'}, {'id': '10eeedc5-3253-42e7-b401-33c4616e4623', 'keypoint': {'x': 396.6792, 'y': 457.6415}, 'name': 'yellow'}, {'id': 'f7642eee-44ca-4f1d-abf7-a79d43dfbd0d', 'keypoint': {'x': 279.934, 'y': 535.4717}, 'name': 'yellow'}, {'id': 'd87c603d-89da-4c0b-8f61-a496275c1297', 'keypoint': {'x': 311.066, 'y': 487.217}, 'name': 'blue'}, {'id': '2065d0e4-12eb-4ab3-bde4-75688bb3c142', 'keypoint': {'x': 457.3868, 'y': 439.7406}, 'name': 'green'}, {'id': '1550df58-31c4-49f4-8e6d-7db4a57bc294', 'keypoint': {'x': 435.5943, 'y': 445.967}, 'name': 'green'}, {'id': '1990bf50-38ac-43ab-8581-26778cea80ed', 'keypoint': {'x': 458.9434, 'y': 484.8821}, 'name': 'green'}, {'id': '7895d85c-2e57-4008-b9d7-74625608a42c', 'keypoint': {'x': 485.4057, 'y': 515.2358}, 'name': 'green'}, {'id': '991a571a-03aa-4a00-b80d-3d6a0845625b', 'keypoint': {'x': 440.2642, 'y': 514.4575}, 'name': 'green'}, {'id': 'e0e1086c-3a47-42f6-98fd-868d0756cd4d', 'keypoint': {'x': 378.7783, 'y': 495.7783}, 'name': 'green'}, {'id': '3d132917-dd96-4ae9-9302-da261696357f', 'keypoint': {'x': 333.6368, 'y': 505.1179}, 'name': 'green'}, {'id': 'bc7494f1-b049-49c6-bf24-85e942969065', 'keypoint': {'x': 383.4481, 'y': 56.0377}, 'name': 'you'}]}
[/INST]
{'x': 378.7783, 'y': 495.7783}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Of the two purple blocks closest to me, pick up the one that is closest to you.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'c7519096-f7ea-4ab1-a66d-97c708976599', 'keypoint': {'x': 380.3349, 'y': 34.2453}, 'name': 'you'}, {'id': 'a95968ee-51d4-482d-ae79-97cb54663aed', 'keypoint': {'x': 293.9434, 'y': 479.434}, 'name': 'blue'}, {'id': '2fc8b299-6074-43eb-92bc-7ed758a958fc', 'keypoint': {'x': 329.7453, 'y': 557.2642}, 'name': 'blue'}, {'id': '11dbc045-de4f-4d3a-bc50-a1c2ba279bd1', 'keypoint': {'x': 366.3255, 'y': 546.3679}, 'name': 'blue'}, {'id': 'a3036737-fb7e-4f86-88c8-b0c5262cc051', 'keypoint': {'x': 379.5566, 'y': 438.9623}, 'name': 'blue'}, {'id': '5370f3f3-ef52-45e8-bdb4-3d96d54f4bd9', 'keypoint': {'x': 455.8302, 'y': 449.0802}, 'name': 'blue'}, {'id': '95294165-9354-41e7-b1be-60d4033bd1a3', 'keypoint': {'x': 318.0708, 'y': 454.5283}, 'name': 'yellow'}, {'id': '71f49ac9-1905-457f-9219-701cac7c42e4', 'keypoint': {'x': 318.0708, 'y': 530.8019}, 'name': 'yellow'}, {'id': '6f7a8a19-794f-4ffc-80ed-8a42dedeab0f', 'keypoint': {'x': 463.6132, 'y': 524.5755}, 'name': 'yellow'}, {'id': '407068c6-6a1e-4a14-b9d0-c0a4ed13b9c2', 'keypoint': {'x': 395.1226, 'y': 466.2028}, 'name': 'green'}, {'id': '847fb0f9-02cc-42df-8984-2b193e2abb8d', 'keypoint': {'x': 403.684, 'y': 523.0189}, 'name': 'green'}, {'id': 'bd78738b-2c39-4b85-9575-960d46ecd66c', 'keypoint': {'x': 325.0755, 'y': 500.4481}, 'name': 'orange'}, {'id': 'dafb3b80-4c1e-48a7-b398-e3aa56c5dcb0', 'keypoint': {'x': 374.1085, 'y': 515.2358}, 'name': 'orange'}, {'id': '36cf3f26-cac5-4368-91eb-ae8937ec2f91', 'keypoint': {'x': 412.2453, 'y': 427.2877}, 'name': 'orange'}, {'id': 'f93ff614-2250-4f4d-8ec0-ac87ec62305c', 'keypoint': {'x': 436.3726, 'y': 437.4057}, 'name': 'orange'}, {'id': '09066da9-88d0-4617-8cf6-7d604ee0190d', 'keypoint': {'x': 485.4057, 'y': 510.566}, 'name': 'orange'}]}
[/INST]
{'x': 366.3255, 'y': 546.3679}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:It is the block on the farthest right. It is the 2nd one from the top.
 {'width': 756, 'height': 660, 'annotations': [{'id': '8f941e4d-8e6b-4e37-a2cd-f19b3dbb1dda', 'keypoint': {'x': 293.1651, 'y': 452.9717}, 'name': 'orange'}, {'id': '77f8b560-0c54-4923-9c65-8e8973d5cc1a', 'keypoint': {'x': 293.9434, 'y': 485.6604}, 'name': 'orange'}, {'id': 'c2fdf254-12fa-43ee-9d04-fd097d6a2fad', 'keypoint': {'x': 332.8585, 'y': 517.5708}, 'name': 'orange'}, {'id': 'ee022624-310e-4867-8360-cd61e6a5e8ec', 'keypoint': {'x': 325.0755, 'y': 557.2642}, 'name': 'orange'}, {'id': 'ca78545f-1817-447e-a767-1ec186e40025', 'keypoint': {'x': 377.2217, 'y': 478.6557}, 'name': 'orange'}, {'id': 'b7ea891b-3095-4df8-a718-b193f34c851c', 'keypoint': {'x': 386.5613, 'y': 492.6651}, 'name': 'orange'}, {'id': '54b85951-9b7d-4a22-9b94-5856c8f4cc7e', 'keypoint': {'x': 441.8208, 'y': 499.6698}, 'name': 'orange'}, {'id': 'ef2b7cbb-fa7e-4894-8e50-310d43a6f3eb', 'keypoint': {'x': 385.0047, 'y': 448.3019}, 'name': 'yellow'}, {'id': 'ab7d8b08-4eaf-41cf-8066-9aba376bb546', 'keypoint': {'x': 392.0094, 'y': 523.0189}, 'name': 'yellow'}, {'id': 'aec059f3-2def-4458-9ea4-3fa5916cab5b', 'keypoint': {'x': 466.7264, 'y': 442.8538}, 'name': 'yellow'}, {'id': '39fe094d-1e93-4828-ae7e-d672aeb88451', 'keypoint': {'x': 356.2075, 'y': 533.9151}, 'name': 'blue'}, {'id': '57cde523-a685-45a5-b623-1b11057840da', 'keypoint': {'x': 487.7406, 'y': 488.7736}, 'name': 'blue'}, {'id': 'da91e8aa-4087-4689-95ed-80e4ba9a93ec', 'keypoint': {'x': 327.4104, 'y': 472.4292}, 'name': 'green'}, {'id': '9406ceb5-4de6-4915-8da6-132b32721d7c', 'keypoint': {'x': 508.7547, 'y': 511.3443}, 'name': 'green'}, {'id': 'fd93334e-be71-4598-9c11-abb001661b87', 'keypoint': {'x': 497.0802, 'y': 560.3774}, 'name': 'green'}, {'id': '3067ae97-d698-4a21-82f6-e94a33ed51a0', 'keypoint': {'x': 380.3349, 'y': 59.1509}, 'name': 'you'}]}
[/INST]
{'x': 487.7406, 'y': 488.7736}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:pick up the yellow cube that is furthest away from you.
 {'width': 756, 'height': 660, 'annotations': [{'id': '82770085-a20a-439c-b1a3-f74a8d8ccdd0', 'keypoint': {'x': 322.0293, 'y': 491.9856}, 'name': 'yellow'}, {'id': '975d2226-07a6-46ec-8aa1-01f44597bb5c', 'keypoint': {'x': 376.8281, 'y': 495.1624}, 'name': 'yellow'}, {'id': '9fbd0dd7-292a-46c1-854e-eb8c4bb880cf', 'keypoint': {'x': 402.2421, 'y': 551.5495}, 'name': 'yellow'}, {'id': 'c2d61b6b-0585-4aef-acaf-17bad430192b', 'keypoint': {'x': 464.9827, 'y': 498.3391}, 'name': 'yellow'}, {'id': '13d3414e-a81b-4dad-8963-6fbda2f0d436', 'keypoint': {'x': 376.0339, 'y': 27.387}, 'name': 'you'}, {'id': '9b80a1cd-fed1-49e9-8b69-3cd0088f7fca', 'keypoint': {'x': 302.9689, 'y': 454.6589}, 'name': 'blue'}, {'id': '17a9c4b0-bf5e-45d6-aa6f-94959ad755e2', 'keypoint': {'x': 421.3025, 'y': 473.7194}, 'name': 'blue'}, {'id': '4a1722fe-5d45-4bf9-a201-c0b8f7acc36b', 'keypoint': {'x': 456.2467, 'y': 437.981}, 'name': 'blue'}, {'id': '680351fd-2723-4f71-9103-17bb8a809738', 'keypoint': {'x': 375.2398, 'y': 452.2763}, 'name': 'orange'}, {'id': '188969f7-812f-47f7-a97e-5a0652f78c64', 'keypoint': {'x': 354.5909, 'y': 508.6635}, 'name': 'orange'}, {'id': 'b57e9350-23ed-40aa-96cc-2811f14cd796', 'keypoint': {'x': 396.6828, 'y': 520.5763}, 'name': 'orange'}, {'id': 'f041b6b6-0396-42fe-99f5-46de7b8e6b23', 'keypoint': {'x': 431.6269, 'y': 507.8693}, 'name': 'orange'}, {'id': 'ae76b953-f8b0-4121-9569-ce66605c9783', 'keypoint': {'x': 433.2153, 'y': 530.9007}, 'name': 'orange'}, {'id': '256e0a15-32e1-445f-a768-a8adb00d4905', 'keypoint': {'x': 341.0898, 'y': 438.7752}, 'name': 'green'}, {'id': '51ccd976-c149-458d-960f-bd00da3a5131', 'keypoint': {'x': 318.0584, 'y': 536.46}, 'name': 'green'}, {'id': 'd3e5315b-8e96-4659-97cb-07d656a65db8', 'keypoint': {'x': 503.8978, 'y': 456.2473}, 'name': 'green'}]}
[/INST]
{'x': 402.2421, 'y': 551.5495}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the green block that is in the middle of the green blocks on your left.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'd5bcc2b7-ff91-4686-8f50-3a73ffd3beeb', 'keypoint': {'x': 315.7358, 'y': 436.6274}, 'name': 'orange'}, {'id': '70515b86-98d5-4fa6-9118-a0bc484f6de1', 'keypoint': {'x': 313.4009, 'y': 525.3538}, 'name': 'orange'}, {'id': 'b40b0958-f706-45d2-a042-0440eb67121b', 'keypoint': {'x': 380.3349, 'y': 515.2358}, 'name': 'orange'}, {'id': '9dcd051f-bad9-4f84-b653-9a30a72cb545', 'keypoint': {'x': 389.6745, 'y': 552.5943}, 'name': 'orange'}, {'id': 'b0de6329-f4d9-486c-91ed-478f28262cff', 'keypoint': {'x': 363.9906, 'y': 439.7406}, 'name': 'yellow'}, {'id': '10eeedc5-3253-42e7-b401-33c4616e4623', 'keypoint': {'x': 396.6792, 'y': 457.6415}, 'name': 'yellow'}, {'id': 'f7642eee-44ca-4f1d-abf7-a79d43dfbd0d', 'keypoint': {'x': 279.934, 'y': 535.4717}, 'name': 'yellow'}, {'id': 'd87c603d-89da-4c0b-8f61-a496275c1297', 'keypoint': {'x': 311.066, 'y': 487.217}, 'name': 'blue'}, {'id': '2065d0e4-12eb-4ab3-bde4-75688bb3c142', 'keypoint': {'x': 457.3868, 'y': 439.7406}, 'name': 'green'}, {'id': '1550df58-31c4-49f4-8e6d-7db4a57bc294', 'keypoint': {'x': 435.5943, 'y': 445.967}, 'name': 'green'}, {'id': '1990bf50-38ac-43ab-8581-26778cea80ed', 'keypoint': {'x': 458.9434, 'y': 484.8821}, 'name': 'green'}, {'id': '7895d85c-2e57-4008-b9d7-74625608a42c', 'keypoint': {'x': 485.4057, 'y': 515.2358}, 'name': 'green'}, {'id': '991a571a-03aa-4a00-b80d-3d6a0845625b', 'keypoint': {'x': 440.2642, 'y': 514.4575}, 'name': 'green'}, {'id': 'e0e1086c-3a47-42f6-98fd-868d0756cd4d', 'keypoint': {'x': 378.7783, 'y': 495.7783}, 'name': 'green'}, {'id': '3d132917-dd96-4ae9-9302-da261696357f', 'keypoint': {'x': 333.6368, 'y': 505.1179}, 'name': 'green'}, {'id': 'bc7494f1-b049-49c6-bf24-85e942969065', 'keypoint': {'x': 383.4481, 'y': 56.0377}, 'name': 'you'}]}
[/INST]
{'x': 458.9434, 'y': 484.8821}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:From your point of view, grab the left most green block.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'cad61a18-85b8-4f6e-93b3-c3ce9e28cda2', 'keypoint': {'x': 316.5142, 'y': 554.9292}, 'name': 'orange'}, {'id': '17682384-25d2-46d1-ae87-f04077458497', 'keypoint': {'x': 330.5236, 'y': 516.7925}, 'name': 'orange'}, {'id': '5d36ae41-004b-494e-b32d-33279b1dbfee', 'keypoint': {'x': 447.2689, 'y': 457.6415}, 'name': 'orange'}, {'id': 'fee58518-97d8-4c98-9b1b-ebeaa7ef3967', 'keypoint': {'x': 488.5189, 'y': 530.8019}, 'name': 'orange'}, {'id': '4fa5b6b4-f98b-4588-95e6-8255a4bef84b', 'keypoint': {'x': 380.3349, 'y': 446.7453}, 'name': 'yellow'}, {'id': '22f0cd3c-99be-40d9-9b3f-a7bc782da1f8', 'keypoint': {'x': 297.8349, 'y': 529.2453}, 'name': 'yellow'}, {'id': 'd5f859c2-a998-4ee4-bff1-d6025abd4883', 'keypoint': {'x': 375.6651, 'y': 536.25}, 'name': 'yellow'}, {'id': '96691580-8803-4b4b-b31f-4920d402e03e', 'keypoint': {'x': 473.7311, 'y': 435.0708}, 'name': 'yellow'}, {'id': '86653055-abe8-4a82-8b79-fb7f461438c0', 'keypoint': {'x': 474.5094, 'y': 487.217}, 'name': 'yellow'}, {'id': '34e55929-db77-4549-8f50-5adc4da9bd5d', 'keypoint': {'x': 370.9953, 'y': 492.6651}, 'name': 'blue'}, {'id': '500fbe66-22dd-4552-a43c-63bf2efeb1c9', 'keypoint': {'x': 417.6934, 'y': 477.0991}, 'name': 'blue'}, {'id': '85cb36b6-c443-488a-85b6-363a2284870a', 'keypoint': {'x': 345.3113, 'y': 424.9528}, 'name': 'green'}, {'id': '1c037117-1e04-4cd7-a544-c3e10b3639bb', 'keypoint': {'x': 295.5, 'y': 466.9811}, 'name': 'green'}, {'id': 'a8f28480-33ff-4a0a-883f-f1f226ced914', 'keypoint': {'x': 299.3915, 'y': 501.2264}, 'name': 'green'}, {'id': 'b9578593-4dcd-4808-9119-c6439b45d587', 'keypoint': {'x': 427.033, 'y': 534.6934}, 'name': 'green'}, {'id': '70024c2f-19b2-412a-a76d-d9e741635b7e', 'keypoint': {'x': 380.3349, 'y': 33.467}, 'name': 'you'}]}
[/INST]
{'x': 427.033, 'y': 534.6934}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:From your perspective you should pick the closest green cube to you.
 {'width': 756, 'height': 660, 'annotations': [{'id': '4665780d-e410-4494-8512-3557afddb1a4', 'keypoint': {'x': 311.8443, 'y': 504.3396}, 'name': None}, {'id': '565e6ab8-2ba1-4136-96ed-e63f3cdba23d', 'keypoint': {'x': 368.6604, 'y': 505.1179}, 'name': 'orange'}, {'id': '487a86d1-7908-404f-99e9-16ab87ea0d04', 'keypoint': {'x': 359.3208, 'y': 560.3774}, 'name': 'orange'}, {'id': 'ab7b691c-c47b-4c26-8083-096140fac645', 'keypoint': {'x': 413.0236, 'y': 447.5236}, 'name': 'orange'}, {'id': 'fdb6ba1e-6ab3-4d82-862c-1bdaf73940dc', 'keypoint': {'x': 464.3915, 'y': 459.9764}, 'name': 'orange'}, {'id': '7768e90f-d1af-4f66-abc0-01fa10bc5e61', 'keypoint': {'x': 447.2689, 'y': 530.8019}, 'name': 'orange'}, {'id': '698efbba-5197-4d9f-96eb-fdc8c950c48f', 'keypoint': {'x': 290.8302, 'y': 554.1509}, 'name': 'yellow'}, {'id': '1e9d76c9-a045-4417-94ae-129e3daeb548', 'keypoint': {'x': 407.5755, 'y': 519.9057}, 'name': 'blue'}, {'id': 'dc0ba9e0-8eb1-478a-88c5-3eb00d02992d', 'keypoint': {'x': 454.2736, 'y': 505.8962}, 'name': 'blue'}, {'id': '13165129-2db5-428d-ba05-982c1ae9a179', 'keypoint': {'x': 503.3066, 'y': 487.217}, 'name': 'green'}, {'id': 'a2feb4b6-353e-4876-b8e6-d5351bc5f056', 'keypoint': {'x': 459.7217, 'y': 490.3302}, 'name': 'green'}, {'id': 'bd636fd9-8ba5-408a-a286-413978cb55e7', 'keypoint': {'x': 362.434, 'y': 476.3208}, 'name': 'green'}, {'id': '4d316993-cf9a-4115-894b-940d569fafc4', 'keypoint': {'x': 311.066, 'y': 439.7406}, 'name': 'green'}, {'id': 'b6f5df26-1415-40e4-bbf7-01af31da49b9', 'keypoint': {'x': 380.3349, 'y': 544.8113}, 'name': 'green'}, {'id': 'f34f5fb5-74bd-4265-8123-8eed80fffdd5', 'keypoint': {'x': 322.7406, 'y': 536.25}, 'name': 'green'}, {'id': '847a9a8d-0e7a-49f2-b274-f833cdac716e', 'keypoint': {'x': 375.6651, 'y': 49.8113}, 'name': 'you'}]}
[/INST]
{'x': 311.066, 'y': 439.7406}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the green block in the center of the 5 green blocks
 {'width': 756, 'height': 660, 'annotations': [{'id': 'd5bcc2b7-ff91-4686-8f50-3a73ffd3beeb', 'keypoint': {'x': 315.7358, 'y': 436.6274}, 'name': 'orange'}, {'id': '70515b86-98d5-4fa6-9118-a0bc484f6de1', 'keypoint': {'x': 313.4009, 'y': 525.3538}, 'name': 'orange'}, {'id': 'b40b0958-f706-45d2-a042-0440eb67121b', 'keypoint': {'x': 380.3349, 'y': 515.2358}, 'name': 'orange'}, {'id': '9dcd051f-bad9-4f84-b653-9a30a72cb545', 'keypoint': {'x': 389.6745, 'y': 552.5943}, 'name': 'orange'}, {'id': 'b0de6329-f4d9-486c-91ed-478f28262cff', 'keypoint': {'x': 363.9906, 'y': 439.7406}, 'name': 'yellow'}, {'id': '10eeedc5-3253-42e7-b401-33c4616e4623', 'keypoint': {'x': 396.6792, 'y': 457.6415}, 'name': 'yellow'}, {'id': 'f7642eee-44ca-4f1d-abf7-a79d43dfbd0d', 'keypoint': {'x': 279.934, 'y': 535.4717}, 'name': 'yellow'}, {'id': 'd87c603d-89da-4c0b-8f61-a496275c1297', 'keypoint': {'x': 311.066, 'y': 487.217}, 'name': 'blue'}, {'id': '2065d0e4-12eb-4ab3-bde4-75688bb3c142', 'keypoint': {'x': 457.3868, 'y': 439.7406}, 'name': 'green'}, {'id': '1550df58-31c4-49f4-8e6d-7db4a57bc294', 'keypoint': {'x': 435.5943, 'y': 445.967}, 'name': 'green'}, {'id': '1990bf50-38ac-43ab-8581-26778cea80ed', 'keypoint': {'x': 458.9434, 'y': 484.8821}, 'name': 'green'}, {'id': '7895d85c-2e57-4008-b9d7-74625608a42c', 'keypoint': {'x': 485.4057, 'y': 515.2358}, 'name': 'green'}, {'id': '991a571a-03aa-4a00-b80d-3d6a0845625b', 'keypoint': {'x': 440.2642, 'y': 514.4575}, 'name': 'green'}, {'id': 'e0e1086c-3a47-42f6-98fd-868d0756cd4d', 'keypoint': {'x': 378.7783, 'y': 495.7783}, 'name': 'green'}, {'id': '3d132917-dd96-4ae9-9302-da261696357f', 'keypoint': {'x': 333.6368, 'y': 505.1179}, 'name': 'green'}, {'id': 'bc7494f1-b049-49c6-bf24-85e942969065', 'keypoint': {'x': 383.4481, 'y': 56.0377}, 'name': 'you'}]}
[/INST]
{'x': 458.9434, 'y': 484.8821}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:The one you want will be green and it will be directly in front of two orange ones stacked up
 {'width': 756, 'height': 660, 'annotations': [{'id': 'd5bcc2b7-ff91-4686-8f50-3a73ffd3beeb', 'keypoint': {'x': 315.7358, 'y': 436.6274}, 'name': 'orange'}, {'id': '70515b86-98d5-4fa6-9118-a0bc484f6de1', 'keypoint': {'x': 313.4009, 'y': 525.3538}, 'name': 'orange'}, {'id': 'b40b0958-f706-45d2-a042-0440eb67121b', 'keypoint': {'x': 380.3349, 'y': 515.2358}, 'name': 'orange'}, {'id': '9dcd051f-bad9-4f84-b653-9a30a72cb545', 'keypoint': {'x': 389.6745, 'y': 552.5943}, 'name': 'orange'}, {'id': 'b0de6329-f4d9-486c-91ed-478f28262cff', 'keypoint': {'x': 363.9906, 'y': 439.7406}, 'name': 'yellow'}, {'id': '10eeedc5-3253-42e7-b401-33c4616e4623', 'keypoint': {'x': 396.6792, 'y': 457.6415}, 'name': 'yellow'}, {'id': 'f7642eee-44ca-4f1d-abf7-a79d43dfbd0d', 'keypoint': {'x': 279.934, 'y': 535.4717}, 'name': 'yellow'}, {'id': 'd87c603d-89da-4c0b-8f61-a496275c1297', 'keypoint': {'x': 311.066, 'y': 487.217}, 'name': 'blue'}, {'id': '2065d0e4-12eb-4ab3-bde4-75688bb3c142', 'keypoint': {'x': 457.3868, 'y': 439.7406}, 'name': 'green'}, {'id': '1550df58-31c4-49f4-8e6d-7db4a57bc294', 'keypoint': {'x': 435.5943, 'y': 445.967}, 'name': 'green'}, {'id': '1990bf50-38ac-43ab-8581-26778cea80ed', 'keypoint': {'x': 458.9434, 'y': 484.8821}, 'name': 'green'}, {'id': '7895d85c-2e57-4008-b9d7-74625608a42c', 'keypoint': {'x': 485.4057, 'y': 515.2358}, 'name': 'green'}, {'id': '991a571a-03aa-4a00-b80d-3d6a0845625b', 'keypoint': {'x': 440.2642, 'y': 514.4575}, 'name': 'green'}, {'id': 'e0e1086c-3a47-42f6-98fd-868d0756cd4d', 'keypoint': {'x': 378.7783, 'y': 495.7783}, 'name': 'green'}, {'id': '3d132917-dd96-4ae9-9302-da261696357f', 'keypoint': {'x': 333.6368, 'y': 505.1179}, 'name': 'green'}, {'id': 'bc7494f1-b049-49c6-bf24-85e942969065', 'keypoint': {'x': 383.4481, 'y': 56.0377}, 'name': 'you'}]}
[/INST]
{'x': 378.7783, 'y': 495.7783}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:There is a cluster of 3 blue blocks to your left. It is near an orange block. Grab the one in the center, it should be the closest one to you in that group.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'a7c6f9ab-55b7-422b-8e7f-70902f2e94d6', 'keypoint': {'x': 320.4057, 'y': 488.7736}, 'name': 'orange'}, {'id': 'acc187bd-2783-4707-be6a-314a58518228', 'keypoint': {'x': 423.9198, 'y': 451.4151}, 'name': 'orange'}, {'id': '7b9652c2-a499-48cc-ab07-68e686e885ac', 'keypoint': {'x': 358.5425, 'y': 462.3113}, 'name': 'yellow'}, {'id': '6e4e6633-28b0-4e37-a08b-d90cdb596a0d', 'keypoint': {'x': 334.4151, 'y': 548.7028}, 'name': 'yellow'}, {'id': 'dc28f152-a36a-458c-9f8a-21f4093b66bd', 'keypoint': {'x': 418.4717, 'y': 547.9245}, 'name': 'yellow'}, {'id': '34029874-d3f4-4998-a6fa-85ee314b35b5', 'keypoint': {'x': 479.1792, 'y': 484.8821}, 'name': 'yellow'}, {'id': '3d4114df-2e6e-4d36-8e12-47bbffbd6150', 'keypoint': {'x': 314.1792, 'y': 452.1934}, 'name': 'blue'}, {'id': '021cc2a3-2e83-4079-9777-64d3bc57e829', 'keypoint': {'x': 293.1651, 'y': 557.2642}, 'name': 'blue'}, {'id': '49911b5f-df0a-4694-9ff5-d8165f0f587a', 'keypoint': {'x': 390.4528, 'y': 513.6792}, 'name': 'blue'}, {'id': '95b20075-c6a2-46e0-b203-f8004a3ee4a3', 'keypoint': {'x': 440.2642, 'y': 484.8821}, 'name': 'blue'}, {'id': '031e7d06-9b93-4602-a1ef-c48bce3518c4', 'keypoint': {'x': 462.0566, 'y': 469.316}, 'name': 'blue'}, {'id': '3f2fdd03-f2f9-4892-999b-713f9e988be6', 'keypoint': {'x': 493.1887, 'y': 465.4245}, 'name': 'blue'}, {'id': '7ed0f47f-3bdf-429a-ab98-1d1a9d3058fa', 'keypoint': {'x': 482.2925, 'y': 507.4528}, 'name': 'green'}, {'id': '1fb5251e-aa36-4074-87f5-edef1e5a25c7', 'keypoint': {'x': 491.6321, 'y': 542.4764}, 'name': 'green'}, {'id': 'e7cf6134-82a1-4efe-99a5-a098dcdb3a3e', 'keypoint': {'x': 334.4151, 'y': 516.0142}, 'name': 'green'}, {'id': '0aeb291a-183c-47ce-9552-8b4d4860c05f', 'keypoint': {'x': 376.4434, 'y': 44.3632}, 'name': 'you'}]}
[/INST]
{'x': 462.0566, 'y': 469.316}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the blue block that is furthest from you.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'c7519096-f7ea-4ab1-a66d-97c708976599', 'keypoint': {'x': 380.3349, 'y': 34.2453}, 'name': 'you'}, {'id': 'a95968ee-51d4-482d-ae79-97cb54663aed', 'keypoint': {'x': 293.9434, 'y': 479.434}, 'name': 'blue'}, {'id': '2fc8b299-6074-43eb-92bc-7ed758a958fc', 'keypoint': {'x': 329.7453, 'y': 557.2642}, 'name': 'blue'}, {'id': '11dbc045-de4f-4d3a-bc50-a1c2ba279bd1', 'keypoint': {'x': 366.3255, 'y': 546.3679}, 'name': 'blue'}, {'id': 'a3036737-fb7e-4f86-88c8-b0c5262cc051', 'keypoint': {'x': 379.5566, 'y': 438.9623}, 'name': 'blue'}, {'id': '5370f3f3-ef52-45e8-bdb4-3d96d54f4bd9', 'keypoint': {'x': 455.8302, 'y': 449.0802}, 'name': 'blue'}, {'id': '95294165-9354-41e7-b1be-60d4033bd1a3', 'keypoint': {'x': 318.0708, 'y': 454.5283}, 'name': 'yellow'}, {'id': '71f49ac9-1905-457f-9219-701cac7c42e4', 'keypoint': {'x': 318.0708, 'y': 530.8019}, 'name': 'yellow'}, {'id': '6f7a8a19-794f-4ffc-80ed-8a42dedeab0f', 'keypoint': {'x': 463.6132, 'y': 524.5755}, 'name': 'yellow'}, {'id': '407068c6-6a1e-4a14-b9d0-c0a4ed13b9c2', 'keypoint': {'x': 395.1226, 'y': 466.2028}, 'name': 'green'}, {'id': '847fb0f9-02cc-42df-8984-2b193e2abb8d', 'keypoint': {'x': 403.684, 'y': 523.0189}, 'name': 'green'}, {'id': 'bd78738b-2c39-4b85-9575-960d46ecd66c', 'keypoint': {'x': 325.0755, 'y': 500.4481}, 'name': 'orange'}, {'id': 'dafb3b80-4c1e-48a7-b398-e3aa56c5dcb0', 'keypoint': {'x': 374.1085, 'y': 515.2358}, 'name': 'orange'}, {'id': '36cf3f26-cac5-4368-91eb-ae8937ec2f91', 'keypoint': {'x': 412.2453, 'y': 427.2877}, 'name': 'orange'}, {'id': 'f93ff614-2250-4f4d-8ec0-ac87ec62305c', 'keypoint': {'x': 436.3726, 'y': 437.4057}, 'name': 'orange'}, {'id': '09066da9-88d0-4617-8cf6-7d604ee0190d', 'keypoint': {'x': 485.4057, 'y': 510.566}, 'name': 'orange'}]}
[/INST]
{'x': 329.7453, 'y': 557.2642}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the yellow block that is between two green blocks, and below a blue block.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'fe98e63d-c226-4a25-9e75-229c9a2d828a', 'keypoint': {'x': 377.2217, 'y': 28.7972}, 'name': 'you'}, {'id': 'c1bfc6fe-9e0d-4152-aab0-9efe03f99ee1', 'keypoint': {'x': 291.6085, 'y': 487.217}, 'name': 'yellow'}, {'id': 'eefd5787-08ef-4674-ae9e-c41f4a2ed951', 'keypoint': {'x': 314.1792, 'y': 539.3632}, 'name': 'yellow'}, {'id': '8987e25d-8eb0-42ab-aa8e-390a7b046e44', 'keypoint': {'x': 363.2123, 'y': 544.8113}, 'name': 'yellow'}, {'id': 'af54d764-7469-4b39-95f2-d25ac6241715', 'keypoint': {'x': 434.0377, 'y': 542.4764}, 'name': 'yellow'}, {'id': 'c426887c-43fa-4c7c-9ee6-447f48c5b353', 'keypoint': {'x': 463.6132, 'y': 487.9953}, 'name': 'yellow'}, {'id': 'f1402f90-fe95-42c0-93e5-75019494beba', 'keypoint': {'x': 497.0802, 'y': 442.0755}, 'name': 'yellow'}, {'id': 'ad7b2c3b-efac-4705-8dc4-076514da9f0d', 'keypoint': {'x': 319.6274, 'y': 463.8679}, 'name': 'blue'}, {'id': '57fd11a6-5352-4e90-8c5e-78e080a9f63b', 'keypoint': {'x': 458.1651, 'y': 516.0142}, 'name': 'blue'}, {'id': 'bffb87fb-647e-44e1-b4c0-9bbf464cc4da', 'keypoint': {'x': 353.8726, 'y': 445.1887}, 'name': 'green'}, {'id': '2be751d9-34f0-411b-89e7-1b20b054e1ee', 'keypoint': {'x': 388.8962, 'y': 498.1132}, 'name': 'green'}, {'id': '98a41430-e570-492e-969a-8ea45857d765', 'keypoint': {'x': 395.9009, 'y': 533.1368}, 'name': 'green'}, {'id': '0327cb90-c17c-4528-8964-e978fbff5999', 'keypoint': {'x': 427.8113, 'y': 484.8821}, 'name': 'green'}, {'id': '97f91dd8-6cd8-4194-8783-e9e3b5786e68', 'keypoint': {'x': 456.6085, 'y': 452.9717}, 'name': 'green'}, {'id': '4ebadfd4-3497-4f80-a4e9-a870c25786bd', 'keypoint': {'x': 504.8632, 'y': 502.0047}, 'name': 'green'}, {'id': 'd2ff782c-210c-4090-91a5-de43a78f1232', 'keypoint': {'x': 331.3019, 'y': 512.9009}, 'name': 'orange'}]}
[/INST]
{'x': 463.6132, 'y': 487.9953}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:pick up the green block closest to me
 {'width': 756, 'height': 660, 'annotations': [{'id': 'cad61a18-85b8-4f6e-93b3-c3ce9e28cda2', 'keypoint': {'x': 316.5142, 'y': 554.9292}, 'name': 'orange'}, {'id': '17682384-25d2-46d1-ae87-f04077458497', 'keypoint': {'x': 330.5236, 'y': 516.7925}, 'name': 'orange'}, {'id': '5d36ae41-004b-494e-b32d-33279b1dbfee', 'keypoint': {'x': 447.2689, 'y': 457.6415}, 'name': 'orange'}, {'id': 'fee58518-97d8-4c98-9b1b-ebeaa7ef3967', 'keypoint': {'x': 488.5189, 'y': 530.8019}, 'name': 'orange'}, {'id': '4fa5b6b4-f98b-4588-95e6-8255a4bef84b', 'keypoint': {'x': 380.3349, 'y': 446.7453}, 'name': 'yellow'}, {'id': '22f0cd3c-99be-40d9-9b3f-a7bc782da1f8', 'keypoint': {'x': 297.8349, 'y': 529.2453}, 'name': 'yellow'}, {'id': 'd5f859c2-a998-4ee4-bff1-d6025abd4883', 'keypoint': {'x': 375.6651, 'y': 536.25}, 'name': 'yellow'}, {'id': '96691580-8803-4b4b-b31f-4920d402e03e', 'keypoint': {'x': 473.7311, 'y': 435.0708}, 'name': 'yellow'}, {'id': '86653055-abe8-4a82-8b79-fb7f461438c0', 'keypoint': {'x': 474.5094, 'y': 487.217}, 'name': 'yellow'}, {'id': '34e55929-db77-4549-8f50-5adc4da9bd5d', 'keypoint': {'x': 370.9953, 'y': 492.6651}, 'name': 'blue'}, {'id': '500fbe66-22dd-4552-a43c-63bf2efeb1c9', 'keypoint': {'x': 417.6934, 'y': 477.0991}, 'name': 'blue'}, {'id': '85cb36b6-c443-488a-85b6-363a2284870a', 'keypoint': {'x': 345.3113, 'y': 424.9528}, 'name': 'green'}, {'id': '1c037117-1e04-4cd7-a544-c3e10b3639bb', 'keypoint': {'x': 295.5, 'y': 466.9811}, 'name': 'green'}, {'id': 'a8f28480-33ff-4a0a-883f-f1f226ced914', 'keypoint': {'x': 299.3915, 'y': 501.2264}, 'name': 'green'}, {'id': 'b9578593-4dcd-4808-9119-c6439b45d587', 'keypoint': {'x': 427.033, 'y': 534.6934}, 'name': 'green'}, {'id': '70024c2f-19b2-412a-a76d-d9e741635b7e', 'keypoint': {'x': 380.3349, 'y': 33.467}, 'name': 'you'}]}
[/INST]
{'x': 427.033, 'y': 534.6934}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the box furthest to your left.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'b1c926ae-92c1-43eb-96c5-76c41d0fab06', 'keypoint': {'x': 342.1981, 'y': 437.4057}, 'name': 'orange'}, {'id': '76267614-95a1-4ded-b4e9-354b821f62bb', 'keypoint': {'x': 314.1792, 'y': 499.6698}, 'name': 'orange'}, {'id': '3124e476-176a-4feb-8844-b3973d48973a', 'keypoint': {'x': 453.4953, 'y': 463.8679}, 'name': 'orange'}, {'id': '728bd5ee-ece2-45c9-b0d7-b887692bb453', 'keypoint': {'x': 494.7453, 'y': 534.6934}, 'name': 'orange'}, {'id': 'da0db0c2-fdee-4550-8e2a-11f1aace764f', 'keypoint': {'x': 376.4434, 'y': 419.5047}, 'name': 'yellow'}, {'id': 'bd27285d-35e5-47b5-9b1e-78bd5fb9e595', 'keypoint': {'x': 360.0991, 'y': 516.7925}, 'name': 'yellow'}, {'id': '5a97f3d8-463a-4c65-9906-b8cf5f772e2b', 'keypoint': {'x': 408.3538, 'y': 526.9104}, 'name': 'yellow'}, {'id': 'f2444027-7c76-40d2-9424-36c46342a49f', 'keypoint': {'x': 318.8491, 'y': 449.0802}, 'name': 'blue'}, {'id': '0b2e3fca-8a71-43d2-a83e-f02af09891b9', 'keypoint': {'x': 315.7358, 'y': 480.2123}, 'name': 'blue'}, {'id': '897e21a9-0e62-42b6-8c01-826ad479026c', 'keypoint': {'x': 378.0, 'y': 490.3302}, 'name': 'blue'}, {'id': 'ca6c0f72-0600-47ba-b4b1-05dc3566a83e', 'keypoint': {'x': 402.1274, 'y': 463.0896}, 'name': 'blue'}, {'id': '41397d83-cab1-4c4d-ba20-b13a1c93f043', 'keypoint': {'x': 419.25, 'y': 432.7358}, 'name': 'blue'}, {'id': '051bcea0-383b-4791-9652-d84f6ef3fe4c', 'keypoint': {'x': 348.4245, 'y': 488.7736}, 'name': 'green'}, {'id': '58f619eb-2384-4a80-97f9-10520e5226a9', 'keypoint': {'x': 311.8443, 'y': 539.3632}, 'name': 'green'}, {'id': 'b9c15fa4-b07d-4f16-9f7d-f007698c83c2', 'keypoint': {'x': 417.6934, 'y': 502.0047}, 'name': 'green'}, {'id': '5c0026dd-e833-4ad7-8339-361bad3686e0', 'keypoint': {'x': 378.0, 'y': 59.1509}, 'name': 'you'}]}
[/INST]
{'x': 494.7453, 'y': 534.6934}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:on your right side there is a blue block at the end of the table, grab yellow block that's touching it.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'b328ac50-08b5-486e-a937-2cddbc04245c', 'keypoint': {'x': 304.8396, 'y': 434.2925}, 'name': 'orange'}, {'id': '7344b92c-5683-4670-9cef-33f4cf8e73fb', 'keypoint': {'x': 472.9528, 'y': 456.0849}, 'name': 'orange'}, {'id': '6958bbb7-3dce-4cfe-9049-02ebc5a32e43', 'keypoint': {'x': 306.3962, 'y': 460.7547}, 'name': 'yellow'}, {'id': '62285f29-522c-498a-9fdb-53e2e8421ea5', 'keypoint': {'x': 330.5236, 'y': 470.8726}, 'name': 'yellow'}, {'id': 'e4402e53-f01e-4254-a874-f6ffd3892ac4', 'keypoint': {'x': 381.8915, 'y': 475.5425}, 'name': 'yellow'}, {'id': '3f3ed2fa-5312-418f-9f29-f4750a72aa9e', 'keypoint': {'x': 402.9057, 'y': 435.0708}, 'name': 'yellow'}, {'id': 'fe5ba8f4-1f70-4712-ae25-cd20ac9670d6', 'keypoint': {'x': 455.8302, 'y': 481.7689}, 'name': 'yellow'}, {'id': 'bd1c4ddb-8345-4f24-a982-f6cd6a64fb0e', 'keypoint': {'x': 501.75, 'y': 505.8962}, 'name': 'yellow'}, {'id': '8f332be0-ad10-4130-9840-f3a1dc330890', 'keypoint': {'x': 352.316, 'y': 513.6792}, 'name': 'yellow'}, {'id': '5b19adbf-7efb-4a90-9f40-4c0bf2f3b9a8', 'keypoint': {'x': 324.2972, 'y': 499.6698}, 'name': 'yellow'}, {'id': '7f124547-f8b7-4280-b7e1-401876ce2bdb', 'keypoint': {'x': 308.7311, 'y': 552.5943}, 'name': 'yellow'}, {'id': '8a1fd773-bc7c-4be5-a292-c1c53d5c77df', 'keypoint': {'x': 298.6132, 'y': 491.8868}, 'name': 'blue'}, {'id': '7be8714d-676d-466e-a70a-e9d078b757f4', 'keypoint': {'x': 381.1132, 'y': 497.3349}, 'name': 'blue'}, {'id': 'b1ee1721-f1d2-4cc8-828b-2e6c5072d54f', 'keypoint': {'x': 451.9387, 'y': 432.7358}, 'name': 'blue'}, {'id': '0416eebd-320e-40d8-bbc7-a94a6333aa95', 'keypoint': {'x': 497.0802, 'y': 485.6604}, 'name': 'green'}, {'id': '2240b1ab-d515-4872-a587-09ca836dc397', 'keypoint': {'x': 378.7783, 'y': 28.7972}, 'name': 'you'}]}
[/INST]
{'x': 324.2972, 'y': 499.6698}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:pick up the farthest away green block
 {'width': 756, 'height': 660, 'annotations': [{'id': 'f8ce6e33-d9e3-4687-bd11-4537937b3def', 'keypoint': {'x': 313.4009, 'y': 480.9906}, 'name': 'orange'}, {'id': '8758b8e0-78a7-4dc2-b595-9eec97d96b3c', 'keypoint': {'x': 403.684, 'y': 466.9811}, 'name': 'orange'}, {'id': '9024f068-b46e-400e-bf69-507f61b74f80', 'keypoint': {'x': 459.7217, 'y': 452.9717}, 'name': 'orange'}, {'id': 'd7ab991d-e3d6-4ffb-8d9b-9cffbb59b9c6', 'keypoint': {'x': 502.5283, 'y': 457.6415}, 'name': 'orange'}, {'id': 'b9fb860f-d2d0-4b4a-a275-416d46a99ab8', 'keypoint': {'x': 335.9717, 'y': 450.6368}, 'name': 'yellow'}, {'id': '7404a736-3ae3-4f5c-b40d-5d5af4635c6e', 'keypoint': {'x': 316.5142, 'y': 531.5802}, 'name': 'yellow'}, {'id': 'd2d6b9ff-fd85-48a1-b368-3f0e1d09cc6f', 'keypoint': {'x': 396.6792, 'y': 491.8868}, 'name': 'yellow'}, {'id': 'c4551bcd-78f8-4205-b828-5b040d49a2c3', 'keypoint': {'x': 444.1557, 'y': 433.5142}, 'name': 'yellow'}, {'id': '3e0cda0a-1e8a-4472-8984-7886c02ff560', 'keypoint': {'x': 347.6462, 'y': 514.4575}, 'name': 'blue'}, {'id': '266b6be1-582b-4c4d-8544-906fad90cbd2', 'keypoint': {'x': 359.3208, 'y': 478.6557}, 'name': 'blue'}, {'id': 'c58796fa-46b4-4231-8589-0612cd52983f', 'keypoint': {'x': 395.9009, 'y': 444.4104}, 'name': 'blue'}, {'id': '3de58fe6-02ae-4e5a-b0eb-ac1e3ac60960', 'keypoint': {'x': 444.934, 'y': 537.8066}, 'name': 'blue'}, {'id': 'a1a96c53-7f31-4121-8983-bf6a69f42935', 'keypoint': {'x': 502.5283, 'y': 541.6981}, 'name': 'blue'}, {'id': 'c2e223f1-9da7-4029-ab29-81edb6be3c94', 'keypoint': {'x': 355.4292, 'y': 424.9528}, 'name': 'green'}, {'id': 'd1f0facf-050a-4a92-bfb1-1367c5fdd80d', 'keypoint': {'x': 466.7264, 'y': 500.4481}, 'name': 'green'}, {'id': '727b2df8-1454-47db-9eff-8cfe072974ff', 'keypoint': {'x': 383.4481, 'y': 44.3632}, 'name': 'you'}]}
[/INST]
{'x': 466.7264, 'y': 500.4481}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:In front of you is a green block. It is right in the middle of the collection of blocks. Please pick up the blue block that is right above that green block.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'bb2508d6-bab0-4991-a4b4-1d60addd89be', 'keypoint': {'x': 374.8868, 'y': 32.6887}, 'name': 'you'}, {'id': '80bad823-3c11-42a6-9b7a-8f92fd053db8', 'keypoint': {'x': 286.1604, 'y': 504.3396}, 'name': 'yellow'}, {'id': '74407b61-1d02-4456-a3be-f44e6d0621e4', 'keypoint': {'x': 427.8113, 'y': 511.3443}, 'name': 'yellow'}, {'id': '0c043462-b620-40d3-b17e-3f17c7108248', 'keypoint': {'x': 391.2311, 'y': 435.8491}, 'name': 'green'}, {'id': 'e97e7600-f671-4774-970c-bcd8e011471e', 'keypoint': {'x': 311.066, 'y': 530.8019}, 'name': 'green'}, {'id': '3c45bdee-a533-4cab-8e50-cd3e05ea379a', 'keypoint': {'x': 375.6651, 'y': 518.3491}, 'name': 'green'}, {'id': 'fbe906a6-d83a-45bc-bd1e-7c81ccde8b9b', 'keypoint': {'x': 487.7406, 'y': 473.2075}, 'name': 'green'}, {'id': '662455ed-994d-45d8-b864-0f07ccfe9a1e', 'keypoint': {'x': 309.5094, 'y': 477.8774}, 'name': 'blue'}, {'id': '92e619ee-cc43-4ba6-80a0-fd23cabc2b68', 'keypoint': {'x': 378.7783, 'y': 457.6415}, 'name': 'blue'}, {'id': '4d3a4397-37ee-4ea1-bff7-ea4e40e4ba91', 'keypoint': {'x': 395.1226, 'y': 483.3255}, 'name': 'blue'}, {'id': '83c6c716-2497-4e0e-b233-3cda2dca8049', 'keypoint': {'x': 438.7075, 'y': 448.3019}, 'name': 'blue'}, {'id': '24b23d82-8281-40fd-9942-dc852b76fea4', 'keypoint': {'x': 444.1557, 'y': 484.1038}, 'name': 'blue'}, {'id': '541179c1-e37f-4069-99a8-fe58be3053c6', 'keypoint': {'x': 466.7264, 'y': 502.783}, 'name': 'blue'}, {'id': 'd950b6f4-0fff-425b-bf69-419e3a15863f', 'keypoint': {'x': 343.7547, 'y': 543.2547}, 'name': 'orange'}, {'id': 'a5aa3394-3b02-4613-8a7d-f14c9cd0fa58', 'keypoint': {'x': 431.7028, 'y': 539.3632}, 'name': 'orange'}, {'id': 'd7eb7db6-f69d-4df7-861e-0cc9dbebac2a', 'keypoint': {'x': 484.6274, 'y': 532.3585}, 'name': 'orange'}]}
[/INST]
{'x': 378.7783, 'y': 457.6415}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the blue block that is next to the other blue block and by the orange block.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'c7519096-f7ea-4ab1-a66d-97c708976599', 'keypoint': {'x': 380.3349, 'y': 34.2453}, 'name': 'you'}, {'id': 'a95968ee-51d4-482d-ae79-97cb54663aed', 'keypoint': {'x': 293.9434, 'y': 479.434}, 'name': 'blue'}, {'id': '2fc8b299-6074-43eb-92bc-7ed758a958fc', 'keypoint': {'x': 329.7453, 'y': 557.2642}, 'name': 'blue'}, {'id': '11dbc045-de4f-4d3a-bc50-a1c2ba279bd1', 'keypoint': {'x': 366.3255, 'y': 546.3679}, 'name': 'blue'}, {'id': 'a3036737-fb7e-4f86-88c8-b0c5262cc051', 'keypoint': {'x': 379.5566, 'y': 438.9623}, 'name': 'blue'}, {'id': '5370f3f3-ef52-45e8-bdb4-3d96d54f4bd9', 'keypoint': {'x': 455.8302, 'y': 449.0802}, 'name': 'blue'}, {'id': '95294165-9354-41e7-b1be-60d4033bd1a3', 'keypoint': {'x': 318.0708, 'y': 454.5283}, 'name': 'yellow'}, {'id': '71f49ac9-1905-457f-9219-701cac7c42e4', 'keypoint': {'x': 318.0708, 'y': 530.8019}, 'name': 'yellow'}, {'id': '6f7a8a19-794f-4ffc-80ed-8a42dedeab0f', 'keypoint': {'x': 463.6132, 'y': 524.5755}, 'name': 'yellow'}, {'id': '407068c6-6a1e-4a14-b9d0-c0a4ed13b9c2', 'keypoint': {'x': 395.1226, 'y': 466.2028}, 'name': 'green'}, {'id': '847fb0f9-02cc-42df-8984-2b193e2abb8d', 'keypoint': {'x': 403.684, 'y': 523.0189}, 'name': 'green'}, {'id': 'bd78738b-2c39-4b85-9575-960d46ecd66c', 'keypoint': {'x': 325.0755, 'y': 500.4481}, 'name': 'orange'}, {'id': 'dafb3b80-4c1e-48a7-b398-e3aa56c5dcb0', 'keypoint': {'x': 374.1085, 'y': 515.2358}, 'name': 'orange'}, {'id': '36cf3f26-cac5-4368-91eb-ae8937ec2f91', 'keypoint': {'x': 412.2453, 'y': 427.2877}, 'name': 'orange'}, {'id': 'f93ff614-2250-4f4d-8ec0-ac87ec62305c', 'keypoint': {'x': 436.3726, 'y': 437.4057}, 'name': 'orange'}, {'id': '09066da9-88d0-4617-8cf6-7d604ee0190d', 'keypoint': {'x': 485.4057, 'y': 510.566}, 'name': 'orange'}]}
[/INST]
{'x': 366.3255, 'y': 546.3679}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the green block that is in the middle of the three green blocks in a straight line.
 {'width': 756, 'height': 660, 'annotations': [{'id': '8beb9a9e-a451-450a-b594-7a9ae18e74f9', 'keypoint': {'x': 458.9434, 'y': 531.5802}, 'name': 'orange'}, {'id': 'ee956837-2183-4b52-8aa2-dfa990f1d376', 'keypoint': {'x': 386.5613, 'y': 529.2453}, 'name': 'yellow'}, {'id': '427a5df9-dc11-494e-8632-81ffcd5e66c8', 'keypoint': {'x': 471.3962, 'y': 453.75}, 'name': 'yellow'}, {'id': 'bcc302a4-c191-4bc2-82e0-1ceedd4a2928', 'keypoint': {'x': 329.7453, 'y': 453.75}, 'name': 'blue'}, {'id': '8584b004-32b4-47bd-9100-a2eab16c5301', 'keypoint': {'x': 294.7217, 'y': 501.2264}, 'name': 'blue'}, {'id': 'ba4430e4-6293-47e3-b512-0e78897c96fb', 'keypoint': {'x': 317.2925, 'y': 520.684}, 'name': 'blue'}, {'id': '9c57d65f-59e8-4d45-a882-721a56841969', 'keypoint': {'x': 433.2594, 'y': 498.1132}, 'name': 'blue'}, {'id': '0e241c7c-e225-4266-9044-a65451aceb68', 'keypoint': {'x': 361.6557, 'y': 428.8443}, 'name': 'green'}, {'id': '050252d9-484a-4a87-a46c-1f69f5d7ea14', 'keypoint': {'x': 416.9151, 'y': 435.8491}, 'name': 'green'}, {'id': '5b9ab273-0e91-4589-9a86-d2969adc4e30', 'keypoint': {'x': 435.5943, 'y': 445.967}, 'name': 'green'}, {'id': '4b452968-0027-4b94-92e6-e8dcd32d1e25', 'keypoint': {'x': 468.283, 'y': 489.5519}, 'name': 'green'}, {'id': '3ab98d12-5e5c-4bdf-931d-55f2ee2d23b2', 'keypoint': {'x': 386.5613, 'y': 498.8915}, 'name': 'green'}, {'id': '416f60f7-59aa-46b3-858c-7dfde5e23875', 'keypoint': {'x': 353.0943, 'y': 497.3349}, 'name': 'green'}, {'id': 'ce8faeb7-52eb-4dc8-8db9-4295f1b077e3', 'keypoint': {'x': 329.7453, 'y': 491.8868}, 'name': 'green'}, {'id': '6ea4f70c-aa80-4f14-a1ac-d26081a22a0d', 'keypoint': {'x': 379.5566, 'y': 52.1462}, 'name': 'you'}]}
[/INST]
{'x': 353.0943, 'y': 497.3349}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:It is more towards the right side of the table. It is the 4th block from the top on the right side.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'fe98e63d-c226-4a25-9e75-229c9a2d828a', 'keypoint': {'x': 377.2217, 'y': 28.7972}, 'name': 'you'}, {'id': 'c1bfc6fe-9e0d-4152-aab0-9efe03f99ee1', 'keypoint': {'x': 291.6085, 'y': 487.217}, 'name': 'yellow'}, {'id': 'eefd5787-08ef-4674-ae9e-c41f4a2ed951', 'keypoint': {'x': 314.1792, 'y': 539.3632}, 'name': 'yellow'}, {'id': '8987e25d-8eb0-42ab-aa8e-390a7b046e44', 'keypoint': {'x': 363.2123, 'y': 544.8113}, 'name': 'yellow'}, {'id': 'af54d764-7469-4b39-95f2-d25ac6241715', 'keypoint': {'x': 434.0377, 'y': 542.4764}, 'name': 'yellow'}, {'id': 'c426887c-43fa-4c7c-9ee6-447f48c5b353', 'keypoint': {'x': 463.6132, 'y': 487.9953}, 'name': 'yellow'}, {'id': 'f1402f90-fe95-42c0-93e5-75019494beba', 'keypoint': {'x': 497.0802, 'y': 442.0755}, 'name': 'yellow'}, {'id': 'ad7b2c3b-efac-4705-8dc4-076514da9f0d', 'keypoint': {'x': 319.6274, 'y': 463.8679}, 'name': 'blue'}, {'id': '57fd11a6-5352-4e90-8c5e-78e080a9f63b', 'keypoint': {'x': 458.1651, 'y': 516.0142}, 'name': 'blue'}, {'id': 'bffb87fb-647e-44e1-b4c0-9bbf464cc4da', 'keypoint': {'x': 353.8726, 'y': 445.1887}, 'name': 'green'}, {'id': '2be751d9-34f0-411b-89e7-1b20b054e1ee', 'keypoint': {'x': 388.8962, 'y': 498.1132}, 'name': 'green'}, {'id': '98a41430-e570-492e-969a-8ea45857d765', 'keypoint': {'x': 395.9009, 'y': 533.1368}, 'name': 'green'}, {'id': '0327cb90-c17c-4528-8964-e978fbff5999', 'keypoint': {'x': 427.8113, 'y': 484.8821}, 'name': 'green'}, {'id': '97f91dd8-6cd8-4194-8783-e9e3b5786e68', 'keypoint': {'x': 456.6085, 'y': 452.9717}, 'name': 'green'}, {'id': '4ebadfd4-3497-4f80-a4e9-a870c25786bd', 'keypoint': {'x': 504.8632, 'y': 502.0047}, 'name': 'green'}, {'id': 'd2ff782c-210c-4090-91a5-de43a78f1232', 'keypoint': {'x': 331.3019, 'y': 512.9009}, 'name': 'orange'}]}
[/INST]
{'x': 463.6132, 'y': 487.9953}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the yellow block that is between a green block and a blue block.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'fe98e63d-c226-4a25-9e75-229c9a2d828a', 'keypoint': {'x': 377.2217, 'y': 28.7972}, 'name': 'you'}, {'id': 'c1bfc6fe-9e0d-4152-aab0-9efe03f99ee1', 'keypoint': {'x': 291.6085, 'y': 487.217}, 'name': 'yellow'}, {'id': 'eefd5787-08ef-4674-ae9e-c41f4a2ed951', 'keypoint': {'x': 314.1792, 'y': 539.3632}, 'name': 'yellow'}, {'id': '8987e25d-8eb0-42ab-aa8e-390a7b046e44', 'keypoint': {'x': 363.2123, 'y': 544.8113}, 'name': 'yellow'}, {'id': 'af54d764-7469-4b39-95f2-d25ac6241715', 'keypoint': {'x': 434.0377, 'y': 542.4764}, 'name': 'yellow'}, {'id': 'c426887c-43fa-4c7c-9ee6-447f48c5b353', 'keypoint': {'x': 463.6132, 'y': 487.9953}, 'name': 'yellow'}, {'id': 'f1402f90-fe95-42c0-93e5-75019494beba', 'keypoint': {'x': 497.0802, 'y': 442.0755}, 'name': 'yellow'}, {'id': 'ad7b2c3b-efac-4705-8dc4-076514da9f0d', 'keypoint': {'x': 319.6274, 'y': 463.8679}, 'name': 'blue'}, {'id': '57fd11a6-5352-4e90-8c5e-78e080a9f63b', 'keypoint': {'x': 458.1651, 'y': 516.0142}, 'name': 'blue'}, {'id': 'bffb87fb-647e-44e1-b4c0-9bbf464cc4da', 'keypoint': {'x': 353.8726, 'y': 445.1887}, 'name': 'green'}, {'id': '2be751d9-34f0-411b-89e7-1b20b054e1ee', 'keypoint': {'x': 388.8962, 'y': 498.1132}, 'name': 'green'}, {'id': '98a41430-e570-492e-969a-8ea45857d765', 'keypoint': {'x': 395.9009, 'y': 533.1368}, 'name': 'green'}, {'id': '0327cb90-c17c-4528-8964-e978fbff5999', 'keypoint': {'x': 427.8113, 'y': 484.8821}, 'name': 'green'}, {'id': '97f91dd8-6cd8-4194-8783-e9e3b5786e68', 'keypoint': {'x': 456.6085, 'y': 452.9717}, 'name': 'green'}, {'id': '4ebadfd4-3497-4f80-a4e9-a870c25786bd', 'keypoint': {'x': 504.8632, 'y': 502.0047}, 'name': 'green'}, {'id': 'd2ff782c-210c-4090-91a5-de43a78f1232', 'keypoint': {'x': 331.3019, 'y': 512.9009}, 'name': 'orange'}]}
[/INST]
{'x': 463.6132, 'y': 487.9953}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the orange block that is between a blue and a yellow block
 {'width': 756, 'height': 660, 'annotations': [{'id': 'b1c926ae-92c1-43eb-96c5-76c41d0fab06', 'keypoint': {'x': 342.1981, 'y': 437.4057}, 'name': 'orange'}, {'id': '76267614-95a1-4ded-b4e9-354b821f62bb', 'keypoint': {'x': 314.1792, 'y': 499.6698}, 'name': 'orange'}, {'id': '3124e476-176a-4feb-8844-b3973d48973a', 'keypoint': {'x': 453.4953, 'y': 463.8679}, 'name': 'orange'}, {'id': '728bd5ee-ece2-45c9-b0d7-b887692bb453', 'keypoint': {'x': 494.7453, 'y': 534.6934}, 'name': 'orange'}, {'id': 'da0db0c2-fdee-4550-8e2a-11f1aace764f', 'keypoint': {'x': 376.4434, 'y': 419.5047}, 'name': 'yellow'}, {'id': 'bd27285d-35e5-47b5-9b1e-78bd5fb9e595', 'keypoint': {'x': 360.0991, 'y': 516.7925}, 'name': 'yellow'}, {'id': '5a97f3d8-463a-4c65-9906-b8cf5f772e2b', 'keypoint': {'x': 408.3538, 'y': 526.9104}, 'name': 'yellow'}, {'id': 'f2444027-7c76-40d2-9424-36c46342a49f', 'keypoint': {'x': 318.8491, 'y': 449.0802}, 'name': 'blue'}, {'id': '0b2e3fca-8a71-43d2-a83e-f02af09891b9', 'keypoint': {'x': 315.7358, 'y': 480.2123}, 'name': 'blue'}, {'id': '897e21a9-0e62-42b6-8c01-826ad479026c', 'keypoint': {'x': 378.0, 'y': 490.3302}, 'name': 'blue'}, {'id': 'ca6c0f72-0600-47ba-b4b1-05dc3566a83e', 'keypoint': {'x': 402.1274, 'y': 463.0896}, 'name': 'blue'}, {'id': '41397d83-cab1-4c4d-ba20-b13a1c93f043', 'keypoint': {'x': 419.25, 'y': 432.7358}, 'name': 'blue'}, {'id': '051bcea0-383b-4791-9652-d84f6ef3fe4c', 'keypoint': {'x': 348.4245, 'y': 488.7736}, 'name': 'green'}, {'id': '58f619eb-2384-4a80-97f9-10520e5226a9', 'keypoint': {'x': 311.8443, 'y': 539.3632}, 'name': 'green'}, {'id': 'b9c15fa4-b07d-4f16-9f7d-f007698c83c2', 'keypoint': {'x': 417.6934, 'y': 502.0047}, 'name': 'green'}, {'id': '5c0026dd-e833-4ad7-8339-361bad3686e0', 'keypoint': {'x': 378.0, 'y': 59.1509}, 'name': 'you'}]}
[/INST]
{'x': 342.1981, 'y': 437.4057}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the green block that is not with the other green blocks.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'cad61a18-85b8-4f6e-93b3-c3ce9e28cda2', 'keypoint': {'x': 316.5142, 'y': 554.9292}, 'name': 'orange'}, {'id': '17682384-25d2-46d1-ae87-f04077458497', 'keypoint': {'x': 330.5236, 'y': 516.7925}, 'name': 'orange'}, {'id': '5d36ae41-004b-494e-b32d-33279b1dbfee', 'keypoint': {'x': 447.2689, 'y': 457.6415}, 'name': 'orange'}, {'id': 'fee58518-97d8-4c98-9b1b-ebeaa7ef3967', 'keypoint': {'x': 488.5189, 'y': 530.8019}, 'name': 'orange'}, {'id': '4fa5b6b4-f98b-4588-95e6-8255a4bef84b', 'keypoint': {'x': 380.3349, 'y': 446.7453}, 'name': 'yellow'}, {'id': '22f0cd3c-99be-40d9-9b3f-a7bc782da1f8', 'keypoint': {'x': 297.8349, 'y': 529.2453}, 'name': 'yellow'}, {'id': 'd5f859c2-a998-4ee4-bff1-d6025abd4883', 'keypoint': {'x': 375.6651, 'y': 536.25}, 'name': 'yellow'}, {'id': '96691580-8803-4b4b-b31f-4920d402e03e', 'keypoint': {'x': 473.7311, 'y': 435.0708}, 'name': 'yellow'}, {'id': '86653055-abe8-4a82-8b79-fb7f461438c0', 'keypoint': {'x': 474.5094, 'y': 487.217}, 'name': 'yellow'}, {'id': '34e55929-db77-4549-8f50-5adc4da9bd5d', 'keypoint': {'x': 370.9953, 'y': 492.6651}, 'name': 'blue'}, {'id': '500fbe66-22dd-4552-a43c-63bf2efeb1c9', 'keypoint': {'x': 417.6934, 'y': 477.0991}, 'name': 'blue'}, {'id': '85cb36b6-c443-488a-85b6-363a2284870a', 'keypoint': {'x': 345.3113, 'y': 424.9528}, 'name': 'green'}, {'id': '1c037117-1e04-4cd7-a544-c3e10b3639bb', 'keypoint': {'x': 295.5, 'y': 466.9811}, 'name': 'green'}, {'id': 'a8f28480-33ff-4a0a-883f-f1f226ced914', 'keypoint': {'x': 299.3915, 'y': 501.2264}, 'name': 'green'}, {'id': 'b9578593-4dcd-4808-9119-c6439b45d587', 'keypoint': {'x': 427.033, 'y': 534.6934}, 'name': 'green'}, {'id': '70024c2f-19b2-412a-a76d-d9e741635b7e', 'keypoint': {'x': 380.3349, 'y': 33.467}, 'name': 'you'}]}
[/INST]
{'x': 427.033, 'y': 534.6934}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the green block closest to you on your left
 {'width': 756, 'height': 660, 'annotations': [{'id': '8f941e4d-8e6b-4e37-a2cd-f19b3dbb1dda', 'keypoint': {'x': 293.1651, 'y': 452.9717}, 'name': 'orange'}, {'id': '77f8b560-0c54-4923-9c65-8e8973d5cc1a', 'keypoint': {'x': 293.9434, 'y': 485.6604}, 'name': 'orange'}, {'id': 'c2fdf254-12fa-43ee-9d04-fd097d6a2fad', 'keypoint': {'x': 332.8585, 'y': 517.5708}, 'name': 'orange'}, {'id': 'ee022624-310e-4867-8360-cd61e6a5e8ec', 'keypoint': {'x': 325.0755, 'y': 557.2642}, 'name': 'orange'}, {'id': 'ca78545f-1817-447e-a767-1ec186e40025', 'keypoint': {'x': 377.2217, 'y': 478.6557}, 'name': 'orange'}, {'id': 'b7ea891b-3095-4df8-a718-b193f34c851c', 'keypoint': {'x': 386.5613, 'y': 492.6651}, 'name': 'orange'}, {'id': '54b85951-9b7d-4a22-9b94-5856c8f4cc7e', 'keypoint': {'x': 441.8208, 'y': 499.6698}, 'name': 'orange'}, {'id': 'ef2b7cbb-fa7e-4894-8e50-310d43a6f3eb', 'keypoint': {'x': 385.0047, 'y': 448.3019}, 'name': 'yellow'}, {'id': 'ab7d8b08-4eaf-41cf-8066-9aba376bb546', 'keypoint': {'x': 392.0094, 'y': 523.0189}, 'name': 'yellow'}, {'id': 'aec059f3-2def-4458-9ea4-3fa5916cab5b', 'keypoint': {'x': 466.7264, 'y': 442.8538}, 'name': 'yellow'}, {'id': '39fe094d-1e93-4828-ae7e-d672aeb88451', 'keypoint': {'x': 356.2075, 'y': 533.9151}, 'name': 'blue'}, {'id': '57cde523-a685-45a5-b623-1b11057840da', 'keypoint': {'x': 487.7406, 'y': 488.7736}, 'name': 'blue'}, {'id': 'da91e8aa-4087-4689-95ed-80e4ba9a93ec', 'keypoint': {'x': 327.4104, 'y': 472.4292}, 'name': 'green'}, {'id': '9406ceb5-4de6-4915-8da6-132b32721d7c', 'keypoint': {'x': 508.7547, 'y': 511.3443}, 'name': 'green'}, {'id': 'fd93334e-be71-4598-9c11-abb001661b87', 'keypoint': {'x': 497.0802, 'y': 560.3774}, 'name': 'green'}, {'id': '3067ae97-d698-4a21-82f6-e94a33ed51a0', 'keypoint': {'x': 380.3349, 'y': 59.1509}, 'name': 'you'}]}
[/INST]
{'x': 508.7547, 'y': 511.3443}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the yellow block that is on your right side and is the second closest to you
 {'width': 756, 'height': 660, 'annotations': [{'id': 'b328ac50-08b5-486e-a937-2cddbc04245c', 'keypoint': {'x': 304.8396, 'y': 434.2925}, 'name': 'orange'}, {'id': '7344b92c-5683-4670-9cef-33f4cf8e73fb', 'keypoint': {'x': 472.9528, 'y': 456.0849}, 'name': 'orange'}, {'id': '6958bbb7-3dce-4cfe-9049-02ebc5a32e43', 'keypoint': {'x': 306.3962, 'y': 460.7547}, 'name': 'yellow'}, {'id': '62285f29-522c-498a-9fdb-53e2e8421ea5', 'keypoint': {'x': 330.5236, 'y': 470.8726}, 'name': 'yellow'}, {'id': 'e4402e53-f01e-4254-a874-f6ffd3892ac4', 'keypoint': {'x': 381.8915, 'y': 475.5425}, 'name': 'yellow'}, {'id': '3f3ed2fa-5312-418f-9f29-f4750a72aa9e', 'keypoint': {'x': 402.9057, 'y': 435.0708}, 'name': 'yellow'}, {'id': 'fe5ba8f4-1f70-4712-ae25-cd20ac9670d6', 'keypoint': {'x': 455.8302, 'y': 481.7689}, 'name': 'yellow'}, {'id': 'bd1c4ddb-8345-4f24-a982-f6cd6a64fb0e', 'keypoint': {'x': 501.75, 'y': 505.8962}, 'name': 'yellow'}, {'id': '8f332be0-ad10-4130-9840-f3a1dc330890', 'keypoint': {'x': 352.316, 'y': 513.6792}, 'name': 'yellow'}, {'id': '5b19adbf-7efb-4a90-9f40-4c0bf2f3b9a8', 'keypoint': {'x': 324.2972, 'y': 499.6698}, 'name': 'yellow'}, {'id': '7f124547-f8b7-4280-b7e1-401876ce2bdb', 'keypoint': {'x': 308.7311, 'y': 552.5943}, 'name': 'yellow'}, {'id': '8a1fd773-bc7c-4be5-a292-c1c53d5c77df', 'keypoint': {'x': 298.6132, 'y': 491.8868}, 'name': 'blue'}, {'id': '7be8714d-676d-466e-a70a-e9d078b757f4', 'keypoint': {'x': 381.1132, 'y': 497.3349}, 'name': 'blue'}, {'id': 'b1ee1721-f1d2-4cc8-828b-2e6c5072d54f', 'keypoint': {'x': 451.9387, 'y': 432.7358}, 'name': 'blue'}, {'id': '0416eebd-320e-40d8-bbc7-a94a6333aa95', 'keypoint': {'x': 497.0802, 'y': 485.6604}, 'name': 'green'}, {'id': '2240b1ab-d515-4872-a587-09ca836dc397', 'keypoint': {'x': 378.7783, 'y': 28.7972}, 'name': 'you'}]}
[/INST]
{'x': 330.5236, 'y': 470.8726}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:pick up the yellow block that is furthest from you.
 {'width': 756, 'height': 660, 'annotations': [{'id': '82770085-a20a-439c-b1a3-f74a8d8ccdd0', 'keypoint': {'x': 322.0293, 'y': 491.9856}, 'name': 'yellow'}, {'id': '975d2226-07a6-46ec-8aa1-01f44597bb5c', 'keypoint': {'x': 376.8281, 'y': 495.1624}, 'name': 'yellow'}, {'id': '9fbd0dd7-292a-46c1-854e-eb8c4bb880cf', 'keypoint': {'x': 402.2421, 'y': 551.5495}, 'name': 'yellow'}, {'id': 'c2d61b6b-0585-4aef-acaf-17bad430192b', 'keypoint': {'x': 464.9827, 'y': 498.3391}, 'name': 'yellow'}, {'id': '13d3414e-a81b-4dad-8963-6fbda2f0d436', 'keypoint': {'x': 376.0339, 'y': 27.387}, 'name': 'you'}, {'id': '9b80a1cd-fed1-49e9-8b69-3cd0088f7fca', 'keypoint': {'x': 302.9689, 'y': 454.6589}, 'name': 'blue'}, {'id': '17a9c4b0-bf5e-45d6-aa6f-94959ad755e2', 'keypoint': {'x': 421.3025, 'y': 473.7194}, 'name': 'blue'}, {'id': '4a1722fe-5d45-4bf9-a201-c0b8f7acc36b', 'keypoint': {'x': 456.2467, 'y': 437.981}, 'name': 'blue'}, {'id': '680351fd-2723-4f71-9103-17bb8a809738', 'keypoint': {'x': 375.2398, 'y': 452.2763}, 'name': 'orange'}, {'id': '188969f7-812f-47f7-a97e-5a0652f78c64', 'keypoint': {'x': 354.5909, 'y': 508.6635}, 'name': 'orange'}, {'id': 'b57e9350-23ed-40aa-96cc-2811f14cd796', 'keypoint': {'x': 396.6828, 'y': 520.5763}, 'name': 'orange'}, {'id': 'f041b6b6-0396-42fe-99f5-46de7b8e6b23', 'keypoint': {'x': 431.6269, 'y': 507.8693}, 'name': 'orange'}, {'id': 'ae76b953-f8b0-4121-9569-ce66605c9783', 'keypoint': {'x': 433.2153, 'y': 530.9007}, 'name': 'orange'}, {'id': '256e0a15-32e1-445f-a768-a8adb00d4905', 'keypoint': {'x': 341.0898, 'y': 438.7752}, 'name': 'green'}, {'id': '51ccd976-c149-458d-960f-bd00da3a5131', 'keypoint': {'x': 318.0584, 'y': 536.46}, 'name': 'green'}, {'id': 'd3e5315b-8e96-4659-97cb-07d656a65db8', 'keypoint': {'x': 503.8978, 'y': 456.2473}, 'name': 'green'}]}
[/INST]
{'x': 402.2421, 'y': 551.5495}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the yellow block in the last row.
 {'width': 756, 'height': 660, 'annotations': [{'id': '82770085-a20a-439c-b1a3-f74a8d8ccdd0', 'keypoint': {'x': 322.0293, 'y': 491.9856}, 'name': 'yellow'}, {'id': '975d2226-07a6-46ec-8aa1-01f44597bb5c', 'keypoint': {'x': 376.8281, 'y': 495.1624}, 'name': 'yellow'}, {'id': '9fbd0dd7-292a-46c1-854e-eb8c4bb880cf', 'keypoint': {'x': 402.2421, 'y': 551.5495}, 'name': 'yellow'}, {'id': 'c2d61b6b-0585-4aef-acaf-17bad430192b', 'keypoint': {'x': 464.9827, 'y': 498.3391}, 'name': 'yellow'}, {'id': '13d3414e-a81b-4dad-8963-6fbda2f0d436', 'keypoint': {'x': 376.0339, 'y': 27.387}, 'name': 'you'}, {'id': '9b80a1cd-fed1-49e9-8b69-3cd0088f7fca', 'keypoint': {'x': 302.9689, 'y': 454.6589}, 'name': 'blue'}, {'id': '17a9c4b0-bf5e-45d6-aa6f-94959ad755e2', 'keypoint': {'x': 421.3025, 'y': 473.7194}, 'name': 'blue'}, {'id': '4a1722fe-5d45-4bf9-a201-c0b8f7acc36b', 'keypoint': {'x': 456.2467, 'y': 437.981}, 'name': 'blue'}, {'id': '680351fd-2723-4f71-9103-17bb8a809738', 'keypoint': {'x': 375.2398, 'y': 452.2763}, 'name': 'orange'}, {'id': '188969f7-812f-47f7-a97e-5a0652f78c64', 'keypoint': {'x': 354.5909, 'y': 508.6635}, 'name': 'orange'}, {'id': 'b57e9350-23ed-40aa-96cc-2811f14cd796', 'keypoint': {'x': 396.6828, 'y': 520.5763}, 'name': 'orange'}, {'id': 'f041b6b6-0396-42fe-99f5-46de7b8e6b23', 'keypoint': {'x': 431.6269, 'y': 507.8693}, 'name': 'orange'}, {'id': 'ae76b953-f8b0-4121-9569-ce66605c9783', 'keypoint': {'x': 433.2153, 'y': 530.9007}, 'name': 'orange'}, {'id': '256e0a15-32e1-445f-a768-a8adb00d4905', 'keypoint': {'x': 341.0898, 'y': 438.7752}, 'name': 'green'}, {'id': '51ccd976-c149-458d-960f-bd00da3a5131', 'keypoint': {'x': 318.0584, 'y': 536.46}, 'name': 'green'}, {'id': 'd3e5315b-8e96-4659-97cb-07d656a65db8', 'keypoint': {'x': 503.8978, 'y': 456.2473}, 'name': 'green'}]}
[/INST]
{'x': 402.2421, 'y': 551.5495}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Raise your right hand, reach across the table directly in front of you and touch the first green block covered by the shadow of your hand.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'f8ce6e33-d9e3-4687-bd11-4537937b3def', 'keypoint': {'x': 313.4009, 'y': 480.9906}, 'name': 'orange'}, {'id': '8758b8e0-78a7-4dc2-b595-9eec97d96b3c', 'keypoint': {'x': 403.684, 'y': 466.9811}, 'name': 'orange'}, {'id': '9024f068-b46e-400e-bf69-507f61b74f80', 'keypoint': {'x': 459.7217, 'y': 452.9717}, 'name': 'orange'}, {'id': 'd7ab991d-e3d6-4ffb-8d9b-9cffbb59b9c6', 'keypoint': {'x': 502.5283, 'y': 457.6415}, 'name': 'orange'}, {'id': 'b9fb860f-d2d0-4b4a-a275-416d46a99ab8', 'keypoint': {'x': 335.9717, 'y': 450.6368}, 'name': 'yellow'}, {'id': '7404a736-3ae3-4f5c-b40d-5d5af4635c6e', 'keypoint': {'x': 316.5142, 'y': 531.5802}, 'name': 'yellow'}, {'id': 'd2d6b9ff-fd85-48a1-b368-3f0e1d09cc6f', 'keypoint': {'x': 396.6792, 'y': 491.8868}, 'name': 'yellow'}, {'id': 'c4551bcd-78f8-4205-b828-5b040d49a2c3', 'keypoint': {'x': 444.1557, 'y': 433.5142}, 'name': 'yellow'}, {'id': '3e0cda0a-1e8a-4472-8984-7886c02ff560', 'keypoint': {'x': 347.6462, 'y': 514.4575}, 'name': 'blue'}, {'id': '266b6be1-582b-4c4d-8544-906fad90cbd2', 'keypoint': {'x': 359.3208, 'y': 478.6557}, 'name': 'blue'}, {'id': 'c58796fa-46b4-4231-8589-0612cd52983f', 'keypoint': {'x': 395.9009, 'y': 444.4104}, 'name': 'blue'}, {'id': '3de58fe6-02ae-4e5a-b0eb-ac1e3ac60960', 'keypoint': {'x': 444.934, 'y': 537.8066}, 'name': 'blue'}, {'id': 'a1a96c53-7f31-4121-8983-bf6a69f42935', 'keypoint': {'x': 502.5283, 'y': 541.6981}, 'name': 'blue'}, {'id': 'c2e223f1-9da7-4029-ab29-81edb6be3c94', 'keypoint': {'x': 355.4292, 'y': 424.9528}, 'name': 'green'}, {'id': 'd1f0facf-050a-4a92-bfb1-1367c5fdd80d', 'keypoint': {'x': 466.7264, 'y': 500.4481}, 'name': 'green'}, {'id': '727b2df8-1454-47db-9eff-8cfe072974ff', 'keypoint': {'x': 383.4481, 'y': 44.3632}, 'name': 'you'}]}
[/INST]
{'x': 466.7264, 'y': 500.4481}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Please pick up the orange block that is located directly behind the yellow block positioned at your far right on the table.
 {'width': 756, 'height': 660, 'annotations': [{'id': '53de559f-b2c6-46fa-821d-b5a4e98c67c3', 'keypoint': {'x': 332.0802, 'y': 517.5708}, 'name': 'orange'}, {'id': 'fc5cb394-ebdb-4064-bb12-1d3da5b0cb6f', 'keypoint': {'x': 316.5142, 'y': 540.1415}, 'name': 'orange'}, {'id': 'b8f5d815-90be-4cd5-951f-44207215b594', 'keypoint': {'x': 373.3302, 'y': 537.8066}, 'name': 'orange'}, {'id': '78d88ded-7431-4260-a864-725e2dfef6ab', 'keypoint': {'x': 309.5094, 'y': 498.1132}, 'name': 'yellow'}, {'id': 'e5215008-d111-4bda-9009-f2c0795ffb0d', 'keypoint': {'x': 388.1179, 'y': 471.6509}, 'name': 'yellow'}, {'id': 'b722e1f1-0a3c-47ef-b35f-e9cd85898169', 'keypoint': {'x': 388.1179, 'y': 501.2264}, 'name': 'yellow'}, {'id': '27ad7266-481b-40f9-aaa0-e71dabd71d5e', 'keypoint': {'x': 413.8019, 'y': 491.1085}, 'name': 'yellow'}, {'id': '71348b8e-08dc-4a76-be0e-68cbee1a56a4', 'keypoint': {'x': 416.1368, 'y': 449.8585}, 'name': 'yellow'}, {'id': 'e46d6178-4284-433d-a81f-bc6cbbf7cdbb', 'keypoint': {'x': 490.8538, 'y': 505.1179}, 'name': 'yellow'}, {'id': '272a554a-aad9-4bba-9062-941465beeaa5', 'keypoint': {'x': 465.1698, 'y': 554.1509}, 'name': 'yellow'}, {'id': 'c6ba259b-1c8e-43d4-8bdb-404594f1c175', 'keypoint': {'x': 310.2877, 'y': 438.9623}, 'name': 'blue'}, {'id': '693a6454-810f-447d-8417-a2c57867901d', 'keypoint': {'x': 354.6509, 'y': 477.8774}, 'name': 'blue'}, {'id': '3ec7fa7f-c792-4c99-a423-b40bbb665d89', 'keypoint': {'x': 427.8113, 'y': 428.8443}, 'name': 'blue'}, {'id': '3e041f2e-9e78-4001-9fa0-3bebee80ac07', 'keypoint': {'x': 440.2642, 'y': 498.8915}, 'name': 'green'}, {'id': 'e2e59b33-f4e7-47cd-b137-77a874c834c7', 'keypoint': {'x': 450.3821, 'y': 519.1274}, 'name': 'green'}, {'id': '707959b0-76f6-463a-bc2f-62cd702815f4', 'keypoint': {'x': 377.2217, 'y': 55.2594}, 'name': 'you'}]}
[/INST]
{'x': 332.0802, 'y': 517.5708}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the orange block that is the furthest from you and right next to a yellow block.
 {'width': 756, 'height': 660, 'annotations': [{'id': '82770085-a20a-439c-b1a3-f74a8d8ccdd0', 'keypoint': {'x': 322.0293, 'y': 491.9856}, 'name': 'yellow'}, {'id': '975d2226-07a6-46ec-8aa1-01f44597bb5c', 'keypoint': {'x': 376.8281, 'y': 495.1624}, 'name': 'yellow'}, {'id': '9fbd0dd7-292a-46c1-854e-eb8c4bb880cf', 'keypoint': {'x': 402.2421, 'y': 551.5495}, 'name': 'yellow'}, {'id': 'c2d61b6b-0585-4aef-acaf-17bad430192b', 'keypoint': {'x': 464.9827, 'y': 498.3391}, 'name': 'yellow'}, {'id': '13d3414e-a81b-4dad-8963-6fbda2f0d436', 'keypoint': {'x': 376.0339, 'y': 27.387}, 'name': 'you'}, {'id': '9b80a1cd-fed1-49e9-8b69-3cd0088f7fca', 'keypoint': {'x': 302.9689, 'y': 454.6589}, 'name': 'blue'}, {'id': '17a9c4b0-bf5e-45d6-aa6f-94959ad755e2', 'keypoint': {'x': 421.3025, 'y': 473.7194}, 'name': 'blue'}, {'id': '4a1722fe-5d45-4bf9-a201-c0b8f7acc36b', 'keypoint': {'x': 456.2467, 'y': 437.981}, 'name': 'blue'}, {'id': '680351fd-2723-4f71-9103-17bb8a809738', 'keypoint': {'x': 375.2398, 'y': 452.2763}, 'name': 'orange'}, {'id': '188969f7-812f-47f7-a97e-5a0652f78c64', 'keypoint': {'x': 354.5909, 'y': 508.6635}, 'name': 'orange'}, {'id': 'b57e9350-23ed-40aa-96cc-2811f14cd796', 'keypoint': {'x': 396.6828, 'y': 520.5763}, 'name': 'orange'}, {'id': 'f041b6b6-0396-42fe-99f5-46de7b8e6b23', 'keypoint': {'x': 431.6269, 'y': 507.8693}, 'name': 'orange'}, {'id': 'ae76b953-f8b0-4121-9569-ce66605c9783', 'keypoint': {'x': 433.2153, 'y': 530.9007}, 'name': 'orange'}, {'id': '256e0a15-32e1-445f-a768-a8adb00d4905', 'keypoint': {'x': 341.0898, 'y': 438.7752}, 'name': 'green'}, {'id': '51ccd976-c149-458d-960f-bd00da3a5131', 'keypoint': {'x': 318.0584, 'y': 536.46}, 'name': 'green'}, {'id': 'd3e5315b-8e96-4659-97cb-07d656a65db8', 'keypoint': {'x': 503.8978, 'y': 456.2473}, 'name': 'green'}]}
[/INST]
{'x': 433.2153, 'y': 530.9007}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Ignore the three blocks on your left most side and the single one on the right. Pick up the purple block of the two in the middle that is closest to you.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'bb2508d6-bab0-4991-a4b4-1d60addd89be', 'keypoint': {'x': 374.8868, 'y': 32.6887}, 'name': 'you'}, {'id': '80bad823-3c11-42a6-9b7a-8f92fd053db8', 'keypoint': {'x': 286.1604, 'y': 504.3396}, 'name': 'yellow'}, {'id': '74407b61-1d02-4456-a3be-f44e6d0621e4', 'keypoint': {'x': 427.8113, 'y': 511.3443}, 'name': 'yellow'}, {'id': '0c043462-b620-40d3-b17e-3f17c7108248', 'keypoint': {'x': 391.2311, 'y': 435.8491}, 'name': 'green'}, {'id': 'e97e7600-f671-4774-970c-bcd8e011471e', 'keypoint': {'x': 311.066, 'y': 530.8019}, 'name': 'green'}, {'id': '3c45bdee-a533-4cab-8e50-cd3e05ea379a', 'keypoint': {'x': 375.6651, 'y': 518.3491}, 'name': 'green'}, {'id': 'fbe906a6-d83a-45bc-bd1e-7c81ccde8b9b', 'keypoint': {'x': 487.7406, 'y': 473.2075}, 'name': 'green'}, {'id': '662455ed-994d-45d8-b864-0f07ccfe9a1e', 'keypoint': {'x': 309.5094, 'y': 477.8774}, 'name': 'blue'}, {'id': '92e619ee-cc43-4ba6-80a0-fd23cabc2b68', 'keypoint': {'x': 378.7783, 'y': 457.6415}, 'name': 'blue'}, {'id': '4d3a4397-37ee-4ea1-bff7-ea4e40e4ba91', 'keypoint': {'x': 395.1226, 'y': 483.3255}, 'name': 'blue'}, {'id': '83c6c716-2497-4e0e-b233-3cda2dca8049', 'keypoint': {'x': 438.7075, 'y': 448.3019}, 'name': 'blue'}, {'id': '24b23d82-8281-40fd-9942-dc852b76fea4', 'keypoint': {'x': 444.1557, 'y': 484.1038}, 'name': 'blue'}, {'id': '541179c1-e37f-4069-99a8-fe58be3053c6', 'keypoint': {'x': 466.7264, 'y': 502.783}, 'name': 'blue'}, {'id': 'd950b6f4-0fff-425b-bf69-419e3a15863f', 'keypoint': {'x': 343.7547, 'y': 543.2547}, 'name': 'orange'}, {'id': 'a5aa3394-3b02-4613-8a7d-f14c9cd0fa58', 'keypoint': {'x': 431.7028, 'y': 539.3632}, 'name': 'orange'}, {'id': 'd7eb7db6-f69d-4df7-861e-0cc9dbebac2a', 'keypoint': {'x': 484.6274, 'y': 532.3585}, 'name': 'orange'}]}
[/INST]
{'x': 378.7783, 'y': 457.6415}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the green block that is closer to you.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'f8ce6e33-d9e3-4687-bd11-4537937b3def', 'keypoint': {'x': 313.4009, 'y': 480.9906}, 'name': 'orange'}, {'id': '8758b8e0-78a7-4dc2-b595-9eec97d96b3c', 'keypoint': {'x': 403.684, 'y': 466.9811}, 'name': 'orange'}, {'id': '9024f068-b46e-400e-bf69-507f61b74f80', 'keypoint': {'x': 459.7217, 'y': 452.9717}, 'name': 'orange'}, {'id': 'd7ab991d-e3d6-4ffb-8d9b-9cffbb59b9c6', 'keypoint': {'x': 502.5283, 'y': 457.6415}, 'name': 'orange'}, {'id': 'b9fb860f-d2d0-4b4a-a275-416d46a99ab8', 'keypoint': {'x': 335.9717, 'y': 450.6368}, 'name': 'yellow'}, {'id': '7404a736-3ae3-4f5c-b40d-5d5af4635c6e', 'keypoint': {'x': 316.5142, 'y': 531.5802}, 'name': 'yellow'}, {'id': 'd2d6b9ff-fd85-48a1-b368-3f0e1d09cc6f', 'keypoint': {'x': 396.6792, 'y': 491.8868}, 'name': 'yellow'}, {'id': 'c4551bcd-78f8-4205-b828-5b040d49a2c3', 'keypoint': {'x': 444.1557, 'y': 433.5142}, 'name': 'yellow'}, {'id': '3e0cda0a-1e8a-4472-8984-7886c02ff560', 'keypoint': {'x': 347.6462, 'y': 514.4575}, 'name': 'blue'}, {'id': '266b6be1-582b-4c4d-8544-906fad90cbd2', 'keypoint': {'x': 359.3208, 'y': 478.6557}, 'name': 'blue'}, {'id': 'c58796fa-46b4-4231-8589-0612cd52983f', 'keypoint': {'x': 395.9009, 'y': 444.4104}, 'name': 'blue'}, {'id': '3de58fe6-02ae-4e5a-b0eb-ac1e3ac60960', 'keypoint': {'x': 444.934, 'y': 537.8066}, 'name': 'blue'}, {'id': 'a1a96c53-7f31-4121-8983-bf6a69f42935', 'keypoint': {'x': 502.5283, 'y': 541.6981}, 'name': 'blue'}, {'id': 'c2e223f1-9da7-4029-ab29-81edb6be3c94', 'keypoint': {'x': 355.4292, 'y': 424.9528}, 'name': 'green'}, {'id': 'd1f0facf-050a-4a92-bfb1-1367c5fdd80d', 'keypoint': {'x': 466.7264, 'y': 500.4481}, 'name': 'green'}, {'id': '727b2df8-1454-47db-9eff-8cfe072974ff', 'keypoint': {'x': 383.4481, 'y': 44.3632}, 'name': 'you'}]}
[/INST]
{'x': 355.4292, 'y': 424.9528}</s>