<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:pick up the blue block on your right, closest to you.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'fe98e63d-c226-4a25-9e75-229c9a2d828a', 'keypoint': {'x': 377.2217, 'y': 28.7972}, 'name': 'you'}, {'id': 'c1bfc6fe-9e0d-4152-aab0-9efe03f99ee1', 'keypoint': {'x': 291.6085, 'y': 487.217}, 'name': 'yellow'}, {'id': 'eefd5787-08ef-4674-ae9e-c41f4a2ed951', 'keypoint': {'x': 314.1792, 'y': 539.3632}, 'name': 'yellow'}, {'id': '8987e25d-8eb0-42ab-aa8e-390a7b046e44', 'keypoint': {'x': 363.2123, 'y': 544.8113}, 'name': 'yellow'}, {'id': 'af54d764-7469-4b39-95f2-d25ac6241715', 'keypoint': {'x': 434.0377, 'y': 542.4764}, 'name': 'yellow'}, {'id': 'c426887c-43fa-4c7c-9ee6-447f48c5b353', 'keypoint': {'x': 463.6132, 'y': 487.9953}, 'name': 'yellow'}, {'id': 'f1402f90-fe95-42c0-93e5-75019494beba', 'keypoint': {'x': 497.0802, 'y': 442.0755}, 'name': 'yellow'}, {'id': 'ad7b2c3b-efac-4705-8dc4-076514da9f0d', 'keypoint': {'x': 319.6274, 'y': 463.8679}, 'name': 'blue'}, {'id': '57fd11a6-5352-4e90-8c5e-78e080a9f63b', 'keypoint': {'x': 458.1651, 'y': 516.0142}, 'name': 'blue'}, {'id': 'bffb87fb-647e-44e1-b4c0-9bbf464cc4da', 'keypoint': {'x': 353.8726, 'y': 445.1887}, 'name': 'green'}, {'id': '2be751d9-34f0-411b-89e7-1b20b054e1ee', 'keypoint': {'x': 388.8962, 'y': 498.1132}, 'name': 'green'}, {'id': '98a41430-e570-492e-969a-8ea45857d765', 'keypoint': {'x': 395.9009, 'y': 533.1368}, 'name': 'green'}, {'id': '0327cb90-c17c-4528-8964-e978fbff5999', 'keypoint': {'x': 427.8113, 'y': 484.8821}, 'name': 'green'}, {'id': '97f91dd8-6cd8-4194-8783-e9e3b5786e68', 'keypoint': {'x': 456.6085, 'y': 452.9717}, 'name': 'green'}, {'id': '4ebadfd4-3497-4f80-a4e9-a870c25786bd', 'keypoint': {'x': 504.8632, 'y': 502.0047}, 'name': 'green'}, {'id': 'd2ff782c-210c-4090-91a5-de43a78f1232', 'keypoint': {'x': 331.3019, 'y': 512.9009}, 'name': 'orange'}]}
[/INST]
{'x': 319.6274, 'y': 463.8679}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the yellow block on your right side, 3 blocks away from you to the left of the blue block.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'b328ac50-08b5-486e-a937-2cddbc04245c', 'keypoint': {'x': 304.8396, 'y': 434.2925}, 'name': 'orange'}, {'id': '7344b92c-5683-4670-9cef-33f4cf8e73fb', 'keypoint': {'x': 472.9528, 'y': 456.0849}, 'name': 'orange'}, {'id': '6958bbb7-3dce-4cfe-9049-02ebc5a32e43', 'keypoint': {'x': 306.3962, 'y': 460.7547}, 'name': 'yellow'}, {'id': '62285f29-522c-498a-9fdb-53e2e8421ea5', 'keypoint': {'x': 330.5236, 'y': 470.8726}, 'name': 'yellow'}, {'id': 'e4402e53-f01e-4254-a874-f6ffd3892ac4', 'keypoint': {'x': 381.8915, 'y': 475.5425}, 'name': 'yellow'}, {'id': '3f3ed2fa-5312-418f-9f29-f4750a72aa9e', 'keypoint': {'x': 402.9057, 'y': 435.0708}, 'name': 'yellow'}, {'id': 'fe5ba8f4-1f70-4712-ae25-cd20ac9670d6', 'keypoint': {'x': 455.8302, 'y': 481.7689}, 'name': 'yellow'}, {'id': 'bd1c4ddb-8345-4f24-a982-f6cd6a64fb0e', 'keypoint': {'x': 501.75, 'y': 505.8962}, 'name': 'yellow'}, {'id': '8f332be0-ad10-4130-9840-f3a1dc330890', 'keypoint': {'x': 352.316, 'y': 513.6792}, 'name': 'yellow'}, {'id': '5b19adbf-7efb-4a90-9f40-4c0bf2f3b9a8', 'keypoint': {'x': 324.2972, 'y': 499.6698}, 'name': 'yellow'}, {'id': '7f124547-f8b7-4280-b7e1-401876ce2bdb', 'keypoint': {'x': 308.7311, 'y': 552.5943}, 'name': 'yellow'}, {'id': '8a1fd773-bc7c-4be5-a292-c1c53d5c77df', 'keypoint': {'x': 298.6132, 'y': 491.8868}, 'name': 'blue'}, {'id': '7be8714d-676d-466e-a70a-e9d078b757f4', 'keypoint': {'x': 381.1132, 'y': 497.3349}, 'name': 'blue'}, {'id': 'b1ee1721-f1d2-4cc8-828b-2e6c5072d54f', 'keypoint': {'x': 451.9387, 'y': 432.7358}, 'name': 'blue'}, {'id': '0416eebd-320e-40d8-bbc7-a94a6333aa95', 'keypoint': {'x': 497.0802, 'y': 485.6604}, 'name': 'green'}, {'id': '2240b1ab-d515-4872-a587-09ca836dc397', 'keypoint': {'x': 378.7783, 'y': 28.7972}, 'name': 'you'}]}
[/INST]
{'x': 324.2972, 'y': 499.6698}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:pick up the blue cube furthest to your right
 {'width': 756, 'height': 660, 'annotations': [{'id': 'bb2508d6-bab0-4991-a4b4-1d60addd89be', 'keypoint': {'x': 374.8868, 'y': 32.6887}, 'name': 'you'}, {'id': '80bad823-3c11-42a6-9b7a-8f92fd053db8', 'keypoint': {'x': 286.1604, 'y': 504.3396}, 'name': 'yellow'}, {'id': '74407b61-1d02-4456-a3be-f44e6d0621e4', 'keypoint': {'x': 427.8113, 'y': 511.3443}, 'name': 'yellow'}, {'id': '0c043462-b620-40d3-b17e-3f17c7108248', 'keypoint': {'x': 391.2311, 'y': 435.8491}, 'name': 'green'}, {'id': 'e97e7600-f671-4774-970c-bcd8e011471e', 'keypoint': {'x': 311.066, 'y': 530.8019}, 'name': 'green'}, {'id': '3c45bdee-a533-4cab-8e50-cd3e05ea379a', 'keypoint': {'x': 375.6651, 'y': 518.3491}, 'name': 'green'}, {'id': 'fbe906a6-d83a-45bc-bd1e-7c81ccde8b9b', 'keypoint': {'x': 487.7406, 'y': 473.2075}, 'name': 'green'}, {'id': '662455ed-994d-45d8-b864-0f07ccfe9a1e', 'keypoint': {'x': 309.5094, 'y': 477.8774}, 'name': 'blue'}, {'id': '92e619ee-cc43-4ba6-80a0-fd23cabc2b68', 'keypoint': {'x': 378.7783, 'y': 457.6415}, 'name': 'blue'}, {'id': '4d3a4397-37ee-4ea1-bff7-ea4e40e4ba91', 'keypoint': {'x': 395.1226, 'y': 483.3255}, 'name': 'blue'}, {'id': '83c6c716-2497-4e0e-b233-3cda2dca8049', 'keypoint': {'x': 438.7075, 'y': 448.3019}, 'name': 'blue'}, {'id': '24b23d82-8281-40fd-9942-dc852b76fea4', 'keypoint': {'x': 444.1557, 'y': 484.1038}, 'name': 'blue'}, {'id': '541179c1-e37f-4069-99a8-fe58be3053c6', 'keypoint': {'x': 466.7264, 'y': 502.783}, 'name': 'blue'}, {'id': 'd950b6f4-0fff-425b-bf69-419e3a15863f', 'keypoint': {'x': 343.7547, 'y': 543.2547}, 'name': 'orange'}, {'id': 'a5aa3394-3b02-4613-8a7d-f14c9cd0fa58', 'keypoint': {'x': 431.7028, 'y': 539.3632}, 'name': 'orange'}, {'id': 'd7eb7db6-f69d-4df7-861e-0cc9dbebac2a', 'keypoint': {'x': 484.6274, 'y': 532.3585}, 'name': 'orange'}]}
[/INST]
{'x': 309.5094, 'y': 477.8774}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the blue block that is furthest away from you.  It is the blue block that is closest to the yellow block that is furthest away from you.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'c7519096-f7ea-4ab1-a66d-97c708976599', 'keypoint': {'x': 380.3349, 'y': 34.2453}, 'name': 'you'}, {'id': 'a95968ee-51d4-482d-ae79-97cb54663aed', 'keypoint': {'x': 293.9434, 'y': 479.434}, 'name': 'blue'}, {'id': '2fc8b299-6074-43eb-92bc-7ed758a958fc', 'keypoint': {'x': 329.7453, 'y': 557.2642}, 'name': 'blue'}, {'id': '11dbc045-de4f-4d3a-bc50-a1c2ba279bd1', 'keypoint': {'x': 366.3255, 'y': 546.3679}, 'name': 'blue'}, {'id': 'a3036737-fb7e-4f86-88c8-b0c5262cc051', 'keypoint': {'x': 379.5566, 'y': 438.9623}, 'name': 'blue'}, {'id': '5370f3f3-ef52-45e8-bdb4-3d96d54f4bd9', 'keypoint': {'x': 455.8302, 'y': 449.0802}, 'name': 'blue'}, {'id': '95294165-9354-41e7-b1be-60d4033bd1a3', 'keypoint': {'x': 318.0708, 'y': 454.5283}, 'name': 'yellow'}, {'id': '71f49ac9-1905-457f-9219-701cac7c42e4', 'keypoint': {'x': 318.0708, 'y': 530.8019}, 'name': 'yellow'}, {'id': '6f7a8a19-794f-4ffc-80ed-8a42dedeab0f', 'keypoint': {'x': 463.6132, 'y': 524.5755}, 'name': 'yellow'}, {'id': '407068c6-6a1e-4a14-b9d0-c0a4ed13b9c2', 'keypoint': {'x': 395.1226, 'y': 466.2028}, 'name': 'green'}, {'id': '847fb0f9-02cc-42df-8984-2b193e2abb8d', 'keypoint': {'x': 403.684, 'y': 523.0189}, 'name': 'green'}, {'id': 'bd78738b-2c39-4b85-9575-960d46ecd66c', 'keypoint': {'x': 325.0755, 'y': 500.4481}, 'name': 'orange'}, {'id': 'dafb3b80-4c1e-48a7-b398-e3aa56c5dcb0', 'keypoint': {'x': 374.1085, 'y': 515.2358}, 'name': 'orange'}, {'id': '36cf3f26-cac5-4368-91eb-ae8937ec2f91', 'keypoint': {'x': 412.2453, 'y': 427.2877}, 'name': 'orange'}, {'id': 'f93ff614-2250-4f4d-8ec0-ac87ec62305c', 'keypoint': {'x': 436.3726, 'y': 437.4057}, 'name': 'orange'}, {'id': '09066da9-88d0-4617-8cf6-7d604ee0190d', 'keypoint': {'x': 485.4057, 'y': 510.566}, 'name': 'orange'}]}
[/INST]
{'x': 329.7453, 'y': 557.2642}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Grab the green block furthest away from you.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'f8ce6e33-d9e3-4687-bd11-4537937b3def', 'keypoint': {'x': 313.4009, 'y': 480.9906}, 'name': 'orange'}, {'id': '8758b8e0-78a7-4dc2-b595-9eec97d96b3c', 'keypoint': {'x': 403.684, 'y': 466.9811}, 'name': 'orange'}, {'id': '9024f068-b46e-400e-bf69-507f61b74f80', 'keypoint': {'x': 459.7217, 'y': 452.9717}, 'name': 'orange'}, {'id': 'd7ab991d-e3d6-4ffb-8d9b-9cffbb59b9c6', 'keypoint': {'x': 502.5283, 'y': 457.6415}, 'name': 'orange'}, {'id': 'b9fb860f-d2d0-4b4a-a275-416d46a99ab8', 'keypoint': {'x': 335.9717, 'y': 450.6368}, 'name': 'yellow'}, {'id': '7404a736-3ae3-4f5c-b40d-5d5af4635c6e', 'keypoint': {'x': 316.5142, 'y': 531.5802}, 'name': 'yellow'}, {'id': 'd2d6b9ff-fd85-48a1-b368-3f0e1d09cc6f', 'keypoint': {'x': 396.6792, 'y': 491.8868}, 'name': 'yellow'}, {'id': 'c4551bcd-78f8-4205-b828-5b040d49a2c3', 'keypoint': {'x': 444.1557, 'y': 433.5142}, 'name': 'yellow'}, {'id': '3e0cda0a-1e8a-4472-8984-7886c02ff560', 'keypoint': {'x': 347.6462, 'y': 514.4575}, 'name': 'blue'}, {'id': '266b6be1-582b-4c4d-8544-906fad90cbd2', 'keypoint': {'x': 359.3208, 'y': 478.6557}, 'name': 'blue'}, {'id': 'c58796fa-46b4-4231-8589-0612cd52983f', 'keypoint': {'x': 395.9009, 'y': 444.4104}, 'name': 'blue'}, {'id': '3de58fe6-02ae-4e5a-b0eb-ac1e3ac60960', 'keypoint': {'x': 444.934, 'y': 537.8066}, 'name': 'blue'}, {'id': 'a1a96c53-7f31-4121-8983-bf6a69f42935', 'keypoint': {'x': 502.5283, 'y': 541.6981}, 'name': 'blue'}, {'id': 'c2e223f1-9da7-4029-ab29-81edb6be3c94', 'keypoint': {'x': 355.4292, 'y': 424.9528}, 'name': 'green'}, {'id': 'd1f0facf-050a-4a92-bfb1-1367c5fdd80d', 'keypoint': {'x': 466.7264, 'y': 500.4481}, 'name': 'green'}, {'id': '727b2df8-1454-47db-9eff-8cfe072974ff', 'keypoint': {'x': 383.4481, 'y': 44.3632}, 'name': 'you'}]}
[/INST]
{'x': 466.7264, 'y': 500.4481}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the yellow block.
 {'width': 756, 'height': 660, 'annotations': [{'id': '4665780d-e410-4494-8512-3557afddb1a4', 'keypoint': {'x': 311.8443, 'y': 504.3396}, 'name': None}, {'id': '565e6ab8-2ba1-4136-96ed-e63f3cdba23d', 'keypoint': {'x': 368.6604, 'y': 505.1179}, 'name': 'orange'}, {'id': '487a86d1-7908-404f-99e9-16ab87ea0d04', 'keypoint': {'x': 359.3208, 'y': 560.3774}, 'name': 'orange'}, {'id': 'ab7b691c-c47b-4c26-8083-096140fac645', 'keypoint': {'x': 413.0236, 'y': 447.5236}, 'name': 'orange'}, {'id': 'fdb6ba1e-6ab3-4d82-862c-1bdaf73940dc', 'keypoint': {'x': 464.3915, 'y': 459.9764}, 'name': 'orange'}, {'id': '7768e90f-d1af-4f66-abc0-01fa10bc5e61', 'keypoint': {'x': 447.2689, 'y': 530.8019}, 'name': 'orange'}, {'id': '698efbba-5197-4d9f-96eb-fdc8c950c48f', 'keypoint': {'x': 290.8302, 'y': 554.1509}, 'name': 'yellow'}, {'id': '1e9d76c9-a045-4417-94ae-129e3daeb548', 'keypoint': {'x': 407.5755, 'y': 519.9057}, 'name': 'blue'}, {'id': 'dc0ba9e0-8eb1-478a-88c5-3eb00d02992d', 'keypoint': {'x': 454.2736, 'y': 505.8962}, 'name': 'blue'}, {'id': '13165129-2db5-428d-ba05-982c1ae9a179', 'keypoint': {'x': 503.3066, 'y': 487.217}, 'name': 'green'}, {'id': 'a2feb4b6-353e-4876-b8e6-d5351bc5f056', 'keypoint': {'x': 459.7217, 'y': 490.3302}, 'name': 'green'}, {'id': 'bd636fd9-8ba5-408a-a286-413978cb55e7', 'keypoint': {'x': 362.434, 'y': 476.3208}, 'name': 'green'}, {'id': '4d316993-cf9a-4115-894b-940d569fafc4', 'keypoint': {'x': 311.066, 'y': 439.7406}, 'name': 'green'}, {'id': 'b6f5df26-1415-40e4-bbf7-01af31da49b9', 'keypoint': {'x': 380.3349, 'y': 544.8113}, 'name': 'green'}, {'id': 'f34f5fb5-74bd-4265-8123-8eed80fffdd5', 'keypoint': {'x': 322.7406, 'y': 536.25}, 'name': 'green'}, {'id': '847a9a8d-0e7a-49f2-b274-f833cdac716e', 'keypoint': {'x': 375.6651, 'y': 49.8113}, 'name': 'you'}]}
[/INST]
{'x': 290.8302, 'y': 554.1509}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the green block that is the farthest to the left.
 {'width': 756, 'height': 660, 'annotations': [{'id': '8f941e4d-8e6b-4e37-a2cd-f19b3dbb1dda', 'keypoint': {'x': 293.1651, 'y': 452.9717}, 'name': 'orange'}, {'id': '77f8b560-0c54-4923-9c65-8e8973d5cc1a', 'keypoint': {'x': 293.9434, 'y': 485.6604}, 'name': 'orange'}, {'id': 'c2fdf254-12fa-43ee-9d04-fd097d6a2fad', 'keypoint': {'x': 332.8585, 'y': 517.5708}, 'name': 'orange'}, {'id': 'ee022624-310e-4867-8360-cd61e6a5e8ec', 'keypoint': {'x': 325.0755, 'y': 557.2642}, 'name': 'orange'}, {'id': 'ca78545f-1817-447e-a767-1ec186e40025', 'keypoint': {'x': 377.2217, 'y': 478.6557}, 'name': 'orange'}, {'id': 'b7ea891b-3095-4df8-a718-b193f34c851c', 'keypoint': {'x': 386.5613, 'y': 492.6651}, 'name': 'orange'}, {'id': '54b85951-9b7d-4a22-9b94-5856c8f4cc7e', 'keypoint': {'x': 441.8208, 'y': 499.6698}, 'name': 'orange'}, {'id': 'ef2b7cbb-fa7e-4894-8e50-310d43a6f3eb', 'keypoint': {'x': 385.0047, 'y': 448.3019}, 'name': 'yellow'}, {'id': 'ab7d8b08-4eaf-41cf-8066-9aba376bb546', 'keypoint': {'x': 392.0094, 'y': 523.0189}, 'name': 'yellow'}, {'id': 'aec059f3-2def-4458-9ea4-3fa5916cab5b', 'keypoint': {'x': 466.7264, 'y': 442.8538}, 'name': 'yellow'}, {'id': '39fe094d-1e93-4828-ae7e-d672aeb88451', 'keypoint': {'x': 356.2075, 'y': 533.9151}, 'name': 'blue'}, {'id': '57cde523-a685-45a5-b623-1b11057840da', 'keypoint': {'x': 487.7406, 'y': 488.7736}, 'name': 'blue'}, {'id': 'da91e8aa-4087-4689-95ed-80e4ba9a93ec', 'keypoint': {'x': 327.4104, 'y': 472.4292}, 'name': 'green'}, {'id': '9406ceb5-4de6-4915-8da6-132b32721d7c', 'keypoint': {'x': 508.7547, 'y': 511.3443}, 'name': 'green'}, {'id': 'fd93334e-be71-4598-9c11-abb001661b87', 'keypoint': {'x': 497.0802, 'y': 560.3774}, 'name': 'green'}, {'id': '3067ae97-d698-4a21-82f6-e94a33ed51a0', 'keypoint': {'x': 380.3349, 'y': 59.1509}, 'name': 'you'}]}
[/INST]
{'x': 508.7547, 'y': 511.3443}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the blue block that is together with the other blue block and is towards the inside.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'a7c6f9ab-55b7-422b-8e7f-70902f2e94d6', 'keypoint': {'x': 320.4057, 'y': 488.7736}, 'name': 'orange'}, {'id': 'acc187bd-2783-4707-be6a-314a58518228', 'keypoint': {'x': 423.9198, 'y': 451.4151}, 'name': 'orange'}, {'id': '7b9652c2-a499-48cc-ab07-68e686e885ac', 'keypoint': {'x': 358.5425, 'y': 462.3113}, 'name': 'yellow'}, {'id': '6e4e6633-28b0-4e37-a08b-d90cdb596a0d', 'keypoint': {'x': 334.4151, 'y': 548.7028}, 'name': 'yellow'}, {'id': 'dc28f152-a36a-458c-9f8a-21f4093b66bd', 'keypoint': {'x': 418.4717, 'y': 547.9245}, 'name': 'yellow'}, {'id': '34029874-d3f4-4998-a6fa-85ee314b35b5', 'keypoint': {'x': 479.1792, 'y': 484.8821}, 'name': 'yellow'}, {'id': '3d4114df-2e6e-4d36-8e12-47bbffbd6150', 'keypoint': {'x': 314.1792, 'y': 452.1934}, 'name': 'blue'}, {'id': '021cc2a3-2e83-4079-9777-64d3bc57e829', 'keypoint': {'x': 293.1651, 'y': 557.2642}, 'name': 'blue'}, {'id': '49911b5f-df0a-4694-9ff5-d8165f0f587a', 'keypoint': {'x': 390.4528, 'y': 513.6792}, 'name': 'blue'}, {'id': '95b20075-c6a2-46e0-b203-f8004a3ee4a3', 'keypoint': {'x': 440.2642, 'y': 484.8821}, 'name': 'blue'}, {'id': '031e7d06-9b93-4602-a1ef-c48bce3518c4', 'keypoint': {'x': 462.0566, 'y': 469.316}, 'name': 'blue'}, {'id': '3f2fdd03-f2f9-4892-999b-713f9e988be6', 'keypoint': {'x': 493.1887, 'y': 465.4245}, 'name': 'blue'}, {'id': '7ed0f47f-3bdf-429a-ab98-1d1a9d3058fa', 'keypoint': {'x': 482.2925, 'y': 507.4528}, 'name': 'green'}, {'id': '1fb5251e-aa36-4074-87f5-edef1e5a25c7', 'keypoint': {'x': 491.6321, 'y': 542.4764}, 'name': 'green'}, {'id': 'e7cf6134-82a1-4efe-99a5-a098dcdb3a3e', 'keypoint': {'x': 334.4151, 'y': 516.0142}, 'name': 'green'}, {'id': '0aeb291a-183c-47ce-9552-8b4d4860c05f', 'keypoint': {'x': 376.4434, 'y': 44.3632}, 'name': 'you'}]}
[/INST]
{'x': 440.2642, 'y': 484.8821}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:pick up the blue block that's furthest to your left.
 {'width': 756, 'height': 660, 'annotations': [{'id': '8f941e4d-8e6b-4e37-a2cd-f19b3dbb1dda', 'keypoint': {'x': 293.1651, 'y': 452.9717}, 'name': 'orange'}, {'id': '77f8b560-0c54-4923-9c65-8e8973d5cc1a', 'keypoint': {'x': 293.9434, 'y': 485.6604}, 'name': 'orange'}, {'id': 'c2fdf254-12fa-43ee-9d04-fd097d6a2fad', 'keypoint': {'x': 332.8585, 'y': 517.5708}, 'name': 'orange'}, {'id': 'ee022624-310e-4867-8360-cd61e6a5e8ec', 'keypoint': {'x': 325.0755, 'y': 557.2642}, 'name': 'orange'}, {'id': 'ca78545f-1817-447e-a767-1ec186e40025', 'keypoint': {'x': 377.2217, 'y': 478.6557}, 'name': 'orange'}, {'id': 'b7ea891b-3095-4df8-a718-b193f34c851c', 'keypoint': {'x': 386.5613, 'y': 492.6651}, 'name': 'orange'}, {'id': '54b85951-9b7d-4a22-9b94-5856c8f4cc7e', 'keypoint': {'x': 441.8208, 'y': 499.6698}, 'name': 'orange'}, {'id': 'ef2b7cbb-fa7e-4894-8e50-310d43a6f3eb', 'keypoint': {'x': 385.0047, 'y': 448.3019}, 'name': 'yellow'}, {'id': 'ab7d8b08-4eaf-41cf-8066-9aba376bb546', 'keypoint': {'x': 392.0094, 'y': 523.0189}, 'name': 'yellow'}, {'id': 'aec059f3-2def-4458-9ea4-3fa5916cab5b', 'keypoint': {'x': 466.7264, 'y': 442.8538}, 'name': 'yellow'}, {'id': '39fe094d-1e93-4828-ae7e-d672aeb88451', 'keypoint': {'x': 356.2075, 'y': 533.9151}, 'name': 'blue'}, {'id': '57cde523-a685-45a5-b623-1b11057840da', 'keypoint': {'x': 487.7406, 'y': 488.7736}, 'name': 'blue'}, {'id': 'da91e8aa-4087-4689-95ed-80e4ba9a93ec', 'keypoint': {'x': 327.4104, 'y': 472.4292}, 'name': 'green'}, {'id': '9406ceb5-4de6-4915-8da6-132b32721d7c', 'keypoint': {'x': 508.7547, 'y': 511.3443}, 'name': 'green'}, {'id': 'fd93334e-be71-4598-9c11-abb001661b87', 'keypoint': {'x': 497.0802, 'y': 560.3774}, 'name': 'green'}, {'id': '3067ae97-d698-4a21-82f6-e94a33ed51a0', 'keypoint': {'x': 380.3349, 'y': 59.1509}, 'name': 'you'}]}
[/INST]
{'x': 487.7406, 'y': 488.7736}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Narrow search to only green blocks. Find the green block that is closest to the left edge of the table. Pick it up.
 {'width': 756, 'height': 660, 'annotations': [{'id': '8f941e4d-8e6b-4e37-a2cd-f19b3dbb1dda', 'keypoint': {'x': 293.1651, 'y': 452.9717}, 'name': 'orange'}, {'id': '77f8b560-0c54-4923-9c65-8e8973d5cc1a', 'keypoint': {'x': 293.9434, 'y': 485.6604}, 'name': 'orange'}, {'id': 'c2fdf254-12fa-43ee-9d04-fd097d6a2fad', 'keypoint': {'x': 332.8585, 'y': 517.5708}, 'name': 'orange'}, {'id': 'ee022624-310e-4867-8360-cd61e6a5e8ec', 'keypoint': {'x': 325.0755, 'y': 557.2642}, 'name': 'orange'}, {'id': 'ca78545f-1817-447e-a767-1ec186e40025', 'keypoint': {'x': 377.2217, 'y': 478.6557}, 'name': 'orange'}, {'id': 'b7ea891b-3095-4df8-a718-b193f34c851c', 'keypoint': {'x': 386.5613, 'y': 492.6651}, 'name': 'orange'}, {'id': '54b85951-9b7d-4a22-9b94-5856c8f4cc7e', 'keypoint': {'x': 441.8208, 'y': 499.6698}, 'name': 'orange'}, {'id': 'ef2b7cbb-fa7e-4894-8e50-310d43a6f3eb', 'keypoint': {'x': 385.0047, 'y': 448.3019}, 'name': 'yellow'}, {'id': 'ab7d8b08-4eaf-41cf-8066-9aba376bb546', 'keypoint': {'x': 392.0094, 'y': 523.0189}, 'name': 'yellow'}, {'id': 'aec059f3-2def-4458-9ea4-3fa5916cab5b', 'keypoint': {'x': 466.7264, 'y': 442.8538}, 'name': 'yellow'}, {'id': '39fe094d-1e93-4828-ae7e-d672aeb88451', 'keypoint': {'x': 356.2075, 'y': 533.9151}, 'name': 'blue'}, {'id': '57cde523-a685-45a5-b623-1b11057840da', 'keypoint': {'x': 487.7406, 'y': 488.7736}, 'name': 'blue'}, {'id': 'da91e8aa-4087-4689-95ed-80e4ba9a93ec', 'keypoint': {'x': 327.4104, 'y': 472.4292}, 'name': 'green'}, {'id': '9406ceb5-4de6-4915-8da6-132b32721d7c', 'keypoint': {'x': 508.7547, 'y': 511.3443}, 'name': 'green'}, {'id': 'fd93334e-be71-4598-9c11-abb001661b87', 'keypoint': {'x': 497.0802, 'y': 560.3774}, 'name': 'green'}, {'id': '3067ae97-d698-4a21-82f6-e94a33ed51a0', 'keypoint': {'x': 380.3349, 'y': 59.1509}, 'name': 'you'}]}
[/INST]
{'x': 508.7547, 'y': 511.3443}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the furthest orange block that is to the left of you.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'b1c926ae-92c1-43eb-96c5-76c41d0fab06', 'keypoint': {'x': 342.1981, 'y': 437.4057}, 'name': 'orange'}, {'id': '76267614-95a1-4ded-b4e9-354b821f62bb', 'keypoint': {'x': 314.1792, 'y': 499.6698}, 'name': 'orange'}, {'id': '3124e476-176a-4feb-8844-b3973d48973a', 'keypoint': {'x': 453.4953, 'y': 463.8679}, 'name': 'orange'}, {'id': '728bd5ee-ece2-45c9-b0d7-b887692bb453', 'keypoint': {'x': 494.7453, 'y': 534.6934}, 'name': 'orange'}, {'id': 'da0db0c2-fdee-4550-8e2a-11f1aace764f', 'keypoint': {'x': 376.4434, 'y': 419.5047}, 'name': 'yellow'}, {'id': 'bd27285d-35e5-47b5-9b1e-78bd5fb9e595', 'keypoint': {'x': 360.0991, 'y': 516.7925}, 'name': 'yellow'}, {'id': '5a97f3d8-463a-4c65-9906-b8cf5f772e2b', 'keypoint': {'x': 408.3538, 'y': 526.9104}, 'name': 'yellow'}, {'id': 'f2444027-7c76-40d2-9424-36c46342a49f', 'keypoint': {'x': 318.8491, 'y': 449.0802}, 'name': 'blue'}, {'id': '0b2e3fca-8a71-43d2-a83e-f02af09891b9', 'keypoint': {'x': 315.7358, 'y': 480.2123}, 'name': 'blue'}, {'id': '897e21a9-0e62-42b6-8c01-826ad479026c', 'keypoint': {'x': 378.0, 'y': 490.3302}, 'name': 'blue'}, {'id': 'ca6c0f72-0600-47ba-b4b1-05dc3566a83e', 'keypoint': {'x': 402.1274, 'y': 463.0896}, 'name': 'blue'}, {'id': '41397d83-cab1-4c4d-ba20-b13a1c93f043', 'keypoint': {'x': 419.25, 'y': 432.7358}, 'name': 'blue'}, {'id': '051bcea0-383b-4791-9652-d84f6ef3fe4c', 'keypoint': {'x': 348.4245, 'y': 488.7736}, 'name': 'green'}, {'id': '58f619eb-2384-4a80-97f9-10520e5226a9', 'keypoint': {'x': 311.8443, 'y': 539.3632}, 'name': 'green'}, {'id': 'b9c15fa4-b07d-4f16-9f7d-f007698c83c2', 'keypoint': {'x': 417.6934, 'y': 502.0047}, 'name': 'green'}, {'id': '5c0026dd-e833-4ad7-8339-361bad3686e0', 'keypoint': {'x': 378.0, 'y': 59.1509}, 'name': 'you'}]}
[/INST]
{'x': 494.7453, 'y': 534.6934}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the green block that is in the middle of the other green blocks.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'd5bcc2b7-ff91-4686-8f50-3a73ffd3beeb', 'keypoint': {'x': 315.7358, 'y': 436.6274}, 'name': 'orange'}, {'id': '70515b86-98d5-4fa6-9118-a0bc484f6de1', 'keypoint': {'x': 313.4009, 'y': 525.3538}, 'name': 'orange'}, {'id': 'b40b0958-f706-45d2-a042-0440eb67121b', 'keypoint': {'x': 380.3349, 'y': 515.2358}, 'name': 'orange'}, {'id': '9dcd051f-bad9-4f84-b653-9a30a72cb545', 'keypoint': {'x': 389.6745, 'y': 552.5943}, 'name': 'orange'}, {'id': 'b0de6329-f4d9-486c-91ed-478f28262cff', 'keypoint': {'x': 363.9906, 'y': 439.7406}, 'name': 'yellow'}, {'id': '10eeedc5-3253-42e7-b401-33c4616e4623', 'keypoint': {'x': 396.6792, 'y': 457.6415}, 'name': 'yellow'}, {'id': 'f7642eee-44ca-4f1d-abf7-a79d43dfbd0d', 'keypoint': {'x': 279.934, 'y': 535.4717}, 'name': 'yellow'}, {'id': 'd87c603d-89da-4c0b-8f61-a496275c1297', 'keypoint': {'x': 311.066, 'y': 487.217}, 'name': 'blue'}, {'id': '2065d0e4-12eb-4ab3-bde4-75688bb3c142', 'keypoint': {'x': 457.3868, 'y': 439.7406}, 'name': 'green'}, {'id': '1550df58-31c4-49f4-8e6d-7db4a57bc294', 'keypoint': {'x': 435.5943, 'y': 445.967}, 'name': 'green'}, {'id': '1990bf50-38ac-43ab-8581-26778cea80ed', 'keypoint': {'x': 458.9434, 'y': 484.8821}, 'name': 'green'}, {'id': '7895d85c-2e57-4008-b9d7-74625608a42c', 'keypoint': {'x': 485.4057, 'y': 515.2358}, 'name': 'green'}, {'id': '991a571a-03aa-4a00-b80d-3d6a0845625b', 'keypoint': {'x': 440.2642, 'y': 514.4575}, 'name': 'green'}, {'id': 'e0e1086c-3a47-42f6-98fd-868d0756cd4d', 'keypoint': {'x': 378.7783, 'y': 495.7783}, 'name': 'green'}, {'id': '3d132917-dd96-4ae9-9302-da261696357f', 'keypoint': {'x': 333.6368, 'y': 505.1179}, 'name': 'green'}, {'id': 'bc7494f1-b049-49c6-bf24-85e942969065', 'keypoint': {'x': 383.4481, 'y': 56.0377}, 'name': 'you'}]}
[/INST]
{'x': 458.9434, 'y': 484.8821}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the green block closest to you.
 {'width': 756, 'height': 660, 'annotations': [{'id': '4665780d-e410-4494-8512-3557afddb1a4', 'keypoint': {'x': 311.8443, 'y': 504.3396}, 'name': None}, {'id': '565e6ab8-2ba1-4136-96ed-e63f3cdba23d', 'keypoint': {'x': 368.6604, 'y': 505.1179}, 'name': 'orange'}, {'id': '487a86d1-7908-404f-99e9-16ab87ea0d04', 'keypoint': {'x': 359.3208, 'y': 560.3774}, 'name': 'orange'}, {'id': 'ab7b691c-c47b-4c26-8083-096140fac645', 'keypoint': {'x': 413.0236, 'y': 447.5236}, 'name': 'orange'}, {'id': 'fdb6ba1e-6ab3-4d82-862c-1bdaf73940dc', 'keypoint': {'x': 464.3915, 'y': 459.9764}, 'name': 'orange'}, {'id': '7768e90f-d1af-4f66-abc0-01fa10bc5e61', 'keypoint': {'x': 447.2689, 'y': 530.8019}, 'name': 'orange'}, {'id': '698efbba-5197-4d9f-96eb-fdc8c950c48f', 'keypoint': {'x': 290.8302, 'y': 554.1509}, 'name': 'yellow'}, {'id': '1e9d76c9-a045-4417-94ae-129e3daeb548', 'keypoint': {'x': 407.5755, 'y': 519.9057}, 'name': 'blue'}, {'id': 'dc0ba9e0-8eb1-478a-88c5-3eb00d02992d', 'keypoint': {'x': 454.2736, 'y': 505.8962}, 'name': 'blue'}, {'id': '13165129-2db5-428d-ba05-982c1ae9a179', 'keypoint': {'x': 503.3066, 'y': 487.217}, 'name': 'green'}, {'id': 'a2feb4b6-353e-4876-b8e6-d5351bc5f056', 'keypoint': {'x': 459.7217, 'y': 490.3302}, 'name': 'green'}, {'id': 'bd636fd9-8ba5-408a-a286-413978cb55e7', 'keypoint': {'x': 362.434, 'y': 476.3208}, 'name': 'green'}, {'id': '4d316993-cf9a-4115-894b-940d569fafc4', 'keypoint': {'x': 311.066, 'y': 439.7406}, 'name': 'green'}, {'id': 'b6f5df26-1415-40e4-bbf7-01af31da49b9', 'keypoint': {'x': 380.3349, 'y': 544.8113}, 'name': 'green'}, {'id': 'f34f5fb5-74bd-4265-8123-8eed80fffdd5', 'keypoint': {'x': 322.7406, 'y': 536.25}, 'name': 'green'}, {'id': '847a9a8d-0e7a-49f2-b274-f833cdac716e', 'keypoint': {'x': 375.6651, 'y': 49.8113}, 'name': 'you'}]}
[/INST]
{'x': 311.066, 'y': 439.7406}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the blue block directly in front of me.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'c7519096-f7ea-4ab1-a66d-97c708976599', 'keypoint': {'x': 380.3349, 'y': 34.2453}, 'name': 'you'}, {'id': 'a95968ee-51d4-482d-ae79-97cb54663aed', 'keypoint': {'x': 293.9434, 'y': 479.434}, 'name': 'blue'}, {'id': '2fc8b299-6074-43eb-92bc-7ed758a958fc', 'keypoint': {'x': 329.7453, 'y': 557.2642}, 'name': 'blue'}, {'id': '11dbc045-de4f-4d3a-bc50-a1c2ba279bd1', 'keypoint': {'x': 366.3255, 'y': 546.3679}, 'name': 'blue'}, {'id': 'a3036737-fb7e-4f86-88c8-b0c5262cc051', 'keypoint': {'x': 379.5566, 'y': 438.9623}, 'name': 'blue'}, {'id': '5370f3f3-ef52-45e8-bdb4-3d96d54f4bd9', 'keypoint': {'x': 455.8302, 'y': 449.0802}, 'name': 'blue'}, {'id': '95294165-9354-41e7-b1be-60d4033bd1a3', 'keypoint': {'x': 318.0708, 'y': 454.5283}, 'name': 'yellow'}, {'id': '71f49ac9-1905-457f-9219-701cac7c42e4', 'keypoint': {'x': 318.0708, 'y': 530.8019}, 'name': 'yellow'}, {'id': '6f7a8a19-794f-4ffc-80ed-8a42dedeab0f', 'keypoint': {'x': 463.6132, 'y': 524.5755}, 'name': 'yellow'}, {'id': '407068c6-6a1e-4a14-b9d0-c0a4ed13b9c2', 'keypoint': {'x': 395.1226, 'y': 466.2028}, 'name': 'green'}, {'id': '847fb0f9-02cc-42df-8984-2b193e2abb8d', 'keypoint': {'x': 403.684, 'y': 523.0189}, 'name': 'green'}, {'id': 'bd78738b-2c39-4b85-9575-960d46ecd66c', 'keypoint': {'x': 325.0755, 'y': 500.4481}, 'name': 'orange'}, {'id': 'dafb3b80-4c1e-48a7-b398-e3aa56c5dcb0', 'keypoint': {'x': 374.1085, 'y': 515.2358}, 'name': 'orange'}, {'id': '36cf3f26-cac5-4368-91eb-ae8937ec2f91', 'keypoint': {'x': 412.2453, 'y': 427.2877}, 'name': 'orange'}, {'id': 'f93ff614-2250-4f4d-8ec0-ac87ec62305c', 'keypoint': {'x': 436.3726, 'y': 437.4057}, 'name': 'orange'}, {'id': '09066da9-88d0-4617-8cf6-7d604ee0190d', 'keypoint': {'x': 485.4057, 'y': 510.566}, 'name': 'orange'}]}
[/INST]
{'x': 329.7453, 'y': 557.2642}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the orange block which is closest to you, between the blue block and yellow block.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'b1c926ae-92c1-43eb-96c5-76c41d0fab06', 'keypoint': {'x': 342.1981, 'y': 437.4057}, 'name': 'orange'}, {'id': '76267614-95a1-4ded-b4e9-354b821f62bb', 'keypoint': {'x': 314.1792, 'y': 499.6698}, 'name': 'orange'}, {'id': '3124e476-176a-4feb-8844-b3973d48973a', 'keypoint': {'x': 453.4953, 'y': 463.8679}, 'name': 'orange'}, {'id': '728bd5ee-ece2-45c9-b0d7-b887692bb453', 'keypoint': {'x': 494.7453, 'y': 534.6934}, 'name': 'orange'}, {'id': 'da0db0c2-fdee-4550-8e2a-11f1aace764f', 'keypoint': {'x': 376.4434, 'y': 419.5047}, 'name': 'yellow'}, {'id': 'bd27285d-35e5-47b5-9b1e-78bd5fb9e595', 'keypoint': {'x': 360.0991, 'y': 516.7925}, 'name': 'yellow'}, {'id': '5a97f3d8-463a-4c65-9906-b8cf5f772e2b', 'keypoint': {'x': 408.3538, 'y': 526.9104}, 'name': 'yellow'}, {'id': 'f2444027-7c76-40d2-9424-36c46342a49f', 'keypoint': {'x': 318.8491, 'y': 449.0802}, 'name': 'blue'}, {'id': '0b2e3fca-8a71-43d2-a83e-f02af09891b9', 'keypoint': {'x': 315.7358, 'y': 480.2123}, 'name': 'blue'}, {'id': '897e21a9-0e62-42b6-8c01-826ad479026c', 'keypoint': {'x': 378.0, 'y': 490.3302}, 'name': 'blue'}, {'id': 'ca6c0f72-0600-47ba-b4b1-05dc3566a83e', 'keypoint': {'x': 402.1274, 'y': 463.0896}, 'name': 'blue'}, {'id': '41397d83-cab1-4c4d-ba20-b13a1c93f043', 'keypoint': {'x': 419.25, 'y': 432.7358}, 'name': 'blue'}, {'id': '051bcea0-383b-4791-9652-d84f6ef3fe4c', 'keypoint': {'x': 348.4245, 'y': 488.7736}, 'name': 'green'}, {'id': '58f619eb-2384-4a80-97f9-10520e5226a9', 'keypoint': {'x': 311.8443, 'y': 539.3632}, 'name': 'green'}, {'id': 'b9c15fa4-b07d-4f16-9f7d-f007698c83c2', 'keypoint': {'x': 417.6934, 'y': 502.0047}, 'name': 'green'}, {'id': '5c0026dd-e833-4ad7-8339-361bad3686e0', 'keypoint': {'x': 378.0, 'y': 59.1509}, 'name': 'you'}]}
[/INST]
{'x': 342.1981, 'y': 437.4057}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:pick up the orange block at the top left corner.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'b1c926ae-92c1-43eb-96c5-76c41d0fab06', 'keypoint': {'x': 342.1981, 'y': 437.4057}, 'name': 'orange'}, {'id': '76267614-95a1-4ded-b4e9-354b821f62bb', 'keypoint': {'x': 314.1792, 'y': 499.6698}, 'name': 'orange'}, {'id': '3124e476-176a-4feb-8844-b3973d48973a', 'keypoint': {'x': 453.4953, 'y': 463.8679}, 'name': 'orange'}, {'id': '728bd5ee-ece2-45c9-b0d7-b887692bb453', 'keypoint': {'x': 494.7453, 'y': 534.6934}, 'name': 'orange'}, {'id': 'da0db0c2-fdee-4550-8e2a-11f1aace764f', 'keypoint': {'x': 376.4434, 'y': 419.5047}, 'name': 'yellow'}, {'id': 'bd27285d-35e5-47b5-9b1e-78bd5fb9e595', 'keypoint': {'x': 360.0991, 'y': 516.7925}, 'name': 'yellow'}, {'id': '5a97f3d8-463a-4c65-9906-b8cf5f772e2b', 'keypoint': {'x': 408.3538, 'y': 526.9104}, 'name': 'yellow'}, {'id': 'f2444027-7c76-40d2-9424-36c46342a49f', 'keypoint': {'x': 318.8491, 'y': 449.0802}, 'name': 'blue'}, {'id': '0b2e3fca-8a71-43d2-a83e-f02af09891b9', 'keypoint': {'x': 315.7358, 'y': 480.2123}, 'name': 'blue'}, {'id': '897e21a9-0e62-42b6-8c01-826ad479026c', 'keypoint': {'x': 378.0, 'y': 490.3302}, 'name': 'blue'}, {'id': 'ca6c0f72-0600-47ba-b4b1-05dc3566a83e', 'keypoint': {'x': 402.1274, 'y': 463.0896}, 'name': 'blue'}, {'id': '41397d83-cab1-4c4d-ba20-b13a1c93f043', 'keypoint': {'x': 419.25, 'y': 432.7358}, 'name': 'blue'}, {'id': '051bcea0-383b-4791-9652-d84f6ef3fe4c', 'keypoint': {'x': 348.4245, 'y': 488.7736}, 'name': 'green'}, {'id': '58f619eb-2384-4a80-97f9-10520e5226a9', 'keypoint': {'x': 311.8443, 'y': 539.3632}, 'name': 'green'}, {'id': 'b9c15fa4-b07d-4f16-9f7d-f007698c83c2', 'keypoint': {'x': 417.6934, 'y': 502.0047}, 'name': 'green'}, {'id': '5c0026dd-e833-4ad7-8339-361bad3686e0', 'keypoint': {'x': 378.0, 'y': 59.1509}, 'name': 'you'}]}
[/INST]
{'x': 494.7453, 'y': 534.6934}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the blue block that is on your far right.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'bb2508d6-bab0-4991-a4b4-1d60addd89be', 'keypoint': {'x': 374.8868, 'y': 32.6887}, 'name': 'you'}, {'id': '80bad823-3c11-42a6-9b7a-8f92fd053db8', 'keypoint': {'x': 286.1604, 'y': 504.3396}, 'name': 'yellow'}, {'id': '74407b61-1d02-4456-a3be-f44e6d0621e4', 'keypoint': {'x': 427.8113, 'y': 511.3443}, 'name': 'yellow'}, {'id': '0c043462-b620-40d3-b17e-3f17c7108248', 'keypoint': {'x': 391.2311, 'y': 435.8491}, 'name': 'green'}, {'id': 'e97e7600-f671-4774-970c-bcd8e011471e', 'keypoint': {'x': 311.066, 'y': 530.8019}, 'name': 'green'}, {'id': '3c45bdee-a533-4cab-8e50-cd3e05ea379a', 'keypoint': {'x': 375.6651, 'y': 518.3491}, 'name': 'green'}, {'id': 'fbe906a6-d83a-45bc-bd1e-7c81ccde8b9b', 'keypoint': {'x': 487.7406, 'y': 473.2075}, 'name': 'green'}, {'id': '662455ed-994d-45d8-b864-0f07ccfe9a1e', 'keypoint': {'x': 309.5094, 'y': 477.8774}, 'name': 'blue'}, {'id': '92e619ee-cc43-4ba6-80a0-fd23cabc2b68', 'keypoint': {'x': 378.7783, 'y': 457.6415}, 'name': 'blue'}, {'id': '4d3a4397-37ee-4ea1-bff7-ea4e40e4ba91', 'keypoint': {'x': 395.1226, 'y': 483.3255}, 'name': 'blue'}, {'id': '83c6c716-2497-4e0e-b233-3cda2dca8049', 'keypoint': {'x': 438.7075, 'y': 448.3019}, 'name': 'blue'}, {'id': '24b23d82-8281-40fd-9942-dc852b76fea4', 'keypoint': {'x': 444.1557, 'y': 484.1038}, 'name': 'blue'}, {'id': '541179c1-e37f-4069-99a8-fe58be3053c6', 'keypoint': {'x': 466.7264, 'y': 502.783}, 'name': 'blue'}, {'id': 'd950b6f4-0fff-425b-bf69-419e3a15863f', 'keypoint': {'x': 343.7547, 'y': 543.2547}, 'name': 'orange'}, {'id': 'a5aa3394-3b02-4613-8a7d-f14c9cd0fa58', 'keypoint': {'x': 431.7028, 'y': 539.3632}, 'name': 'orange'}, {'id': 'd7eb7db6-f69d-4df7-861e-0cc9dbebac2a', 'keypoint': {'x': 484.6274, 'y': 532.3585}, 'name': 'orange'}]}
[/INST]
{'x': 309.5094, 'y': 477.8774}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:In the center, there's a row of three blocks. A green block followed by two blue. Pick up the second blue.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'bb2508d6-bab0-4991-a4b4-1d60addd89be', 'keypoint': {'x': 374.8868, 'y': 32.6887}, 'name': 'you'}, {'id': '80bad823-3c11-42a6-9b7a-8f92fd053db8', 'keypoint': {'x': 286.1604, 'y': 504.3396}, 'name': 'yellow'}, {'id': '74407b61-1d02-4456-a3be-f44e6d0621e4', 'keypoint': {'x': 427.8113, 'y': 511.3443}, 'name': 'yellow'}, {'id': '0c043462-b620-40d3-b17e-3f17c7108248', 'keypoint': {'x': 391.2311, 'y': 435.8491}, 'name': 'green'}, {'id': 'e97e7600-f671-4774-970c-bcd8e011471e', 'keypoint': {'x': 311.066, 'y': 530.8019}, 'name': 'green'}, {'id': '3c45bdee-a533-4cab-8e50-cd3e05ea379a', 'keypoint': {'x': 375.6651, 'y': 518.3491}, 'name': 'green'}, {'id': 'fbe906a6-d83a-45bc-bd1e-7c81ccde8b9b', 'keypoint': {'x': 487.7406, 'y': 473.2075}, 'name': 'green'}, {'id': '662455ed-994d-45d8-b864-0f07ccfe9a1e', 'keypoint': {'x': 309.5094, 'y': 477.8774}, 'name': 'blue'}, {'id': '92e619ee-cc43-4ba6-80a0-fd23cabc2b68', 'keypoint': {'x': 378.7783, 'y': 457.6415}, 'name': 'blue'}, {'id': '4d3a4397-37ee-4ea1-bff7-ea4e40e4ba91', 'keypoint': {'x': 395.1226, 'y': 483.3255}, 'name': 'blue'}, {'id': '83c6c716-2497-4e0e-b233-3cda2dca8049', 'keypoint': {'x': 438.7075, 'y': 448.3019}, 'name': 'blue'}, {'id': '24b23d82-8281-40fd-9942-dc852b76fea4', 'keypoint': {'x': 444.1557, 'y': 484.1038}, 'name': 'blue'}, {'id': '541179c1-e37f-4069-99a8-fe58be3053c6', 'keypoint': {'x': 466.7264, 'y': 502.783}, 'name': 'blue'}, {'id': 'd950b6f4-0fff-425b-bf69-419e3a15863f', 'keypoint': {'x': 343.7547, 'y': 543.2547}, 'name': 'orange'}, {'id': 'a5aa3394-3b02-4613-8a7d-f14c9cd0fa58', 'keypoint': {'x': 431.7028, 'y': 539.3632}, 'name': 'orange'}, {'id': 'd7eb7db6-f69d-4df7-861e-0cc9dbebac2a', 'keypoint': {'x': 484.6274, 'y': 532.3585}, 'name': 'orange'}]}
[/INST]
{'x': 378.7783, 'y': 457.6415}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:pick up the blue block that is second to the farthest left.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'a7c6f9ab-55b7-422b-8e7f-70902f2e94d6', 'keypoint': {'x': 320.4057, 'y': 488.7736}, 'name': 'orange'}, {'id': 'acc187bd-2783-4707-be6a-314a58518228', 'keypoint': {'x': 423.9198, 'y': 451.4151}, 'name': 'orange'}, {'id': '7b9652c2-a499-48cc-ab07-68e686e885ac', 'keypoint': {'x': 358.5425, 'y': 462.3113}, 'name': 'yellow'}, {'id': '6e4e6633-28b0-4e37-a08b-d90cdb596a0d', 'keypoint': {'x': 334.4151, 'y': 548.7028}, 'name': 'yellow'}, {'id': 'dc28f152-a36a-458c-9f8a-21f4093b66bd', 'keypoint': {'x': 418.4717, 'y': 547.9245}, 'name': 'yellow'}, {'id': '34029874-d3f4-4998-a6fa-85ee314b35b5', 'keypoint': {'x': 479.1792, 'y': 484.8821}, 'name': 'yellow'}, {'id': '3d4114df-2e6e-4d36-8e12-47bbffbd6150', 'keypoint': {'x': 314.1792, 'y': 452.1934}, 'name': 'blue'}, {'id': '021cc2a3-2e83-4079-9777-64d3bc57e829', 'keypoint': {'x': 293.1651, 'y': 557.2642}, 'name': 'blue'}, {'id': '49911b5f-df0a-4694-9ff5-d8165f0f587a', 'keypoint': {'x': 390.4528, 'y': 513.6792}, 'name': 'blue'}, {'id': '95b20075-c6a2-46e0-b203-f8004a3ee4a3', 'keypoint': {'x': 440.2642, 'y': 484.8821}, 'name': 'blue'}, {'id': '031e7d06-9b93-4602-a1ef-c48bce3518c4', 'keypoint': {'x': 462.0566, 'y': 469.316}, 'name': 'blue'}, {'id': '3f2fdd03-f2f9-4892-999b-713f9e988be6', 'keypoint': {'x': 493.1887, 'y': 465.4245}, 'name': 'blue'}, {'id': '7ed0f47f-3bdf-429a-ab98-1d1a9d3058fa', 'keypoint': {'x': 482.2925, 'y': 507.4528}, 'name': 'green'}, {'id': '1fb5251e-aa36-4074-87f5-edef1e5a25c7', 'keypoint': {'x': 491.6321, 'y': 542.4764}, 'name': 'green'}, {'id': 'e7cf6134-82a1-4efe-99a5-a098dcdb3a3e', 'keypoint': {'x': 334.4151, 'y': 516.0142}, 'name': 'green'}, {'id': '0aeb291a-183c-47ce-9552-8b4d4860c05f', 'keypoint': {'x': 376.4434, 'y': 44.3632}, 'name': 'you'}]}
[/INST]
{'x': 462.0566, 'y': 469.316}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Looking at the table, you are to pick up the green block that is closest to your end of the table.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'cad61a18-85b8-4f6e-93b3-c3ce9e28cda2', 'keypoint': {'x': 316.5142, 'y': 554.9292}, 'name': 'orange'}, {'id': '17682384-25d2-46d1-ae87-f04077458497', 'keypoint': {'x': 330.5236, 'y': 516.7925}, 'name': 'orange'}, {'id': '5d36ae41-004b-494e-b32d-33279b1dbfee', 'keypoint': {'x': 447.2689, 'y': 457.6415}, 'name': 'orange'}, {'id': 'fee58518-97d8-4c98-9b1b-ebeaa7ef3967', 'keypoint': {'x': 488.5189, 'y': 530.8019}, 'name': 'orange'}, {'id': '4fa5b6b4-f98b-4588-95e6-8255a4bef84b', 'keypoint': {'x': 380.3349, 'y': 446.7453}, 'name': 'yellow'}, {'id': '22f0cd3c-99be-40d9-9b3f-a7bc782da1f8', 'keypoint': {'x': 297.8349, 'y': 529.2453}, 'name': 'yellow'}, {'id': 'd5f859c2-a998-4ee4-bff1-d6025abd4883', 'keypoint': {'x': 375.6651, 'y': 536.25}, 'name': 'yellow'}, {'id': '96691580-8803-4b4b-b31f-4920d402e03e', 'keypoint': {'x': 473.7311, 'y': 435.0708}, 'name': 'yellow'}, {'id': '86653055-abe8-4a82-8b79-fb7f461438c0', 'keypoint': {'x': 474.5094, 'y': 487.217}, 'name': 'yellow'}, {'id': '34e55929-db77-4549-8f50-5adc4da9bd5d', 'keypoint': {'x': 370.9953, 'y': 492.6651}, 'name': 'blue'}, {'id': '500fbe66-22dd-4552-a43c-63bf2efeb1c9', 'keypoint': {'x': 417.6934, 'y': 477.0991}, 'name': 'blue'}, {'id': '85cb36b6-c443-488a-85b6-363a2284870a', 'keypoint': {'x': 345.3113, 'y': 424.9528}, 'name': 'green'}, {'id': '1c037117-1e04-4cd7-a544-c3e10b3639bb', 'keypoint': {'x': 295.5, 'y': 466.9811}, 'name': 'green'}, {'id': 'a8f28480-33ff-4a0a-883f-f1f226ced914', 'keypoint': {'x': 299.3915, 'y': 501.2264}, 'name': 'green'}, {'id': 'b9578593-4dcd-4808-9119-c6439b45d587', 'keypoint': {'x': 427.033, 'y': 534.6934}, 'name': 'green'}, {'id': '70024c2f-19b2-412a-a76d-d9e741635b7e', 'keypoint': {'x': 380.3349, 'y': 33.467}, 'name': 'you'}]}
[/INST]
{'x': 427.033, 'y': 534.6934}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:The one you want will be green but it will be in the middle of the pack in between a blue and a yellow one
 {'width': 756, 'height': 660, 'annotations': [{'id': '8beb9a9e-a451-450a-b594-7a9ae18e74f9', 'keypoint': {'x': 458.9434, 'y': 531.5802}, 'name': 'orange'}, {'id': 'ee956837-2183-4b52-8aa2-dfa990f1d376', 'keypoint': {'x': 386.5613, 'y': 529.2453}, 'name': 'yellow'}, {'id': '427a5df9-dc11-494e-8632-81ffcd5e66c8', 'keypoint': {'x': 471.3962, 'y': 453.75}, 'name': 'yellow'}, {'id': 'bcc302a4-c191-4bc2-82e0-1ceedd4a2928', 'keypoint': {'x': 329.7453, 'y': 453.75}, 'name': 'blue'}, {'id': '8584b004-32b4-47bd-9100-a2eab16c5301', 'keypoint': {'x': 294.7217, 'y': 501.2264}, 'name': 'blue'}, {'id': 'ba4430e4-6293-47e3-b512-0e78897c96fb', 'keypoint': {'x': 317.2925, 'y': 520.684}, 'name': 'blue'}, {'id': '9c57d65f-59e8-4d45-a882-721a56841969', 'keypoint': {'x': 433.2594, 'y': 498.1132}, 'name': 'blue'}, {'id': '0e241c7c-e225-4266-9044-a65451aceb68', 'keypoint': {'x': 361.6557, 'y': 428.8443}, 'name': 'green'}, {'id': '050252d9-484a-4a87-a46c-1f69f5d7ea14', 'keypoint': {'x': 416.9151, 'y': 435.8491}, 'name': 'green'}, {'id': '5b9ab273-0e91-4589-9a86-d2969adc4e30', 'keypoint': {'x': 435.5943, 'y': 445.967}, 'name': 'green'}, {'id': '4b452968-0027-4b94-92e6-e8dcd32d1e25', 'keypoint': {'x': 468.283, 'y': 489.5519}, 'name': 'green'}, {'id': '3ab98d12-5e5c-4bdf-931d-55f2ee2d23b2', 'keypoint': {'x': 386.5613, 'y': 498.8915}, 'name': 'green'}, {'id': '416f60f7-59aa-46b3-858c-7dfde5e23875', 'keypoint': {'x': 353.0943, 'y': 497.3349}, 'name': 'green'}, {'id': 'ce8faeb7-52eb-4dc8-8db9-4295f1b077e3', 'keypoint': {'x': 329.7453, 'y': 491.8868}, 'name': 'green'}, {'id': '6ea4f70c-aa80-4f14-a1ac-d26081a22a0d', 'keypoint': {'x': 379.5566, 'y': 52.1462}, 'name': 'you'}]}
[/INST]
{'x': 386.5613, 'y': 498.8915}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Stand up, reach out over the table, and grab the yellow block that is touching the blue block closest to me.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'b328ac50-08b5-486e-a937-2cddbc04245c', 'keypoint': {'x': 304.8396, 'y': 434.2925}, 'name': 'orange'}, {'id': '7344b92c-5683-4670-9cef-33f4cf8e73fb', 'keypoint': {'x': 472.9528, 'y': 456.0849}, 'name': 'orange'}, {'id': '6958bbb7-3dce-4cfe-9049-02ebc5a32e43', 'keypoint': {'x': 306.3962, 'y': 460.7547}, 'name': 'yellow'}, {'id': '62285f29-522c-498a-9fdb-53e2e8421ea5', 'keypoint': {'x': 330.5236, 'y': 470.8726}, 'name': 'yellow'}, {'id': 'e4402e53-f01e-4254-a874-f6ffd3892ac4', 'keypoint': {'x': 381.8915, 'y': 475.5425}, 'name': 'yellow'}, {'id': '3f3ed2fa-5312-418f-9f29-f4750a72aa9e', 'keypoint': {'x': 402.9057, 'y': 435.0708}, 'name': 'yellow'}, {'id': 'fe5ba8f4-1f70-4712-ae25-cd20ac9670d6', 'keypoint': {'x': 455.8302, 'y': 481.7689}, 'name': 'yellow'}, {'id': 'bd1c4ddb-8345-4f24-a982-f6cd6a64fb0e', 'keypoint': {'x': 501.75, 'y': 505.8962}, 'name': 'yellow'}, {'id': '8f332be0-ad10-4130-9840-f3a1dc330890', 'keypoint': {'x': 352.316, 'y': 513.6792}, 'name': 'yellow'}, {'id': '5b19adbf-7efb-4a90-9f40-4c0bf2f3b9a8', 'keypoint': {'x': 324.2972, 'y': 499.6698}, 'name': 'yellow'}, {'id': '7f124547-f8b7-4280-b7e1-401876ce2bdb', 'keypoint': {'x': 308.7311, 'y': 552.5943}, 'name': 'yellow'}, {'id': '8a1fd773-bc7c-4be5-a292-c1c53d5c77df', 'keypoint': {'x': 298.6132, 'y': 491.8868}, 'name': 'blue'}, {'id': '7be8714d-676d-466e-a70a-e9d078b757f4', 'keypoint': {'x': 381.1132, 'y': 497.3349}, 'name': 'blue'}, {'id': 'b1ee1721-f1d2-4cc8-828b-2e6c5072d54f', 'keypoint': {'x': 451.9387, 'y': 432.7358}, 'name': 'blue'}, {'id': '0416eebd-320e-40d8-bbc7-a94a6333aa95', 'keypoint': {'x': 497.0802, 'y': 485.6604}, 'name': 'green'}, {'id': '2240b1ab-d515-4872-a587-09ca836dc397', 'keypoint': {'x': 378.7783, 'y': 28.7972}, 'name': 'you'}]}
[/INST]
{'x': 324.2972, 'y': 499.6698}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the 2nd green block from your left.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'd5bcc2b7-ff91-4686-8f50-3a73ffd3beeb', 'keypoint': {'x': 315.7358, 'y': 436.6274}, 'name': 'orange'}, {'id': '70515b86-98d5-4fa6-9118-a0bc484f6de1', 'keypoint': {'x': 313.4009, 'y': 525.3538}, 'name': 'orange'}, {'id': 'b40b0958-f706-45d2-a042-0440eb67121b', 'keypoint': {'x': 380.3349, 'y': 515.2358}, 'name': 'orange'}, {'id': '9dcd051f-bad9-4f84-b653-9a30a72cb545', 'keypoint': {'x': 389.6745, 'y': 552.5943}, 'name': 'orange'}, {'id': 'b0de6329-f4d9-486c-91ed-478f28262cff', 'keypoint': {'x': 363.9906, 'y': 439.7406}, 'name': 'yellow'}, {'id': '10eeedc5-3253-42e7-b401-33c4616e4623', 'keypoint': {'x': 396.6792, 'y': 457.6415}, 'name': 'yellow'}, {'id': 'f7642eee-44ca-4f1d-abf7-a79d43dfbd0d', 'keypoint': {'x': 279.934, 'y': 535.4717}, 'name': 'yellow'}, {'id': 'd87c603d-89da-4c0b-8f61-a496275c1297', 'keypoint': {'x': 311.066, 'y': 487.217}, 'name': 'blue'}, {'id': '2065d0e4-12eb-4ab3-bde4-75688bb3c142', 'keypoint': {'x': 457.3868, 'y': 439.7406}, 'name': 'green'}, {'id': '1550df58-31c4-49f4-8e6d-7db4a57bc294', 'keypoint': {'x': 435.5943, 'y': 445.967}, 'name': 'green'}, {'id': '1990bf50-38ac-43ab-8581-26778cea80ed', 'keypoint': {'x': 458.9434, 'y': 484.8821}, 'name': 'green'}, {'id': '7895d85c-2e57-4008-b9d7-74625608a42c', 'keypoint': {'x': 485.4057, 'y': 515.2358}, 'name': 'green'}, {'id': '991a571a-03aa-4a00-b80d-3d6a0845625b', 'keypoint': {'x': 440.2642, 'y': 514.4575}, 'name': 'green'}, {'id': 'e0e1086c-3a47-42f6-98fd-868d0756cd4d', 'keypoint': {'x': 378.7783, 'y': 495.7783}, 'name': 'green'}, {'id': '3d132917-dd96-4ae9-9302-da261696357f', 'keypoint': {'x': 333.6368, 'y': 505.1179}, 'name': 'green'}, {'id': 'bc7494f1-b049-49c6-bf24-85e942969065', 'keypoint': {'x': 383.4481, 'y': 56.0377}, 'name': 'you'}]}
[/INST]
{'x': 378.7783, 'y': 495.7783}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the furthest orange block on your left
 {'width': 756, 'height': 660, 'annotations': [{'id': '82770085-a20a-439c-b1a3-f74a8d8ccdd0', 'keypoint': {'x': 322.0293, 'y': 491.9856}, 'name': 'yellow'}, {'id': '975d2226-07a6-46ec-8aa1-01f44597bb5c', 'keypoint': {'x': 376.8281, 'y': 495.1624}, 'name': 'yellow'}, {'id': '9fbd0dd7-292a-46c1-854e-eb8c4bb880cf', 'keypoint': {'x': 402.2421, 'y': 551.5495}, 'name': 'yellow'}, {'id': 'c2d61b6b-0585-4aef-acaf-17bad430192b', 'keypoint': {'x': 464.9827, 'y': 498.3391}, 'name': 'yellow'}, {'id': '13d3414e-a81b-4dad-8963-6fbda2f0d436', 'keypoint': {'x': 376.0339, 'y': 27.387}, 'name': 'you'}, {'id': '9b80a1cd-fed1-49e9-8b69-3cd0088f7fca', 'keypoint': {'x': 302.9689, 'y': 454.6589}, 'name': 'blue'}, {'id': '17a9c4b0-bf5e-45d6-aa6f-94959ad755e2', 'keypoint': {'x': 421.3025, 'y': 473.7194}, 'name': 'blue'}, {'id': '4a1722fe-5d45-4bf9-a201-c0b8f7acc36b', 'keypoint': {'x': 456.2467, 'y': 437.981}, 'name': 'blue'}, {'id': '680351fd-2723-4f71-9103-17bb8a809738', 'keypoint': {'x': 375.2398, 'y': 452.2763}, 'name': 'orange'}, {'id': '188969f7-812f-47f7-a97e-5a0652f78c64', 'keypoint': {'x': 354.5909, 'y': 508.6635}, 'name': 'orange'}, {'id': 'b57e9350-23ed-40aa-96cc-2811f14cd796', 'keypoint': {'x': 396.6828, 'y': 520.5763}, 'name': 'orange'}, {'id': 'f041b6b6-0396-42fe-99f5-46de7b8e6b23', 'keypoint': {'x': 431.6269, 'y': 507.8693}, 'name': 'orange'}, {'id': 'ae76b953-f8b0-4121-9569-ce66605c9783', 'keypoint': {'x': 433.2153, 'y': 530.9007}, 'name': 'orange'}, {'id': '256e0a15-32e1-445f-a768-a8adb00d4905', 'keypoint': {'x': 341.0898, 'y': 438.7752}, 'name': 'green'}, {'id': '51ccd976-c149-458d-960f-bd00da3a5131', 'keypoint': {'x': 318.0584, 'y': 536.46}, 'name': 'green'}, {'id': 'd3e5315b-8e96-4659-97cb-07d656a65db8', 'keypoint': {'x': 503.8978, 'y': 456.2473}, 'name': 'green'}]}
[/INST]
{'x': 433.2153, 'y': 530.9007}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the yellow block that is just to your left of the blue block that is farthest to your right.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'b328ac50-08b5-486e-a937-2cddbc04245c', 'keypoint': {'x': 304.8396, 'y': 434.2925}, 'name': 'orange'}, {'id': '7344b92c-5683-4670-9cef-33f4cf8e73fb', 'keypoint': {'x': 472.9528, 'y': 456.0849}, 'name': 'orange'}, {'id': '6958bbb7-3dce-4cfe-9049-02ebc5a32e43', 'keypoint': {'x': 306.3962, 'y': 460.7547}, 'name': 'yellow'}, {'id': '62285f29-522c-498a-9fdb-53e2e8421ea5', 'keypoint': {'x': 330.5236, 'y': 470.8726}, 'name': 'yellow'}, {'id': 'e4402e53-f01e-4254-a874-f6ffd3892ac4', 'keypoint': {'x': 381.8915, 'y': 475.5425}, 'name': 'yellow'}, {'id': '3f3ed2fa-5312-418f-9f29-f4750a72aa9e', 'keypoint': {'x': 402.9057, 'y': 435.0708}, 'name': 'yellow'}, {'id': 'fe5ba8f4-1f70-4712-ae25-cd20ac9670d6', 'keypoint': {'x': 455.8302, 'y': 481.7689}, 'name': 'yellow'}, {'id': 'bd1c4ddb-8345-4f24-a982-f6cd6a64fb0e', 'keypoint': {'x': 501.75, 'y': 505.8962}, 'name': 'yellow'}, {'id': '8f332be0-ad10-4130-9840-f3a1dc330890', 'keypoint': {'x': 352.316, 'y': 513.6792}, 'name': 'yellow'}, {'id': '5b19adbf-7efb-4a90-9f40-4c0bf2f3b9a8', 'keypoint': {'x': 324.2972, 'y': 499.6698}, 'name': 'yellow'}, {'id': '7f124547-f8b7-4280-b7e1-401876ce2bdb', 'keypoint': {'x': 308.7311, 'y': 552.5943}, 'name': 'yellow'}, {'id': '8a1fd773-bc7c-4be5-a292-c1c53d5c77df', 'keypoint': {'x': 298.6132, 'y': 491.8868}, 'name': 'blue'}, {'id': '7be8714d-676d-466e-a70a-e9d078b757f4', 'keypoint': {'x': 381.1132, 'y': 497.3349}, 'name': 'blue'}, {'id': 'b1ee1721-f1d2-4cc8-828b-2e6c5072d54f', 'keypoint': {'x': 451.9387, 'y': 432.7358}, 'name': 'blue'}, {'id': '0416eebd-320e-40d8-bbc7-a94a6333aa95', 'keypoint': {'x': 497.0802, 'y': 485.6604}, 'name': 'green'}, {'id': '2240b1ab-d515-4872-a587-09ca836dc397', 'keypoint': {'x': 378.7783, 'y': 28.7972}, 'name': 'you'}]}
[/INST]
{'x': 324.2972, 'y': 499.6698}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:the blue block that is the closest to you
 {'width': 756, 'height': 660, 'annotations': [{'id': 'fe98e63d-c226-4a25-9e75-229c9a2d828a', 'keypoint': {'x': 377.2217, 'y': 28.7972}, 'name': 'you'}, {'id': 'c1bfc6fe-9e0d-4152-aab0-9efe03f99ee1', 'keypoint': {'x': 291.6085, 'y': 487.217}, 'name': 'yellow'}, {'id': 'eefd5787-08ef-4674-ae9e-c41f4a2ed951', 'keypoint': {'x': 314.1792, 'y': 539.3632}, 'name': 'yellow'}, {'id': '8987e25d-8eb0-42ab-aa8e-390a7b046e44', 'keypoint': {'x': 363.2123, 'y': 544.8113}, 'name': 'yellow'}, {'id': 'af54d764-7469-4b39-95f2-d25ac6241715', 'keypoint': {'x': 434.0377, 'y': 542.4764}, 'name': 'yellow'}, {'id': 'c426887c-43fa-4c7c-9ee6-447f48c5b353', 'keypoint': {'x': 463.6132, 'y': 487.9953}, 'name': 'yellow'}, {'id': 'f1402f90-fe95-42c0-93e5-75019494beba', 'keypoint': {'x': 497.0802, 'y': 442.0755}, 'name': 'yellow'}, {'id': 'ad7b2c3b-efac-4705-8dc4-076514da9f0d', 'keypoint': {'x': 319.6274, 'y': 463.8679}, 'name': 'blue'}, {'id': '57fd11a6-5352-4e90-8c5e-78e080a9f63b', 'keypoint': {'x': 458.1651, 'y': 516.0142}, 'name': 'blue'}, {'id': 'bffb87fb-647e-44e1-b4c0-9bbf464cc4da', 'keypoint': {'x': 353.8726, 'y': 445.1887}, 'name': 'green'}, {'id': '2be751d9-34f0-411b-89e7-1b20b054e1ee', 'keypoint': {'x': 388.8962, 'y': 498.1132}, 'name': 'green'}, {'id': '98a41430-e570-492e-969a-8ea45857d765', 'keypoint': {'x': 395.9009, 'y': 533.1368}, 'name': 'green'}, {'id': '0327cb90-c17c-4528-8964-e978fbff5999', 'keypoint': {'x': 427.8113, 'y': 484.8821}, 'name': 'green'}, {'id': '97f91dd8-6cd8-4194-8783-e9e3b5786e68', 'keypoint': {'x': 456.6085, 'y': 452.9717}, 'name': 'green'}, {'id': '4ebadfd4-3497-4f80-a4e9-a870c25786bd', 'keypoint': {'x': 504.8632, 'y': 502.0047}, 'name': 'green'}, {'id': 'd2ff782c-210c-4090-91a5-de43a78f1232', 'keypoint': {'x': 331.3019, 'y': 512.9009}, 'name': 'orange'}]}
[/INST]
{'x': 319.6274, 'y': 463.8679}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the middle orange block
 {'width': 756, 'height': 660, 'annotations': [{'id': '53de559f-b2c6-46fa-821d-b5a4e98c67c3', 'keypoint': {'x': 332.0802, 'y': 517.5708}, 'name': 'orange'}, {'id': 'fc5cb394-ebdb-4064-bb12-1d3da5b0cb6f', 'keypoint': {'x': 316.5142, 'y': 540.1415}, 'name': 'orange'}, {'id': 'b8f5d815-90be-4cd5-951f-44207215b594', 'keypoint': {'x': 373.3302, 'y': 537.8066}, 'name': 'orange'}, {'id': '78d88ded-7431-4260-a864-725e2dfef6ab', 'keypoint': {'x': 309.5094, 'y': 498.1132}, 'name': 'yellow'}, {'id': 'e5215008-d111-4bda-9009-f2c0795ffb0d', 'keypoint': {'x': 388.1179, 'y': 471.6509}, 'name': 'yellow'}, {'id': 'b722e1f1-0a3c-47ef-b35f-e9cd85898169', 'keypoint': {'x': 388.1179, 'y': 501.2264}, 'name': 'yellow'}, {'id': '27ad7266-481b-40f9-aaa0-e71dabd71d5e', 'keypoint': {'x': 413.8019, 'y': 491.1085}, 'name': 'yellow'}, {'id': '71348b8e-08dc-4a76-be0e-68cbee1a56a4', 'keypoint': {'x': 416.1368, 'y': 449.8585}, 'name': 'yellow'}, {'id': 'e46d6178-4284-433d-a81f-bc6cbbf7cdbb', 'keypoint': {'x': 490.8538, 'y': 505.1179}, 'name': 'yellow'}, {'id': '272a554a-aad9-4bba-9062-941465beeaa5', 'keypoint': {'x': 465.1698, 'y': 554.1509}, 'name': 'yellow'}, {'id': 'c6ba259b-1c8e-43d4-8bdb-404594f1c175', 'keypoint': {'x': 310.2877, 'y': 438.9623}, 'name': 'blue'}, {'id': '693a6454-810f-447d-8417-a2c57867901d', 'keypoint': {'x': 354.6509, 'y': 477.8774}, 'name': 'blue'}, {'id': '3ec7fa7f-c792-4c99-a423-b40bbb665d89', 'keypoint': {'x': 427.8113, 'y': 428.8443}, 'name': 'blue'}, {'id': '3e041f2e-9e78-4001-9fa0-3bebee80ac07', 'keypoint': {'x': 440.2642, 'y': 498.8915}, 'name': 'green'}, {'id': 'e2e59b33-f4e7-47cd-b137-77a874c834c7', 'keypoint': {'x': 450.3821, 'y': 519.1274}, 'name': 'green'}, {'id': '707959b0-76f6-463a-bc2f-62cd702815f4', 'keypoint': {'x': 377.2217, 'y': 55.2594}, 'name': 'you'}]}
[/INST]
{'x': 332.0802, 'y': 517.5708}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:there is a row of three green blocks with a yeellow underneath. You need to pick up the center green one on this row.
 {'width': 756, 'height': 660, 'annotations': [{'id': '8beb9a9e-a451-450a-b594-7a9ae18e74f9', 'keypoint': {'x': 458.9434, 'y': 531.5802}, 'name': 'orange'}, {'id': 'ee956837-2183-4b52-8aa2-dfa990f1d376', 'keypoint': {'x': 386.5613, 'y': 529.2453}, 'name': 'yellow'}, {'id': '427a5df9-dc11-494e-8632-81ffcd5e66c8', 'keypoint': {'x': 471.3962, 'y': 453.75}, 'name': 'yellow'}, {'id': 'bcc302a4-c191-4bc2-82e0-1ceedd4a2928', 'keypoint': {'x': 329.7453, 'y': 453.75}, 'name': 'blue'}, {'id': '8584b004-32b4-47bd-9100-a2eab16c5301', 'keypoint': {'x': 294.7217, 'y': 501.2264}, 'name': 'blue'}, {'id': 'ba4430e4-6293-47e3-b512-0e78897c96fb', 'keypoint': {'x': 317.2925, 'y': 520.684}, 'name': 'blue'}, {'id': '9c57d65f-59e8-4d45-a882-721a56841969', 'keypoint': {'x': 433.2594, 'y': 498.1132}, 'name': 'blue'}, {'id': '0e241c7c-e225-4266-9044-a65451aceb68', 'keypoint': {'x': 361.6557, 'y': 428.8443}, 'name': 'green'}, {'id': '050252d9-484a-4a87-a46c-1f69f5d7ea14', 'keypoint': {'x': 416.9151, 'y': 435.8491}, 'name': 'green'}, {'id': '5b9ab273-0e91-4589-9a86-d2969adc4e30', 'keypoint': {'x': 435.5943, 'y': 445.967}, 'name': 'green'}, {'id': '4b452968-0027-4b94-92e6-e8dcd32d1e25', 'keypoint': {'x': 468.283, 'y': 489.5519}, 'name': 'green'}, {'id': '3ab98d12-5e5c-4bdf-931d-55f2ee2d23b2', 'keypoint': {'x': 386.5613, 'y': 498.8915}, 'name': 'green'}, {'id': '416f60f7-59aa-46b3-858c-7dfde5e23875', 'keypoint': {'x': 353.0943, 'y': 497.3349}, 'name': 'green'}, {'id': 'ce8faeb7-52eb-4dc8-8db9-4295f1b077e3', 'keypoint': {'x': 329.7453, 'y': 491.8868}, 'name': 'green'}, {'id': '6ea4f70c-aa80-4f14-a1ac-d26081a22a0d', 'keypoint': {'x': 379.5566, 'y': 52.1462}, 'name': 'you'}]}
[/INST]
{'x': 353.0943, 'y': 497.3349}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the green block closest to you.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'cad61a18-85b8-4f6e-93b3-c3ce9e28cda2', 'keypoint': {'x': 316.5142, 'y': 554.9292}, 'name': 'orange'}, {'id': '17682384-25d2-46d1-ae87-f04077458497', 'keypoint': {'x': 330.5236, 'y': 516.7925}, 'name': 'orange'}, {'id': '5d36ae41-004b-494e-b32d-33279b1dbfee', 'keypoint': {'x': 447.2689, 'y': 457.6415}, 'name': 'orange'}, {'id': 'fee58518-97d8-4c98-9b1b-ebeaa7ef3967', 'keypoint': {'x': 488.5189, 'y': 530.8019}, 'name': 'orange'}, {'id': '4fa5b6b4-f98b-4588-95e6-8255a4bef84b', 'keypoint': {'x': 380.3349, 'y': 446.7453}, 'name': 'yellow'}, {'id': '22f0cd3c-99be-40d9-9b3f-a7bc782da1f8', 'keypoint': {'x': 297.8349, 'y': 529.2453}, 'name': 'yellow'}, {'id': 'd5f859c2-a998-4ee4-bff1-d6025abd4883', 'keypoint': {'x': 375.6651, 'y': 536.25}, 'name': 'yellow'}, {'id': '96691580-8803-4b4b-b31f-4920d402e03e', 'keypoint': {'x': 473.7311, 'y': 435.0708}, 'name': 'yellow'}, {'id': '86653055-abe8-4a82-8b79-fb7f461438c0', 'keypoint': {'x': 474.5094, 'y': 487.217}, 'name': 'yellow'}, {'id': '34e55929-db77-4549-8f50-5adc4da9bd5d', 'keypoint': {'x': 370.9953, 'y': 492.6651}, 'name': 'blue'}, {'id': '500fbe66-22dd-4552-a43c-63bf2efeb1c9', 'keypoint': {'x': 417.6934, 'y': 477.0991}, 'name': 'blue'}, {'id': '85cb36b6-c443-488a-85b6-363a2284870a', 'keypoint': {'x': 345.3113, 'y': 424.9528}, 'name': 'green'}, {'id': '1c037117-1e04-4cd7-a544-c3e10b3639bb', 'keypoint': {'x': 295.5, 'y': 466.9811}, 'name': 'green'}, {'id': 'a8f28480-33ff-4a0a-883f-f1f226ced914', 'keypoint': {'x': 299.3915, 'y': 501.2264}, 'name': 'green'}, {'id': 'b9578593-4dcd-4808-9119-c6439b45d587', 'keypoint': {'x': 427.033, 'y': 534.6934}, 'name': 'green'}, {'id': '70024c2f-19b2-412a-a76d-d9e741635b7e', 'keypoint': {'x': 380.3349, 'y': 33.467}, 'name': 'you'}]}
[/INST]
{'x': 345.3113, 'y': 424.9528}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the blue block on your right.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'fe98e63d-c226-4a25-9e75-229c9a2d828a', 'keypoint': {'x': 377.2217, 'y': 28.7972}, 'name': 'you'}, {'id': 'c1bfc6fe-9e0d-4152-aab0-9efe03f99ee1', 'keypoint': {'x': 291.6085, 'y': 487.217}, 'name': 'yellow'}, {'id': 'eefd5787-08ef-4674-ae9e-c41f4a2ed951', 'keypoint': {'x': 314.1792, 'y': 539.3632}, 'name': 'yellow'}, {'id': '8987e25d-8eb0-42ab-aa8e-390a7b046e44', 'keypoint': {'x': 363.2123, 'y': 544.8113}, 'name': 'yellow'}, {'id': 'af54d764-7469-4b39-95f2-d25ac6241715', 'keypoint': {'x': 434.0377, 'y': 542.4764}, 'name': 'yellow'}, {'id': 'c426887c-43fa-4c7c-9ee6-447f48c5b353', 'keypoint': {'x': 463.6132, 'y': 487.9953}, 'name': 'yellow'}, {'id': 'f1402f90-fe95-42c0-93e5-75019494beba', 'keypoint': {'x': 497.0802, 'y': 442.0755}, 'name': 'yellow'}, {'id': 'ad7b2c3b-efac-4705-8dc4-076514da9f0d', 'keypoint': {'x': 319.6274, 'y': 463.8679}, 'name': 'blue'}, {'id': '57fd11a6-5352-4e90-8c5e-78e080a9f63b', 'keypoint': {'x': 458.1651, 'y': 516.0142}, 'name': 'blue'}, {'id': 'bffb87fb-647e-44e1-b4c0-9bbf464cc4da', 'keypoint': {'x': 353.8726, 'y': 445.1887}, 'name': 'green'}, {'id': '2be751d9-34f0-411b-89e7-1b20b054e1ee', 'keypoint': {'x': 388.8962, 'y': 498.1132}, 'name': 'green'}, {'id': '98a41430-e570-492e-969a-8ea45857d765', 'keypoint': {'x': 395.9009, 'y': 533.1368}, 'name': 'green'}, {'id': '0327cb90-c17c-4528-8964-e978fbff5999', 'keypoint': {'x': 427.8113, 'y': 484.8821}, 'name': 'green'}, {'id': '97f91dd8-6cd8-4194-8783-e9e3b5786e68', 'keypoint': {'x': 456.6085, 'y': 452.9717}, 'name': 'green'}, {'id': '4ebadfd4-3497-4f80-a4e9-a870c25786bd', 'keypoint': {'x': 504.8632, 'y': 502.0047}, 'name': 'green'}, {'id': 'd2ff782c-210c-4090-91a5-de43a78f1232', 'keypoint': {'x': 331.3019, 'y': 512.9009}, 'name': 'orange'}]}
[/INST]
{'x': 319.6274, 'y': 463.8679}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:pick up the yellow block
 {'width': 756, 'height': 660, 'annotations': [{'id': '4665780d-e410-4494-8512-3557afddb1a4', 'keypoint': {'x': 311.8443, 'y': 504.3396}, 'name': None}, {'id': '565e6ab8-2ba1-4136-96ed-e63f3cdba23d', 'keypoint': {'x': 368.6604, 'y': 505.1179}, 'name': 'orange'}, {'id': '487a86d1-7908-404f-99e9-16ab87ea0d04', 'keypoint': {'x': 359.3208, 'y': 560.3774}, 'name': 'orange'}, {'id': 'ab7b691c-c47b-4c26-8083-096140fac645', 'keypoint': {'x': 413.0236, 'y': 447.5236}, 'name': 'orange'}, {'id': 'fdb6ba1e-6ab3-4d82-862c-1bdaf73940dc', 'keypoint': {'x': 464.3915, 'y': 459.9764}, 'name': 'orange'}, {'id': '7768e90f-d1af-4f66-abc0-01fa10bc5e61', 'keypoint': {'x': 447.2689, 'y': 530.8019}, 'name': 'orange'}, {'id': '698efbba-5197-4d9f-96eb-fdc8c950c48f', 'keypoint': {'x': 290.8302, 'y': 554.1509}, 'name': 'yellow'}, {'id': '1e9d76c9-a045-4417-94ae-129e3daeb548', 'keypoint': {'x': 407.5755, 'y': 519.9057}, 'name': 'blue'}, {'id': 'dc0ba9e0-8eb1-478a-88c5-3eb00d02992d', 'keypoint': {'x': 454.2736, 'y': 505.8962}, 'name': 'blue'}, {'id': '13165129-2db5-428d-ba05-982c1ae9a179', 'keypoint': {'x': 503.3066, 'y': 487.217}, 'name': 'green'}, {'id': 'a2feb4b6-353e-4876-b8e6-d5351bc5f056', 'keypoint': {'x': 459.7217, 'y': 490.3302}, 'name': 'green'}, {'id': 'bd636fd9-8ba5-408a-a286-413978cb55e7', 'keypoint': {'x': 362.434, 'y': 476.3208}, 'name': 'green'}, {'id': '4d316993-cf9a-4115-894b-940d569fafc4', 'keypoint': {'x': 311.066, 'y': 439.7406}, 'name': 'green'}, {'id': 'b6f5df26-1415-40e4-bbf7-01af31da49b9', 'keypoint': {'x': 380.3349, 'y': 544.8113}, 'name': 'green'}, {'id': 'f34f5fb5-74bd-4265-8123-8eed80fffdd5', 'keypoint': {'x': 322.7406, 'y': 536.25}, 'name': 'green'}, {'id': '847a9a8d-0e7a-49f2-b274-f833cdac716e', 'keypoint': {'x': 375.6651, 'y': 49.8113}, 'name': 'you'}]}
[/INST]
{'x': 290.8302, 'y': 554.1509}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:I would ask them to pick the fourth block on that's in front of them.
 {'width': 756, 'height': 660, 'annotations': [{'id': '53de559f-b2c6-46fa-821d-b5a4e98c67c3', 'keypoint': {'x': 332.0802, 'y': 517.5708}, 'name': 'orange'}, {'id': 'fc5cb394-ebdb-4064-bb12-1d3da5b0cb6f', 'keypoint': {'x': 316.5142, 'y': 540.1415}, 'name': 'orange'}, {'id': 'b8f5d815-90be-4cd5-951f-44207215b594', 'keypoint': {'x': 373.3302, 'y': 537.8066}, 'name': 'orange'}, {'id': '78d88ded-7431-4260-a864-725e2dfef6ab', 'keypoint': {'x': 309.5094, 'y': 498.1132}, 'name': 'yellow'}, {'id': 'e5215008-d111-4bda-9009-f2c0795ffb0d', 'keypoint': {'x': 388.1179, 'y': 471.6509}, 'name': 'yellow'}, {'id': 'b722e1f1-0a3c-47ef-b35f-e9cd85898169', 'keypoint': {'x': 388.1179, 'y': 501.2264}, 'name': 'yellow'}, {'id': '27ad7266-481b-40f9-aaa0-e71dabd71d5e', 'keypoint': {'x': 413.8019, 'y': 491.1085}, 'name': 'yellow'}, {'id': '71348b8e-08dc-4a76-be0e-68cbee1a56a4', 'keypoint': {'x': 416.1368, 'y': 449.8585}, 'name': 'yellow'}, {'id': 'e46d6178-4284-433d-a81f-bc6cbbf7cdbb', 'keypoint': {'x': 490.8538, 'y': 505.1179}, 'name': 'yellow'}, {'id': '272a554a-aad9-4bba-9062-941465beeaa5', 'keypoint': {'x': 465.1698, 'y': 554.1509}, 'name': 'yellow'}, {'id': 'c6ba259b-1c8e-43d4-8bdb-404594f1c175', 'keypoint': {'x': 310.2877, 'y': 438.9623}, 'name': 'blue'}, {'id': '693a6454-810f-447d-8417-a2c57867901d', 'keypoint': {'x': 354.6509, 'y': 477.8774}, 'name': 'blue'}, {'id': '3ec7fa7f-c792-4c99-a423-b40bbb665d89', 'keypoint': {'x': 427.8113, 'y': 428.8443}, 'name': 'blue'}, {'id': '3e041f2e-9e78-4001-9fa0-3bebee80ac07', 'keypoint': {'x': 440.2642, 'y': 498.8915}, 'name': 'green'}, {'id': 'e2e59b33-f4e7-47cd-b137-77a874c834c7', 'keypoint': {'x': 450.3821, 'y': 519.1274}, 'name': 'green'}, {'id': '707959b0-76f6-463a-bc2f-62cd702815f4', 'keypoint': {'x': 377.2217, 'y': 55.2594}, 'name': 'you'}]}
[/INST]
{'x': 413.8019, 'y': 491.1085}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the blue block that is closest to the right side of the table.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'bb2508d6-bab0-4991-a4b4-1d60addd89be', 'keypoint': {'x': 374.8868, 'y': 32.6887}, 'name': 'you'}, {'id': '80bad823-3c11-42a6-9b7a-8f92fd053db8', 'keypoint': {'x': 286.1604, 'y': 504.3396}, 'name': 'yellow'}, {'id': '74407b61-1d02-4456-a3be-f44e6d0621e4', 'keypoint': {'x': 427.8113, 'y': 511.3443}, 'name': 'yellow'}, {'id': '0c043462-b620-40d3-b17e-3f17c7108248', 'keypoint': {'x': 391.2311, 'y': 435.8491}, 'name': 'green'}, {'id': 'e97e7600-f671-4774-970c-bcd8e011471e', 'keypoint': {'x': 311.066, 'y': 530.8019}, 'name': 'green'}, {'id': '3c45bdee-a533-4cab-8e50-cd3e05ea379a', 'keypoint': {'x': 375.6651, 'y': 518.3491}, 'name': 'green'}, {'id': 'fbe906a6-d83a-45bc-bd1e-7c81ccde8b9b', 'keypoint': {'x': 487.7406, 'y': 473.2075}, 'name': 'green'}, {'id': '662455ed-994d-45d8-b864-0f07ccfe9a1e', 'keypoint': {'x': 309.5094, 'y': 477.8774}, 'name': 'blue'}, {'id': '92e619ee-cc43-4ba6-80a0-fd23cabc2b68', 'keypoint': {'x': 378.7783, 'y': 457.6415}, 'name': 'blue'}, {'id': '4d3a4397-37ee-4ea1-bff7-ea4e40e4ba91', 'keypoint': {'x': 395.1226, 'y': 483.3255}, 'name': 'blue'}, {'id': '83c6c716-2497-4e0e-b233-3cda2dca8049', 'keypoint': {'x': 438.7075, 'y': 448.3019}, 'name': 'blue'}, {'id': '24b23d82-8281-40fd-9942-dc852b76fea4', 'keypoint': {'x': 444.1557, 'y': 484.1038}, 'name': 'blue'}, {'id': '541179c1-e37f-4069-99a8-fe58be3053c6', 'keypoint': {'x': 466.7264, 'y': 502.783}, 'name': 'blue'}, {'id': 'd950b6f4-0fff-425b-bf69-419e3a15863f', 'keypoint': {'x': 343.7547, 'y': 543.2547}, 'name': 'orange'}, {'id': 'a5aa3394-3b02-4613-8a7d-f14c9cd0fa58', 'keypoint': {'x': 431.7028, 'y': 539.3632}, 'name': 'orange'}, {'id': 'd7eb7db6-f69d-4df7-861e-0cc9dbebac2a', 'keypoint': {'x': 484.6274, 'y': 532.3585}, 'name': 'orange'}]}
[/INST]
{'x': 309.5094, 'y': 477.8774}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the blue block 2nd closest to you.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'c7519096-f7ea-4ab1-a66d-97c708976599', 'keypoint': {'x': 380.3349, 'y': 34.2453}, 'name': 'you'}, {'id': 'a95968ee-51d4-482d-ae79-97cb54663aed', 'keypoint': {'x': 293.9434, 'y': 479.434}, 'name': 'blue'}, {'id': '2fc8b299-6074-43eb-92bc-7ed758a958fc', 'keypoint': {'x': 329.7453, 'y': 557.2642}, 'name': 'blue'}, {'id': '11dbc045-de4f-4d3a-bc50-a1c2ba279bd1', 'keypoint': {'x': 366.3255, 'y': 546.3679}, 'name': 'blue'}, {'id': 'a3036737-fb7e-4f86-88c8-b0c5262cc051', 'keypoint': {'x': 379.5566, 'y': 438.9623}, 'name': 'blue'}, {'id': '5370f3f3-ef52-45e8-bdb4-3d96d54f4bd9', 'keypoint': {'x': 455.8302, 'y': 449.0802}, 'name': 'blue'}, {'id': '95294165-9354-41e7-b1be-60d4033bd1a3', 'keypoint': {'x': 318.0708, 'y': 454.5283}, 'name': 'yellow'}, {'id': '71f49ac9-1905-457f-9219-701cac7c42e4', 'keypoint': {'x': 318.0708, 'y': 530.8019}, 'name': 'yellow'}, {'id': '6f7a8a19-794f-4ffc-80ed-8a42dedeab0f', 'keypoint': {'x': 463.6132, 'y': 524.5755}, 'name': 'yellow'}, {'id': '407068c6-6a1e-4a14-b9d0-c0a4ed13b9c2', 'keypoint': {'x': 395.1226, 'y': 466.2028}, 'name': 'green'}, {'id': '847fb0f9-02cc-42df-8984-2b193e2abb8d', 'keypoint': {'x': 403.684, 'y': 523.0189}, 'name': 'green'}, {'id': 'bd78738b-2c39-4b85-9575-960d46ecd66c', 'keypoint': {'x': 325.0755, 'y': 500.4481}, 'name': 'orange'}, {'id': 'dafb3b80-4c1e-48a7-b398-e3aa56c5dcb0', 'keypoint': {'x': 374.1085, 'y': 515.2358}, 'name': 'orange'}, {'id': '36cf3f26-cac5-4368-91eb-ae8937ec2f91', 'keypoint': {'x': 412.2453, 'y': 427.2877}, 'name': 'orange'}, {'id': 'f93ff614-2250-4f4d-8ec0-ac87ec62305c', 'keypoint': {'x': 436.3726, 'y': 437.4057}, 'name': 'orange'}, {'id': '09066da9-88d0-4617-8cf6-7d604ee0190d', 'keypoint': {'x': 485.4057, 'y': 510.566}, 'name': 'orange'}]}
[/INST]
{'x': 366.3255, 'y': 546.3679}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the blue block the is in front of the green block that is closet to you..
 {'width': 756, 'height': 660, 'annotations': [{'id': 'bb2508d6-bab0-4991-a4b4-1d60addd89be', 'keypoint': {'x': 374.8868, 'y': 32.6887}, 'name': 'you'}, {'id': '80bad823-3c11-42a6-9b7a-8f92fd053db8', 'keypoint': {'x': 286.1604, 'y': 504.3396}, 'name': 'yellow'}, {'id': '74407b61-1d02-4456-a3be-f44e6d0621e4', 'keypoint': {'x': 427.8113, 'y': 511.3443}, 'name': 'yellow'}, {'id': '0c043462-b620-40d3-b17e-3f17c7108248', 'keypoint': {'x': 391.2311, 'y': 435.8491}, 'name': 'green'}, {'id': 'e97e7600-f671-4774-970c-bcd8e011471e', 'keypoint': {'x': 311.066, 'y': 530.8019}, 'name': 'green'}, {'id': '3c45bdee-a533-4cab-8e50-cd3e05ea379a', 'keypoint': {'x': 375.6651, 'y': 518.3491}, 'name': 'green'}, {'id': 'fbe906a6-d83a-45bc-bd1e-7c81ccde8b9b', 'keypoint': {'x': 487.7406, 'y': 473.2075}, 'name': 'green'}, {'id': '662455ed-994d-45d8-b864-0f07ccfe9a1e', 'keypoint': {'x': 309.5094, 'y': 477.8774}, 'name': 'blue'}, {'id': '92e619ee-cc43-4ba6-80a0-fd23cabc2b68', 'keypoint': {'x': 378.7783, 'y': 457.6415}, 'name': 'blue'}, {'id': '4d3a4397-37ee-4ea1-bff7-ea4e40e4ba91', 'keypoint': {'x': 395.1226, 'y': 483.3255}, 'name': 'blue'}, {'id': '83c6c716-2497-4e0e-b233-3cda2dca8049', 'keypoint': {'x': 438.7075, 'y': 448.3019}, 'name': 'blue'}, {'id': '24b23d82-8281-40fd-9942-dc852b76fea4', 'keypoint': {'x': 444.1557, 'y': 484.1038}, 'name': 'blue'}, {'id': '541179c1-e37f-4069-99a8-fe58be3053c6', 'keypoint': {'x': 466.7264, 'y': 502.783}, 'name': 'blue'}, {'id': 'd950b6f4-0fff-425b-bf69-419e3a15863f', 'keypoint': {'x': 343.7547, 'y': 543.2547}, 'name': 'orange'}, {'id': 'a5aa3394-3b02-4613-8a7d-f14c9cd0fa58', 'keypoint': {'x': 431.7028, 'y': 539.3632}, 'name': 'orange'}, {'id': 'd7eb7db6-f69d-4df7-861e-0cc9dbebac2a', 'keypoint': {'x': 484.6274, 'y': 532.3585}, 'name': 'orange'}]}
[/INST]
{'x': 378.7783, 'y': 457.6415}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the blue block that is directly in front of the closest green block to you.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'bb2508d6-bab0-4991-a4b4-1d60addd89be', 'keypoint': {'x': 374.8868, 'y': 32.6887}, 'name': 'you'}, {'id': '80bad823-3c11-42a6-9b7a-8f92fd053db8', 'keypoint': {'x': 286.1604, 'y': 504.3396}, 'name': 'yellow'}, {'id': '74407b61-1d02-4456-a3be-f44e6d0621e4', 'keypoint': {'x': 427.8113, 'y': 511.3443}, 'name': 'yellow'}, {'id': '0c043462-b620-40d3-b17e-3f17c7108248', 'keypoint': {'x': 391.2311, 'y': 435.8491}, 'name': 'green'}, {'id': 'e97e7600-f671-4774-970c-bcd8e011471e', 'keypoint': {'x': 311.066, 'y': 530.8019}, 'name': 'green'}, {'id': '3c45bdee-a533-4cab-8e50-cd3e05ea379a', 'keypoint': {'x': 375.6651, 'y': 518.3491}, 'name': 'green'}, {'id': 'fbe906a6-d83a-45bc-bd1e-7c81ccde8b9b', 'keypoint': {'x': 487.7406, 'y': 473.2075}, 'name': 'green'}, {'id': '662455ed-994d-45d8-b864-0f07ccfe9a1e', 'keypoint': {'x': 309.5094, 'y': 477.8774}, 'name': 'blue'}, {'id': '92e619ee-cc43-4ba6-80a0-fd23cabc2b68', 'keypoint': {'x': 378.7783, 'y': 457.6415}, 'name': 'blue'}, {'id': '4d3a4397-37ee-4ea1-bff7-ea4e40e4ba91', 'keypoint': {'x': 395.1226, 'y': 483.3255}, 'name': 'blue'}, {'id': '83c6c716-2497-4e0e-b233-3cda2dca8049', 'keypoint': {'x': 438.7075, 'y': 448.3019}, 'name': 'blue'}, {'id': '24b23d82-8281-40fd-9942-dc852b76fea4', 'keypoint': {'x': 444.1557, 'y': 484.1038}, 'name': 'blue'}, {'id': '541179c1-e37f-4069-99a8-fe58be3053c6', 'keypoint': {'x': 466.7264, 'y': 502.783}, 'name': 'blue'}, {'id': 'd950b6f4-0fff-425b-bf69-419e3a15863f', 'keypoint': {'x': 343.7547, 'y': 543.2547}, 'name': 'orange'}, {'id': 'a5aa3394-3b02-4613-8a7d-f14c9cd0fa58', 'keypoint': {'x': 431.7028, 'y': 539.3632}, 'name': 'orange'}, {'id': 'd7eb7db6-f69d-4df7-861e-0cc9dbebac2a', 'keypoint': {'x': 484.6274, 'y': 532.3585}, 'name': 'orange'}]}
[/INST]
{'x': 378.7783, 'y': 457.6415}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Towards your left, pick the yellow block that is surrounded by green blocks and one blue block.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'fe98e63d-c226-4a25-9e75-229c9a2d828a', 'keypoint': {'x': 377.2217, 'y': 28.7972}, 'name': 'you'}, {'id': 'c1bfc6fe-9e0d-4152-aab0-9efe03f99ee1', 'keypoint': {'x': 291.6085, 'y': 487.217}, 'name': 'yellow'}, {'id': 'eefd5787-08ef-4674-ae9e-c41f4a2ed951', 'keypoint': {'x': 314.1792, 'y': 539.3632}, 'name': 'yellow'}, {'id': '8987e25d-8eb0-42ab-aa8e-390a7b046e44', 'keypoint': {'x': 363.2123, 'y': 544.8113}, 'name': 'yellow'}, {'id': 'af54d764-7469-4b39-95f2-d25ac6241715', 'keypoint': {'x': 434.0377, 'y': 542.4764}, 'name': 'yellow'}, {'id': 'c426887c-43fa-4c7c-9ee6-447f48c5b353', 'keypoint': {'x': 463.6132, 'y': 487.9953}, 'name': 'yellow'}, {'id': 'f1402f90-fe95-42c0-93e5-75019494beba', 'keypoint': {'x': 497.0802, 'y': 442.0755}, 'name': 'yellow'}, {'id': 'ad7b2c3b-efac-4705-8dc4-076514da9f0d', 'keypoint': {'x': 319.6274, 'y': 463.8679}, 'name': 'blue'}, {'id': '57fd11a6-5352-4e90-8c5e-78e080a9f63b', 'keypoint': {'x': 458.1651, 'y': 516.0142}, 'name': 'blue'}, {'id': 'bffb87fb-647e-44e1-b4c0-9bbf464cc4da', 'keypoint': {'x': 353.8726, 'y': 445.1887}, 'name': 'green'}, {'id': '2be751d9-34f0-411b-89e7-1b20b054e1ee', 'keypoint': {'x': 388.8962, 'y': 498.1132}, 'name': 'green'}, {'id': '98a41430-e570-492e-969a-8ea45857d765', 'keypoint': {'x': 395.9009, 'y': 533.1368}, 'name': 'green'}, {'id': '0327cb90-c17c-4528-8964-e978fbff5999', 'keypoint': {'x': 427.8113, 'y': 484.8821}, 'name': 'green'}, {'id': '97f91dd8-6cd8-4194-8783-e9e3b5786e68', 'keypoint': {'x': 456.6085, 'y': 452.9717}, 'name': 'green'}, {'id': '4ebadfd4-3497-4f80-a4e9-a870c25786bd', 'keypoint': {'x': 504.8632, 'y': 502.0047}, 'name': 'green'}, {'id': 'd2ff782c-210c-4090-91a5-de43a78f1232', 'keypoint': {'x': 331.3019, 'y': 512.9009}, 'name': 'orange'}]}
[/INST]
{'x': 463.6132, 'y': 487.9953}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:green block at the very top of the table just above yellow block.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'f8ce6e33-d9e3-4687-bd11-4537937b3def', 'keypoint': {'x': 313.4009, 'y': 480.9906}, 'name': 'orange'}, {'id': '8758b8e0-78a7-4dc2-b595-9eec97d96b3c', 'keypoint': {'x': 403.684, 'y': 466.9811}, 'name': 'orange'}, {'id': '9024f068-b46e-400e-bf69-507f61b74f80', 'keypoint': {'x': 459.7217, 'y': 452.9717}, 'name': 'orange'}, {'id': 'd7ab991d-e3d6-4ffb-8d9b-9cffbb59b9c6', 'keypoint': {'x': 502.5283, 'y': 457.6415}, 'name': 'orange'}, {'id': 'b9fb860f-d2d0-4b4a-a275-416d46a99ab8', 'keypoint': {'x': 335.9717, 'y': 450.6368}, 'name': 'yellow'}, {'id': '7404a736-3ae3-4f5c-b40d-5d5af4635c6e', 'keypoint': {'x': 316.5142, 'y': 531.5802}, 'name': 'yellow'}, {'id': 'd2d6b9ff-fd85-48a1-b368-3f0e1d09cc6f', 'keypoint': {'x': 396.6792, 'y': 491.8868}, 'name': 'yellow'}, {'id': 'c4551bcd-78f8-4205-b828-5b040d49a2c3', 'keypoint': {'x': 444.1557, 'y': 433.5142}, 'name': 'yellow'}, {'id': '3e0cda0a-1e8a-4472-8984-7886c02ff560', 'keypoint': {'x': 347.6462, 'y': 514.4575}, 'name': 'blue'}, {'id': '266b6be1-582b-4c4d-8544-906fad90cbd2', 'keypoint': {'x': 359.3208, 'y': 478.6557}, 'name': 'blue'}, {'id': 'c58796fa-46b4-4231-8589-0612cd52983f', 'keypoint': {'x': 395.9009, 'y': 444.4104}, 'name': 'blue'}, {'id': '3de58fe6-02ae-4e5a-b0eb-ac1e3ac60960', 'keypoint': {'x': 444.934, 'y': 537.8066}, 'name': 'blue'}, {'id': 'a1a96c53-7f31-4121-8983-bf6a69f42935', 'keypoint': {'x': 502.5283, 'y': 541.6981}, 'name': 'blue'}, {'id': 'c2e223f1-9da7-4029-ab29-81edb6be3c94', 'keypoint': {'x': 355.4292, 'y': 424.9528}, 'name': 'green'}, {'id': 'd1f0facf-050a-4a92-bfb1-1367c5fdd80d', 'keypoint': {'x': 466.7264, 'y': 500.4481}, 'name': 'green'}, {'id': '727b2df8-1454-47db-9eff-8cfe072974ff', 'keypoint': {'x': 383.4481, 'y': 44.3632}, 'name': 'you'}]}
[/INST]
{'x': 355.4292, 'y': 424.9528}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the closest orange block to you.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'b1c926ae-92c1-43eb-96c5-76c41d0fab06', 'keypoint': {'x': 342.1981, 'y': 437.4057}, 'name': 'orange'}, {'id': '76267614-95a1-4ded-b4e9-354b821f62bb', 'keypoint': {'x': 314.1792, 'y': 499.6698}, 'name': 'orange'}, {'id': '3124e476-176a-4feb-8844-b3973d48973a', 'keypoint': {'x': 453.4953, 'y': 463.8679}, 'name': 'orange'}, {'id': '728bd5ee-ece2-45c9-b0d7-b887692bb453', 'keypoint': {'x': 494.7453, 'y': 534.6934}, 'name': 'orange'}, {'id': 'da0db0c2-fdee-4550-8e2a-11f1aace764f', 'keypoint': {'x': 376.4434, 'y': 419.5047}, 'name': 'yellow'}, {'id': 'bd27285d-35e5-47b5-9b1e-78bd5fb9e595', 'keypoint': {'x': 360.0991, 'y': 516.7925}, 'name': 'yellow'}, {'id': '5a97f3d8-463a-4c65-9906-b8cf5f772e2b', 'keypoint': {'x': 408.3538, 'y': 526.9104}, 'name': 'yellow'}, {'id': 'f2444027-7c76-40d2-9424-36c46342a49f', 'keypoint': {'x': 318.8491, 'y': 449.0802}, 'name': 'blue'}, {'id': '0b2e3fca-8a71-43d2-a83e-f02af09891b9', 'keypoint': {'x': 315.7358, 'y': 480.2123}, 'name': 'blue'}, {'id': '897e21a9-0e62-42b6-8c01-826ad479026c', 'keypoint': {'x': 378.0, 'y': 490.3302}, 'name': 'blue'}, {'id': 'ca6c0f72-0600-47ba-b4b1-05dc3566a83e', 'keypoint': {'x': 402.1274, 'y': 463.0896}, 'name': 'blue'}, {'id': '41397d83-cab1-4c4d-ba20-b13a1c93f043', 'keypoint': {'x': 419.25, 'y': 432.7358}, 'name': 'blue'}, {'id': '051bcea0-383b-4791-9652-d84f6ef3fe4c', 'keypoint': {'x': 348.4245, 'y': 488.7736}, 'name': 'green'}, {'id': '58f619eb-2384-4a80-97f9-10520e5226a9', 'keypoint': {'x': 311.8443, 'y': 539.3632}, 'name': 'green'}, {'id': 'b9c15fa4-b07d-4f16-9f7d-f007698c83c2', 'keypoint': {'x': 417.6934, 'y': 502.0047}, 'name': 'green'}, {'id': '5c0026dd-e833-4ad7-8339-361bad3686e0', 'keypoint': {'x': 378.0, 'y': 59.1509}, 'name': 'you'}]}
[/INST]
{'x': 342.1981, 'y': 437.4057}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the green block that is below another green block.
 {'width': 756, 'height': 660, 'annotations': [{'id': '8f941e4d-8e6b-4e37-a2cd-f19b3dbb1dda', 'keypoint': {'x': 293.1651, 'y': 452.9717}, 'name': 'orange'}, {'id': '77f8b560-0c54-4923-9c65-8e8973d5cc1a', 'keypoint': {'x': 293.9434, 'y': 485.6604}, 'name': 'orange'}, {'id': 'c2fdf254-12fa-43ee-9d04-fd097d6a2fad', 'keypoint': {'x': 332.8585, 'y': 517.5708}, 'name': 'orange'}, {'id': 'ee022624-310e-4867-8360-cd61e6a5e8ec', 'keypoint': {'x': 325.0755, 'y': 557.2642}, 'name': 'orange'}, {'id': 'ca78545f-1817-447e-a767-1ec186e40025', 'keypoint': {'x': 377.2217, 'y': 478.6557}, 'name': 'orange'}, {'id': 'b7ea891b-3095-4df8-a718-b193f34c851c', 'keypoint': {'x': 386.5613, 'y': 492.6651}, 'name': 'orange'}, {'id': '54b85951-9b7d-4a22-9b94-5856c8f4cc7e', 'keypoint': {'x': 441.8208, 'y': 499.6698}, 'name': 'orange'}, {'id': 'ef2b7cbb-fa7e-4894-8e50-310d43a6f3eb', 'keypoint': {'x': 385.0047, 'y': 448.3019}, 'name': 'yellow'}, {'id': 'ab7d8b08-4eaf-41cf-8066-9aba376bb546', 'keypoint': {'x': 392.0094, 'y': 523.0189}, 'name': 'yellow'}, {'id': 'aec059f3-2def-4458-9ea4-3fa5916cab5b', 'keypoint': {'x': 466.7264, 'y': 442.8538}, 'name': 'yellow'}, {'id': '39fe094d-1e93-4828-ae7e-d672aeb88451', 'keypoint': {'x': 356.2075, 'y': 533.9151}, 'name': 'blue'}, {'id': '57cde523-a685-45a5-b623-1b11057840da', 'keypoint': {'x': 487.7406, 'y': 488.7736}, 'name': 'blue'}, {'id': 'da91e8aa-4087-4689-95ed-80e4ba9a93ec', 'keypoint': {'x': 327.4104, 'y': 472.4292}, 'name': 'green'}, {'id': '9406ceb5-4de6-4915-8da6-132b32721d7c', 'keypoint': {'x': 508.7547, 'y': 511.3443}, 'name': 'green'}, {'id': 'fd93334e-be71-4598-9c11-abb001661b87', 'keypoint': {'x': 497.0802, 'y': 560.3774}, 'name': 'green'}, {'id': '3067ae97-d698-4a21-82f6-e94a33ed51a0', 'keypoint': {'x': 380.3349, 'y': 59.1509}, 'name': 'you'}]}
[/INST]
{'x': 508.7547, 'y': 511.3443}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick the blue block that is right above the orange block
 {'width': 756, 'height': 660, 'annotations': [{'id': 'a7c6f9ab-55b7-422b-8e7f-70902f2e94d6', 'keypoint': {'x': 320.4057, 'y': 488.7736}, 'name': 'orange'}, {'id': 'acc187bd-2783-4707-be6a-314a58518228', 'keypoint': {'x': 423.9198, 'y': 451.4151}, 'name': 'orange'}, {'id': '7b9652c2-a499-48cc-ab07-68e686e885ac', 'keypoint': {'x': 358.5425, 'y': 462.3113}, 'name': 'yellow'}, {'id': '6e4e6633-28b0-4e37-a08b-d90cdb596a0d', 'keypoint': {'x': 334.4151, 'y': 548.7028}, 'name': 'yellow'}, {'id': 'dc28f152-a36a-458c-9f8a-21f4093b66bd', 'keypoint': {'x': 418.4717, 'y': 547.9245}, 'name': 'yellow'}, {'id': '34029874-d3f4-4998-a6fa-85ee314b35b5', 'keypoint': {'x': 479.1792, 'y': 484.8821}, 'name': 'yellow'}, {'id': '3d4114df-2e6e-4d36-8e12-47bbffbd6150', 'keypoint': {'x': 314.1792, 'y': 452.1934}, 'name': 'blue'}, {'id': '021cc2a3-2e83-4079-9777-64d3bc57e829', 'keypoint': {'x': 293.1651, 'y': 557.2642}, 'name': 'blue'}, {'id': '49911b5f-df0a-4694-9ff5-d8165f0f587a', 'keypoint': {'x': 390.4528, 'y': 513.6792}, 'name': 'blue'}, {'id': '95b20075-c6a2-46e0-b203-f8004a3ee4a3', 'keypoint': {'x': 440.2642, 'y': 484.8821}, 'name': 'blue'}, {'id': '031e7d06-9b93-4602-a1ef-c48bce3518c4', 'keypoint': {'x': 462.0566, 'y': 469.316}, 'name': 'blue'}, {'id': '3f2fdd03-f2f9-4892-999b-713f9e988be6', 'keypoint': {'x': 493.1887, 'y': 465.4245}, 'name': 'blue'}, {'id': '7ed0f47f-3bdf-429a-ab98-1d1a9d3058fa', 'keypoint': {'x': 482.2925, 'y': 507.4528}, 'name': 'green'}, {'id': '1fb5251e-aa36-4074-87f5-edef1e5a25c7', 'keypoint': {'x': 491.6321, 'y': 542.4764}, 'name': 'green'}, {'id': 'e7cf6134-82a1-4efe-99a5-a098dcdb3a3e', 'keypoint': {'x': 334.4151, 'y': 516.0142}, 'name': 'green'}, {'id': '0aeb291a-183c-47ce-9552-8b4d4860c05f', 'keypoint': {'x': 376.4434, 'y': 44.3632}, 'name': 'you'}]}
[/INST]
{'x': 440.2642, 'y': 484.8821}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the blue block furthest to the left.
 {'width': 756, 'height': 660, 'annotations': [{'id': '8f941e4d-8e6b-4e37-a2cd-f19b3dbb1dda', 'keypoint': {'x': 293.1651, 'y': 452.9717}, 'name': 'orange'}, {'id': '77f8b560-0c54-4923-9c65-8e8973d5cc1a', 'keypoint': {'x': 293.9434, 'y': 485.6604}, 'name': 'orange'}, {'id': 'c2fdf254-12fa-43ee-9d04-fd097d6a2fad', 'keypoint': {'x': 332.8585, 'y': 517.5708}, 'name': 'orange'}, {'id': 'ee022624-310e-4867-8360-cd61e6a5e8ec', 'keypoint': {'x': 325.0755, 'y': 557.2642}, 'name': 'orange'}, {'id': 'ca78545f-1817-447e-a767-1ec186e40025', 'keypoint': {'x': 377.2217, 'y': 478.6557}, 'name': 'orange'}, {'id': 'b7ea891b-3095-4df8-a718-b193f34c851c', 'keypoint': {'x': 386.5613, 'y': 492.6651}, 'name': 'orange'}, {'id': '54b85951-9b7d-4a22-9b94-5856c8f4cc7e', 'keypoint': {'x': 441.8208, 'y': 499.6698}, 'name': 'orange'}, {'id': 'ef2b7cbb-fa7e-4894-8e50-310d43a6f3eb', 'keypoint': {'x': 385.0047, 'y': 448.3019}, 'name': 'yellow'}, {'id': 'ab7d8b08-4eaf-41cf-8066-9aba376bb546', 'keypoint': {'x': 392.0094, 'y': 523.0189}, 'name': 'yellow'}, {'id': 'aec059f3-2def-4458-9ea4-3fa5916cab5b', 'keypoint': {'x': 466.7264, 'y': 442.8538}, 'name': 'yellow'}, {'id': '39fe094d-1e93-4828-ae7e-d672aeb88451', 'keypoint': {'x': 356.2075, 'y': 533.9151}, 'name': 'blue'}, {'id': '57cde523-a685-45a5-b623-1b11057840da', 'keypoint': {'x': 487.7406, 'y': 488.7736}, 'name': 'blue'}, {'id': 'da91e8aa-4087-4689-95ed-80e4ba9a93ec', 'keypoint': {'x': 327.4104, 'y': 472.4292}, 'name': 'green'}, {'id': '9406ceb5-4de6-4915-8da6-132b32721d7c', 'keypoint': {'x': 508.7547, 'y': 511.3443}, 'name': 'green'}, {'id': 'fd93334e-be71-4598-9c11-abb001661b87', 'keypoint': {'x': 497.0802, 'y': 560.3774}, 'name': 'green'}, {'id': '3067ae97-d698-4a21-82f6-e94a33ed51a0', 'keypoint': {'x': 380.3349, 'y': 59.1509}, 'name': 'you'}]}
[/INST]
{'x': 487.7406, 'y': 488.7736}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the green block that is farther from you than all the other green blocks.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'cad61a18-85b8-4f6e-93b3-c3ce9e28cda2', 'keypoint': {'x': 316.5142, 'y': 554.9292}, 'name': 'orange'}, {'id': '17682384-25d2-46d1-ae87-f04077458497', 'keypoint': {'x': 330.5236, 'y': 516.7925}, 'name': 'orange'}, {'id': '5d36ae41-004b-494e-b32d-33279b1dbfee', 'keypoint': {'x': 447.2689, 'y': 457.6415}, 'name': 'orange'}, {'id': 'fee58518-97d8-4c98-9b1b-ebeaa7ef3967', 'keypoint': {'x': 488.5189, 'y': 530.8019}, 'name': 'orange'}, {'id': '4fa5b6b4-f98b-4588-95e6-8255a4bef84b', 'keypoint': {'x': 380.3349, 'y': 446.7453}, 'name': 'yellow'}, {'id': '22f0cd3c-99be-40d9-9b3f-a7bc782da1f8', 'keypoint': {'x': 297.8349, 'y': 529.2453}, 'name': 'yellow'}, {'id': 'd5f859c2-a998-4ee4-bff1-d6025abd4883', 'keypoint': {'x': 375.6651, 'y': 536.25}, 'name': 'yellow'}, {'id': '96691580-8803-4b4b-b31f-4920d402e03e', 'keypoint': {'x': 473.7311, 'y': 435.0708}, 'name': 'yellow'}, {'id': '86653055-abe8-4a82-8b79-fb7f461438c0', 'keypoint': {'x': 474.5094, 'y': 487.217}, 'name': 'yellow'}, {'id': '34e55929-db77-4549-8f50-5adc4da9bd5d', 'keypoint': {'x': 370.9953, 'y': 492.6651}, 'name': 'blue'}, {'id': '500fbe66-22dd-4552-a43c-63bf2efeb1c9', 'keypoint': {'x': 417.6934, 'y': 477.0991}, 'name': 'blue'}, {'id': '85cb36b6-c443-488a-85b6-363a2284870a', 'keypoint': {'x': 345.3113, 'y': 424.9528}, 'name': 'green'}, {'id': '1c037117-1e04-4cd7-a544-c3e10b3639bb', 'keypoint': {'x': 295.5, 'y': 466.9811}, 'name': 'green'}, {'id': 'a8f28480-33ff-4a0a-883f-f1f226ced914', 'keypoint': {'x': 299.3915, 'y': 501.2264}, 'name': 'green'}, {'id': 'b9578593-4dcd-4808-9119-c6439b45d587', 'keypoint': {'x': 427.033, 'y': 534.6934}, 'name': 'green'}, {'id': '70024c2f-19b2-412a-a76d-d9e741635b7e', 'keypoint': {'x': 380.3349, 'y': 33.467}, 'name': 'you'}]}
[/INST]
{'x': 427.033, 'y': 534.6934}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the yellow block that is the farthest away from you.
 {'width': 756, 'height': 660, 'annotations': [{'id': '82770085-a20a-439c-b1a3-f74a8d8ccdd0', 'keypoint': {'x': 322.0293, 'y': 491.9856}, 'name': 'yellow'}, {'id': '975d2226-07a6-46ec-8aa1-01f44597bb5c', 'keypoint': {'x': 376.8281, 'y': 495.1624}, 'name': 'yellow'}, {'id': '9fbd0dd7-292a-46c1-854e-eb8c4bb880cf', 'keypoint': {'x': 402.2421, 'y': 551.5495}, 'name': 'yellow'}, {'id': 'c2d61b6b-0585-4aef-acaf-17bad430192b', 'keypoint': {'x': 464.9827, 'y': 498.3391}, 'name': 'yellow'}, {'id': '13d3414e-a81b-4dad-8963-6fbda2f0d436', 'keypoint': {'x': 376.0339, 'y': 27.387}, 'name': 'you'}, {'id': '9b80a1cd-fed1-49e9-8b69-3cd0088f7fca', 'keypoint': {'x': 302.9689, 'y': 454.6589}, 'name': 'blue'}, {'id': '17a9c4b0-bf5e-45d6-aa6f-94959ad755e2', 'keypoint': {'x': 421.3025, 'y': 473.7194}, 'name': 'blue'}, {'id': '4a1722fe-5d45-4bf9-a201-c0b8f7acc36b', 'keypoint': {'x': 456.2467, 'y': 437.981}, 'name': 'blue'}, {'id': '680351fd-2723-4f71-9103-17bb8a809738', 'keypoint': {'x': 375.2398, 'y': 452.2763}, 'name': 'orange'}, {'id': '188969f7-812f-47f7-a97e-5a0652f78c64', 'keypoint': {'x': 354.5909, 'y': 508.6635}, 'name': 'orange'}, {'id': 'b57e9350-23ed-40aa-96cc-2811f14cd796', 'keypoint': {'x': 396.6828, 'y': 520.5763}, 'name': 'orange'}, {'id': 'f041b6b6-0396-42fe-99f5-46de7b8e6b23', 'keypoint': {'x': 431.6269, 'y': 507.8693}, 'name': 'orange'}, {'id': 'ae76b953-f8b0-4121-9569-ce66605c9783', 'keypoint': {'x': 433.2153, 'y': 530.9007}, 'name': 'orange'}, {'id': '256e0a15-32e1-445f-a768-a8adb00d4905', 'keypoint': {'x': 341.0898, 'y': 438.7752}, 'name': 'green'}, {'id': '51ccd976-c149-458d-960f-bd00da3a5131', 'keypoint': {'x': 318.0584, 'y': 536.46}, 'name': 'green'}, {'id': 'd3e5315b-8e96-4659-97cb-07d656a65db8', 'keypoint': {'x': 503.8978, 'y': 456.2473}, 'name': 'green'}]}
[/INST]
{'x': 402.2421, 'y': 551.5495}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the blue block that's between two other blue blocks and near a brown block.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'a7c6f9ab-55b7-422b-8e7f-70902f2e94d6', 'keypoint': {'x': 320.4057, 'y': 488.7736}, 'name': 'orange'}, {'id': 'acc187bd-2783-4707-be6a-314a58518228', 'keypoint': {'x': 423.9198, 'y': 451.4151}, 'name': 'orange'}, {'id': '7b9652c2-a499-48cc-ab07-68e686e885ac', 'keypoint': {'x': 358.5425, 'y': 462.3113}, 'name': 'yellow'}, {'id': '6e4e6633-28b0-4e37-a08b-d90cdb596a0d', 'keypoint': {'x': 334.4151, 'y': 548.7028}, 'name': 'yellow'}, {'id': 'dc28f152-a36a-458c-9f8a-21f4093b66bd', 'keypoint': {'x': 418.4717, 'y': 547.9245}, 'name': 'yellow'}, {'id': '34029874-d3f4-4998-a6fa-85ee314b35b5', 'keypoint': {'x': 479.1792, 'y': 484.8821}, 'name': 'yellow'}, {'id': '3d4114df-2e6e-4d36-8e12-47bbffbd6150', 'keypoint': {'x': 314.1792, 'y': 452.1934}, 'name': 'blue'}, {'id': '021cc2a3-2e83-4079-9777-64d3bc57e829', 'keypoint': {'x': 293.1651, 'y': 557.2642}, 'name': 'blue'}, {'id': '49911b5f-df0a-4694-9ff5-d8165f0f587a', 'keypoint': {'x': 390.4528, 'y': 513.6792}, 'name': 'blue'}, {'id': '95b20075-c6a2-46e0-b203-f8004a3ee4a3', 'keypoint': {'x': 440.2642, 'y': 484.8821}, 'name': 'blue'}, {'id': '031e7d06-9b93-4602-a1ef-c48bce3518c4', 'keypoint': {'x': 462.0566, 'y': 469.316}, 'name': 'blue'}, {'id': '3f2fdd03-f2f9-4892-999b-713f9e988be6', 'keypoint': {'x': 493.1887, 'y': 465.4245}, 'name': 'blue'}, {'id': '7ed0f47f-3bdf-429a-ab98-1d1a9d3058fa', 'keypoint': {'x': 482.2925, 'y': 507.4528}, 'name': 'green'}, {'id': '1fb5251e-aa36-4074-87f5-edef1e5a25c7', 'keypoint': {'x': 491.6321, 'y': 542.4764}, 'name': 'green'}, {'id': 'e7cf6134-82a1-4efe-99a5-a098dcdb3a3e', 'keypoint': {'x': 334.4151, 'y': 516.0142}, 'name': 'green'}, {'id': '0aeb291a-183c-47ce-9552-8b4d4860c05f', 'keypoint': {'x': 376.4434, 'y': 44.3632}, 'name': 'you'}]}
[/INST]
{'x': 462.0566, 'y': 469.316}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the nearest blue block.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'fe98e63d-c226-4a25-9e75-229c9a2d828a', 'keypoint': {'x': 377.2217, 'y': 28.7972}, 'name': 'you'}, {'id': 'c1bfc6fe-9e0d-4152-aab0-9efe03f99ee1', 'keypoint': {'x': 291.6085, 'y': 487.217}, 'name': 'yellow'}, {'id': 'eefd5787-08ef-4674-ae9e-c41f4a2ed951', 'keypoint': {'x': 314.1792, 'y': 539.3632}, 'name': 'yellow'}, {'id': '8987e25d-8eb0-42ab-aa8e-390a7b046e44', 'keypoint': {'x': 363.2123, 'y': 544.8113}, 'name': 'yellow'}, {'id': 'af54d764-7469-4b39-95f2-d25ac6241715', 'keypoint': {'x': 434.0377, 'y': 542.4764}, 'name': 'yellow'}, {'id': 'c426887c-43fa-4c7c-9ee6-447f48c5b353', 'keypoint': {'x': 463.6132, 'y': 487.9953}, 'name': 'yellow'}, {'id': 'f1402f90-fe95-42c0-93e5-75019494beba', 'keypoint': {'x': 497.0802, 'y': 442.0755}, 'name': 'yellow'}, {'id': 'ad7b2c3b-efac-4705-8dc4-076514da9f0d', 'keypoint': {'x': 319.6274, 'y': 463.8679}, 'name': 'blue'}, {'id': '57fd11a6-5352-4e90-8c5e-78e080a9f63b', 'keypoint': {'x': 458.1651, 'y': 516.0142}, 'name': 'blue'}, {'id': 'bffb87fb-647e-44e1-b4c0-9bbf464cc4da', 'keypoint': {'x': 353.8726, 'y': 445.1887}, 'name': 'green'}, {'id': '2be751d9-34f0-411b-89e7-1b20b054e1ee', 'keypoint': {'x': 388.8962, 'y': 498.1132}, 'name': 'green'}, {'id': '98a41430-e570-492e-969a-8ea45857d765', 'keypoint': {'x': 395.9009, 'y': 533.1368}, 'name': 'green'}, {'id': '0327cb90-c17c-4528-8964-e978fbff5999', 'keypoint': {'x': 427.8113, 'y': 484.8821}, 'name': 'green'}, {'id': '97f91dd8-6cd8-4194-8783-e9e3b5786e68', 'keypoint': {'x': 456.6085, 'y': 452.9717}, 'name': 'green'}, {'id': '4ebadfd4-3497-4f80-a4e9-a870c25786bd', 'keypoint': {'x': 504.8632, 'y': 502.0047}, 'name': 'green'}, {'id': 'd2ff782c-210c-4090-91a5-de43a78f1232', 'keypoint': {'x': 331.3019, 'y': 512.9009}, 'name': 'orange'}]}
[/INST]
{'x': 319.6274, 'y': 463.8679}</s>