<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the blue block that is next to another blue block and green block, and above an orange block.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'c7519096-f7ea-4ab1-a66d-97c708976599', 'keypoint': {'x': 380.3349, 'y': 34.2453}, 'name': 'you'}, {'id': 'a95968ee-51d4-482d-ae79-97cb54663aed', 'keypoint': {'x': 293.9434, 'y': 479.434}, 'name': 'blue'}, {'id': '2fc8b299-6074-43eb-92bc-7ed758a958fc', 'keypoint': {'x': 329.7453, 'y': 557.2642}, 'name': 'blue'}, {'id': '11dbc045-de4f-4d3a-bc50-a1c2ba279bd1', 'keypoint': {'x': 366.3255, 'y': 546.3679}, 'name': 'blue'}, {'id': 'a3036737-fb7e-4f86-88c8-b0c5262cc051', 'keypoint': {'x': 379.5566, 'y': 438.9623}, 'name': 'blue'}, {'id': '5370f3f3-ef52-45e8-bdb4-3d96d54f4bd9', 'keypoint': {'x': 455.8302, 'y': 449.0802}, 'name': 'blue'}, {'id': '95294165-9354-41e7-b1be-60d4033bd1a3', 'keypoint': {'x': 318.0708, 'y': 454.5283}, 'name': 'yellow'}, {'id': '71f49ac9-1905-457f-9219-701cac7c42e4', 'keypoint': {'x': 318.0708, 'y': 530.8019}, 'name': 'yellow'}, {'id': '6f7a8a19-794f-4ffc-80ed-8a42dedeab0f', 'keypoint': {'x': 463.6132, 'y': 524.5755}, 'name': 'yellow'}, {'id': '407068c6-6a1e-4a14-b9d0-c0a4ed13b9c2', 'keypoint': {'x': 395.1226, 'y': 466.2028}, 'name': 'green'}, {'id': '847fb0f9-02cc-42df-8984-2b193e2abb8d', 'keypoint': {'x': 403.684, 'y': 523.0189}, 'name': 'green'}, {'id': 'bd78738b-2c39-4b85-9575-960d46ecd66c', 'keypoint': {'x': 325.0755, 'y': 500.4481}, 'name': 'orange'}, {'id': 'dafb3b80-4c1e-48a7-b398-e3aa56c5dcb0', 'keypoint': {'x': 374.1085, 'y': 515.2358}, 'name': 'orange'}, {'id': '36cf3f26-cac5-4368-91eb-ae8937ec2f91', 'keypoint': {'x': 412.2453, 'y': 427.2877}, 'name': 'orange'}, {'id': 'f93ff614-2250-4f4d-8ec0-ac87ec62305c', 'keypoint': {'x': 436.3726, 'y': 437.4057}, 'name': 'orange'}, {'id': '09066da9-88d0-4617-8cf6-7d604ee0190d', 'keypoint': {'x': 485.4057, 'y': 510.566}, 'name': 'orange'}]}
[/INST]
{'x': 366.3255, 'y': 546.3679}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Do you see the orange block towards your right? There is a yellow block behind it. Do not grab that block but the other yellow one right next to it.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'b328ac50-08b5-486e-a937-2cddbc04245c', 'keypoint': {'x': 304.8396, 'y': 434.2925}, 'name': 'orange'}, {'id': '7344b92c-5683-4670-9cef-33f4cf8e73fb', 'keypoint': {'x': 472.9528, 'y': 456.0849}, 'name': 'orange'}, {'id': '6958bbb7-3dce-4cfe-9049-02ebc5a32e43', 'keypoint': {'x': 306.3962, 'y': 460.7547}, 'name': 'yellow'}, {'id': '62285f29-522c-498a-9fdb-53e2e8421ea5', 'keypoint': {'x': 330.5236, 'y': 470.8726}, 'name': 'yellow'}, {'id': 'e4402e53-f01e-4254-a874-f6ffd3892ac4', 'keypoint': {'x': 381.8915, 'y': 475.5425}, 'name': 'yellow'}, {'id': '3f3ed2fa-5312-418f-9f29-f4750a72aa9e', 'keypoint': {'x': 402.9057, 'y': 435.0708}, 'name': 'yellow'}, {'id': 'fe5ba8f4-1f70-4712-ae25-cd20ac9670d6', 'keypoint': {'x': 455.8302, 'y': 481.7689}, 'name': 'yellow'}, {'id': 'bd1c4ddb-8345-4f24-a982-f6cd6a64fb0e', 'keypoint': {'x': 501.75, 'y': 505.8962}, 'name': 'yellow'}, {'id': '8f332be0-ad10-4130-9840-f3a1dc330890', 'keypoint': {'x': 352.316, 'y': 513.6792}, 'name': 'yellow'}, {'id': '5b19adbf-7efb-4a90-9f40-4c0bf2f3b9a8', 'keypoint': {'x': 324.2972, 'y': 499.6698}, 'name': 'yellow'}, {'id': '7f124547-f8b7-4280-b7e1-401876ce2bdb', 'keypoint': {'x': 308.7311, 'y': 552.5943}, 'name': 'yellow'}, {'id': '8a1fd773-bc7c-4be5-a292-c1c53d5c77df', 'keypoint': {'x': 298.6132, 'y': 491.8868}, 'name': 'blue'}, {'id': '7be8714d-676d-466e-a70a-e9d078b757f4', 'keypoint': {'x': 381.1132, 'y': 497.3349}, 'name': 'blue'}, {'id': 'b1ee1721-f1d2-4cc8-828b-2e6c5072d54f', 'keypoint': {'x': 451.9387, 'y': 432.7358}, 'name': 'blue'}, {'id': '0416eebd-320e-40d8-bbc7-a94a6333aa95', 'keypoint': {'x': 497.0802, 'y': 485.6604}, 'name': 'green'}, {'id': '2240b1ab-d515-4872-a587-09ca836dc397', 'keypoint': {'x': 378.7783, 'y': 28.7972}, 'name': 'you'}]}
[/INST]
{'x': 330.5236, 'y': 470.8726}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:To your left, there is a group of three blue blocks between a yellow block and an orange block: pick up the middle blue block.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'a7c6f9ab-55b7-422b-8e7f-70902f2e94d6', 'keypoint': {'x': 320.4057, 'y': 488.7736}, 'name': 'orange'}, {'id': 'acc187bd-2783-4707-be6a-314a58518228', 'keypoint': {'x': 423.9198, 'y': 451.4151}, 'name': 'orange'}, {'id': '7b9652c2-a499-48cc-ab07-68e686e885ac', 'keypoint': {'x': 358.5425, 'y': 462.3113}, 'name': 'yellow'}, {'id': '6e4e6633-28b0-4e37-a08b-d90cdb596a0d', 'keypoint': {'x': 334.4151, 'y': 548.7028}, 'name': 'yellow'}, {'id': 'dc28f152-a36a-458c-9f8a-21f4093b66bd', 'keypoint': {'x': 418.4717, 'y': 547.9245}, 'name': 'yellow'}, {'id': '34029874-d3f4-4998-a6fa-85ee314b35b5', 'keypoint': {'x': 479.1792, 'y': 484.8821}, 'name': 'yellow'}, {'id': '3d4114df-2e6e-4d36-8e12-47bbffbd6150', 'keypoint': {'x': 314.1792, 'y': 452.1934}, 'name': 'blue'}, {'id': '021cc2a3-2e83-4079-9777-64d3bc57e829', 'keypoint': {'x': 293.1651, 'y': 557.2642}, 'name': 'blue'}, {'id': '49911b5f-df0a-4694-9ff5-d8165f0f587a', 'keypoint': {'x': 390.4528, 'y': 513.6792}, 'name': 'blue'}, {'id': '95b20075-c6a2-46e0-b203-f8004a3ee4a3', 'keypoint': {'x': 440.2642, 'y': 484.8821}, 'name': 'blue'}, {'id': '031e7d06-9b93-4602-a1ef-c48bce3518c4', 'keypoint': {'x': 462.0566, 'y': 469.316}, 'name': 'blue'}, {'id': '3f2fdd03-f2f9-4892-999b-713f9e988be6', 'keypoint': {'x': 493.1887, 'y': 465.4245}, 'name': 'blue'}, {'id': '7ed0f47f-3bdf-429a-ab98-1d1a9d3058fa', 'keypoint': {'x': 482.2925, 'y': 507.4528}, 'name': 'green'}, {'id': '1fb5251e-aa36-4074-87f5-edef1e5a25c7', 'keypoint': {'x': 491.6321, 'y': 542.4764}, 'name': 'green'}, {'id': 'e7cf6134-82a1-4efe-99a5-a098dcdb3a3e', 'keypoint': {'x': 334.4151, 'y': 516.0142}, 'name': 'green'}, {'id': '0aeb291a-183c-47ce-9552-8b4d4860c05f', 'keypoint': {'x': 376.4434, 'y': 44.3632}, 'name': 'you'}]}
[/INST]
{'x': 462.0566, 'y': 469.316}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the green block that is directly below two orange blocks.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'd5bcc2b7-ff91-4686-8f50-3a73ffd3beeb', 'keypoint': {'x': 315.7358, 'y': 436.6274}, 'name': 'orange'}, {'id': '70515b86-98d5-4fa6-9118-a0bc484f6de1', 'keypoint': {'x': 313.4009, 'y': 525.3538}, 'name': 'orange'}, {'id': 'b40b0958-f706-45d2-a042-0440eb67121b', 'keypoint': {'x': 380.3349, 'y': 515.2358}, 'name': 'orange'}, {'id': '9dcd051f-bad9-4f84-b653-9a30a72cb545', 'keypoint': {'x': 389.6745, 'y': 552.5943}, 'name': 'orange'}, {'id': 'b0de6329-f4d9-486c-91ed-478f28262cff', 'keypoint': {'x': 363.9906, 'y': 439.7406}, 'name': 'yellow'}, {'id': '10eeedc5-3253-42e7-b401-33c4616e4623', 'keypoint': {'x': 396.6792, 'y': 457.6415}, 'name': 'yellow'}, {'id': 'f7642eee-44ca-4f1d-abf7-a79d43dfbd0d', 'keypoint': {'x': 279.934, 'y': 535.4717}, 'name': 'yellow'}, {'id': 'd87c603d-89da-4c0b-8f61-a496275c1297', 'keypoint': {'x': 311.066, 'y': 487.217}, 'name': 'blue'}, {'id': '2065d0e4-12eb-4ab3-bde4-75688bb3c142', 'keypoint': {'x': 457.3868, 'y': 439.7406}, 'name': 'green'}, {'id': '1550df58-31c4-49f4-8e6d-7db4a57bc294', 'keypoint': {'x': 435.5943, 'y': 445.967}, 'name': 'green'}, {'id': '1990bf50-38ac-43ab-8581-26778cea80ed', 'keypoint': {'x': 458.9434, 'y': 484.8821}, 'name': 'green'}, {'id': '7895d85c-2e57-4008-b9d7-74625608a42c', 'keypoint': {'x': 485.4057, 'y': 515.2358}, 'name': 'green'}, {'id': '991a571a-03aa-4a00-b80d-3d6a0845625b', 'keypoint': {'x': 440.2642, 'y': 514.4575}, 'name': 'green'}, {'id': 'e0e1086c-3a47-42f6-98fd-868d0756cd4d', 'keypoint': {'x': 378.7783, 'y': 495.7783}, 'name': 'green'}, {'id': '3d132917-dd96-4ae9-9302-da261696357f', 'keypoint': {'x': 333.6368, 'y': 505.1179}, 'name': 'green'}, {'id': 'bc7494f1-b049-49c6-bf24-85e942969065', 'keypoint': {'x': 383.4481, 'y': 56.0377}, 'name': 'you'}]}
[/INST]
{'x': 378.7783, 'y': 495.7783}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:the blue block on your right hand side
 {'width': 756, 'height': 660, 'annotations': [{'id': 'fe98e63d-c226-4a25-9e75-229c9a2d828a', 'keypoint': {'x': 377.2217, 'y': 28.7972}, 'name': 'you'}, {'id': 'c1bfc6fe-9e0d-4152-aab0-9efe03f99ee1', 'keypoint': {'x': 291.6085, 'y': 487.217}, 'name': 'yellow'}, {'id': 'eefd5787-08ef-4674-ae9e-c41f4a2ed951', 'keypoint': {'x': 314.1792, 'y': 539.3632}, 'name': 'yellow'}, {'id': '8987e25d-8eb0-42ab-aa8e-390a7b046e44', 'keypoint': {'x': 363.2123, 'y': 544.8113}, 'name': 'yellow'}, {'id': 'af54d764-7469-4b39-95f2-d25ac6241715', 'keypoint': {'x': 434.0377, 'y': 542.4764}, 'name': 'yellow'}, {'id': 'c426887c-43fa-4c7c-9ee6-447f48c5b353', 'keypoint': {'x': 463.6132, 'y': 487.9953}, 'name': 'yellow'}, {'id': 'f1402f90-fe95-42c0-93e5-75019494beba', 'keypoint': {'x': 497.0802, 'y': 442.0755}, 'name': 'yellow'}, {'id': 'ad7b2c3b-efac-4705-8dc4-076514da9f0d', 'keypoint': {'x': 319.6274, 'y': 463.8679}, 'name': 'blue'}, {'id': '57fd11a6-5352-4e90-8c5e-78e080a9f63b', 'keypoint': {'x': 458.1651, 'y': 516.0142}, 'name': 'blue'}, {'id': 'bffb87fb-647e-44e1-b4c0-9bbf464cc4da', 'keypoint': {'x': 353.8726, 'y': 445.1887}, 'name': 'green'}, {'id': '2be751d9-34f0-411b-89e7-1b20b054e1ee', 'keypoint': {'x': 388.8962, 'y': 498.1132}, 'name': 'green'}, {'id': '98a41430-e570-492e-969a-8ea45857d765', 'keypoint': {'x': 395.9009, 'y': 533.1368}, 'name': 'green'}, {'id': '0327cb90-c17c-4528-8964-e978fbff5999', 'keypoint': {'x': 427.8113, 'y': 484.8821}, 'name': 'green'}, {'id': '97f91dd8-6cd8-4194-8783-e9e3b5786e68', 'keypoint': {'x': 456.6085, 'y': 452.9717}, 'name': 'green'}, {'id': '4ebadfd4-3497-4f80-a4e9-a870c25786bd', 'keypoint': {'x': 504.8632, 'y': 502.0047}, 'name': 'green'}, {'id': 'd2ff782c-210c-4090-91a5-de43a78f1232', 'keypoint': {'x': 331.3019, 'y': 512.9009}, 'name': 'orange'}]}
[/INST]
{'x': 319.6274, 'y': 463.8679}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the orange block that is the farthest away from you on your left side.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'b1c926ae-92c1-43eb-96c5-76c41d0fab06', 'keypoint': {'x': 342.1981, 'y': 437.4057}, 'name': 'orange'}, {'id': '76267614-95a1-4ded-b4e9-354b821f62bb', 'keypoint': {'x': 314.1792, 'y': 499.6698}, 'name': 'orange'}, {'id': '3124e476-176a-4feb-8844-b3973d48973a', 'keypoint': {'x': 453.4953, 'y': 463.8679}, 'name': 'orange'}, {'id': '728bd5ee-ece2-45c9-b0d7-b887692bb453', 'keypoint': {'x': 494.7453, 'y': 534.6934}, 'name': 'orange'}, {'id': 'da0db0c2-fdee-4550-8e2a-11f1aace764f', 'keypoint': {'x': 376.4434, 'y': 419.5047}, 'name': 'yellow'}, {'id': 'bd27285d-35e5-47b5-9b1e-78bd5fb9e595', 'keypoint': {'x': 360.0991, 'y': 516.7925}, 'name': 'yellow'}, {'id': '5a97f3d8-463a-4c65-9906-b8cf5f772e2b', 'keypoint': {'x': 408.3538, 'y': 526.9104}, 'name': 'yellow'}, {'id': 'f2444027-7c76-40d2-9424-36c46342a49f', 'keypoint': {'x': 318.8491, 'y': 449.0802}, 'name': 'blue'}, {'id': '0b2e3fca-8a71-43d2-a83e-f02af09891b9', 'keypoint': {'x': 315.7358, 'y': 480.2123}, 'name': 'blue'}, {'id': '897e21a9-0e62-42b6-8c01-826ad479026c', 'keypoint': {'x': 378.0, 'y': 490.3302}, 'name': 'blue'}, {'id': 'ca6c0f72-0600-47ba-b4b1-05dc3566a83e', 'keypoint': {'x': 402.1274, 'y': 463.0896}, 'name': 'blue'}, {'id': '41397d83-cab1-4c4d-ba20-b13a1c93f043', 'keypoint': {'x': 419.25, 'y': 432.7358}, 'name': 'blue'}, {'id': '051bcea0-383b-4791-9652-d84f6ef3fe4c', 'keypoint': {'x': 348.4245, 'y': 488.7736}, 'name': 'green'}, {'id': '58f619eb-2384-4a80-97f9-10520e5226a9', 'keypoint': {'x': 311.8443, 'y': 539.3632}, 'name': 'green'}, {'id': 'b9c15fa4-b07d-4f16-9f7d-f007698c83c2', 'keypoint': {'x': 417.6934, 'y': 502.0047}, 'name': 'green'}, {'id': '5c0026dd-e833-4ad7-8339-361bad3686e0', 'keypoint': {'x': 378.0, 'y': 59.1509}, 'name': 'you'}]}
[/INST]
{'x': 494.7453, 'y': 534.6934}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:pick up the most northern yellow block
 {'width': 756, 'height': 660, 'annotations': [{'id': '82770085-a20a-439c-b1a3-f74a8d8ccdd0', 'keypoint': {'x': 322.0293, 'y': 491.9856}, 'name': 'yellow'}, {'id': '975d2226-07a6-46ec-8aa1-01f44597bb5c', 'keypoint': {'x': 376.8281, 'y': 495.1624}, 'name': 'yellow'}, {'id': '9fbd0dd7-292a-46c1-854e-eb8c4bb880cf', 'keypoint': {'x': 402.2421, 'y': 551.5495}, 'name': 'yellow'}, {'id': 'c2d61b6b-0585-4aef-acaf-17bad430192b', 'keypoint': {'x': 464.9827, 'y': 498.3391}, 'name': 'yellow'}, {'id': '13d3414e-a81b-4dad-8963-6fbda2f0d436', 'keypoint': {'x': 376.0339, 'y': 27.387}, 'name': 'you'}, {'id': '9b80a1cd-fed1-49e9-8b69-3cd0088f7fca', 'keypoint': {'x': 302.9689, 'y': 454.6589}, 'name': 'blue'}, {'id': '17a9c4b0-bf5e-45d6-aa6f-94959ad755e2', 'keypoint': {'x': 421.3025, 'y': 473.7194}, 'name': 'blue'}, {'id': '4a1722fe-5d45-4bf9-a201-c0b8f7acc36b', 'keypoint': {'x': 456.2467, 'y': 437.981}, 'name': 'blue'}, {'id': '680351fd-2723-4f71-9103-17bb8a809738', 'keypoint': {'x': 375.2398, 'y': 452.2763}, 'name': 'orange'}, {'id': '188969f7-812f-47f7-a97e-5a0652f78c64', 'keypoint': {'x': 354.5909, 'y': 508.6635}, 'name': 'orange'}, {'id': 'b57e9350-23ed-40aa-96cc-2811f14cd796', 'keypoint': {'x': 396.6828, 'y': 520.5763}, 'name': 'orange'}, {'id': 'f041b6b6-0396-42fe-99f5-46de7b8e6b23', 'keypoint': {'x': 431.6269, 'y': 507.8693}, 'name': 'orange'}, {'id': 'ae76b953-f8b0-4121-9569-ce66605c9783', 'keypoint': {'x': 433.2153, 'y': 530.9007}, 'name': 'orange'}, {'id': '256e0a15-32e1-445f-a768-a8adb00d4905', 'keypoint': {'x': 341.0898, 'y': 438.7752}, 'name': 'green'}, {'id': '51ccd976-c149-458d-960f-bd00da3a5131', 'keypoint': {'x': 318.0584, 'y': 536.46}, 'name': 'green'}, {'id': 'd3e5315b-8e96-4659-97cb-07d656a65db8', 'keypoint': {'x': 503.8978, 'y': 456.2473}, 'name': 'green'}]}
[/INST]
{'x': 402.2421, 'y': 551.5495}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the orange block that is set apart from the rest of the blocks. It is to your left side and close to me.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'b1c926ae-92c1-43eb-96c5-76c41d0fab06', 'keypoint': {'x': 342.1981, 'y': 437.4057}, 'name': 'orange'}, {'id': '76267614-95a1-4ded-b4e9-354b821f62bb', 'keypoint': {'x': 314.1792, 'y': 499.6698}, 'name': 'orange'}, {'id': '3124e476-176a-4feb-8844-b3973d48973a', 'keypoint': {'x': 453.4953, 'y': 463.8679}, 'name': 'orange'}, {'id': '728bd5ee-ece2-45c9-b0d7-b887692bb453', 'keypoint': {'x': 494.7453, 'y': 534.6934}, 'name': 'orange'}, {'id': 'da0db0c2-fdee-4550-8e2a-11f1aace764f', 'keypoint': {'x': 376.4434, 'y': 419.5047}, 'name': 'yellow'}, {'id': 'bd27285d-35e5-47b5-9b1e-78bd5fb9e595', 'keypoint': {'x': 360.0991, 'y': 516.7925}, 'name': 'yellow'}, {'id': '5a97f3d8-463a-4c65-9906-b8cf5f772e2b', 'keypoint': {'x': 408.3538, 'y': 526.9104}, 'name': 'yellow'}, {'id': 'f2444027-7c76-40d2-9424-36c46342a49f', 'keypoint': {'x': 318.8491, 'y': 449.0802}, 'name': 'blue'}, {'id': '0b2e3fca-8a71-43d2-a83e-f02af09891b9', 'keypoint': {'x': 315.7358, 'y': 480.2123}, 'name': 'blue'}, {'id': '897e21a9-0e62-42b6-8c01-826ad479026c', 'keypoint': {'x': 378.0, 'y': 490.3302}, 'name': 'blue'}, {'id': 'ca6c0f72-0600-47ba-b4b1-05dc3566a83e', 'keypoint': {'x': 402.1274, 'y': 463.0896}, 'name': 'blue'}, {'id': '41397d83-cab1-4c4d-ba20-b13a1c93f043', 'keypoint': {'x': 419.25, 'y': 432.7358}, 'name': 'blue'}, {'id': '051bcea0-383b-4791-9652-d84f6ef3fe4c', 'keypoint': {'x': 348.4245, 'y': 488.7736}, 'name': 'green'}, {'id': '58f619eb-2384-4a80-97f9-10520e5226a9', 'keypoint': {'x': 311.8443, 'y': 539.3632}, 'name': 'green'}, {'id': 'b9c15fa4-b07d-4f16-9f7d-f007698c83c2', 'keypoint': {'x': 417.6934, 'y': 502.0047}, 'name': 'green'}, {'id': '5c0026dd-e833-4ad7-8339-361bad3686e0', 'keypoint': {'x': 378.0, 'y': 59.1509}, 'name': 'you'}]}
[/INST]
{'x': 494.7453, 'y': 534.6934}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the green block closest to you.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'f8ce6e33-d9e3-4687-bd11-4537937b3def', 'keypoint': {'x': 313.4009, 'y': 480.9906}, 'name': 'orange'}, {'id': '8758b8e0-78a7-4dc2-b595-9eec97d96b3c', 'keypoint': {'x': 403.684, 'y': 466.9811}, 'name': 'orange'}, {'id': '9024f068-b46e-400e-bf69-507f61b74f80', 'keypoint': {'x': 459.7217, 'y': 452.9717}, 'name': 'orange'}, {'id': 'd7ab991d-e3d6-4ffb-8d9b-9cffbb59b9c6', 'keypoint': {'x': 502.5283, 'y': 457.6415}, 'name': 'orange'}, {'id': 'b9fb860f-d2d0-4b4a-a275-416d46a99ab8', 'keypoint': {'x': 335.9717, 'y': 450.6368}, 'name': 'yellow'}, {'id': '7404a736-3ae3-4f5c-b40d-5d5af4635c6e', 'keypoint': {'x': 316.5142, 'y': 531.5802}, 'name': 'yellow'}, {'id': 'd2d6b9ff-fd85-48a1-b368-3f0e1d09cc6f', 'keypoint': {'x': 396.6792, 'y': 491.8868}, 'name': 'yellow'}, {'id': 'c4551bcd-78f8-4205-b828-5b040d49a2c3', 'keypoint': {'x': 444.1557, 'y': 433.5142}, 'name': 'yellow'}, {'id': '3e0cda0a-1e8a-4472-8984-7886c02ff560', 'keypoint': {'x': 347.6462, 'y': 514.4575}, 'name': 'blue'}, {'id': '266b6be1-582b-4c4d-8544-906fad90cbd2', 'keypoint': {'x': 359.3208, 'y': 478.6557}, 'name': 'blue'}, {'id': 'c58796fa-46b4-4231-8589-0612cd52983f', 'keypoint': {'x': 395.9009, 'y': 444.4104}, 'name': 'blue'}, {'id': '3de58fe6-02ae-4e5a-b0eb-ac1e3ac60960', 'keypoint': {'x': 444.934, 'y': 537.8066}, 'name': 'blue'}, {'id': 'a1a96c53-7f31-4121-8983-bf6a69f42935', 'keypoint': {'x': 502.5283, 'y': 541.6981}, 'name': 'blue'}, {'id': 'c2e223f1-9da7-4029-ab29-81edb6be3c94', 'keypoint': {'x': 355.4292, 'y': 424.9528}, 'name': 'green'}, {'id': 'd1f0facf-050a-4a92-bfb1-1367c5fdd80d', 'keypoint': {'x': 466.7264, 'y': 500.4481}, 'name': 'green'}, {'id': '727b2df8-1454-47db-9eff-8cfe072974ff', 'keypoint': {'x': 383.4481, 'y': 44.3632}, 'name': 'you'}]}
[/INST]
{'x': 355.4292, 'y': 424.9528}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the green block nearest to you.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'cad61a18-85b8-4f6e-93b3-c3ce9e28cda2', 'keypoint': {'x': 316.5142, 'y': 554.9292}, 'name': 'orange'}, {'id': '17682384-25d2-46d1-ae87-f04077458497', 'keypoint': {'x': 330.5236, 'y': 516.7925}, 'name': 'orange'}, {'id': '5d36ae41-004b-494e-b32d-33279b1dbfee', 'keypoint': {'x': 447.2689, 'y': 457.6415}, 'name': 'orange'}, {'id': 'fee58518-97d8-4c98-9b1b-ebeaa7ef3967', 'keypoint': {'x': 488.5189, 'y': 530.8019}, 'name': 'orange'}, {'id': '4fa5b6b4-f98b-4588-95e6-8255a4bef84b', 'keypoint': {'x': 380.3349, 'y': 446.7453}, 'name': 'yellow'}, {'id': '22f0cd3c-99be-40d9-9b3f-a7bc782da1f8', 'keypoint': {'x': 297.8349, 'y': 529.2453}, 'name': 'yellow'}, {'id': 'd5f859c2-a998-4ee4-bff1-d6025abd4883', 'keypoint': {'x': 375.6651, 'y': 536.25}, 'name': 'yellow'}, {'id': '96691580-8803-4b4b-b31f-4920d402e03e', 'keypoint': {'x': 473.7311, 'y': 435.0708}, 'name': 'yellow'}, {'id': '86653055-abe8-4a82-8b79-fb7f461438c0', 'keypoint': {'x': 474.5094, 'y': 487.217}, 'name': 'yellow'}, {'id': '34e55929-db77-4549-8f50-5adc4da9bd5d', 'keypoint': {'x': 370.9953, 'y': 492.6651}, 'name': 'blue'}, {'id': '500fbe66-22dd-4552-a43c-63bf2efeb1c9', 'keypoint': {'x': 417.6934, 'y': 477.0991}, 'name': 'blue'}, {'id': '85cb36b6-c443-488a-85b6-363a2284870a', 'keypoint': {'x': 345.3113, 'y': 424.9528}, 'name': 'green'}, {'id': '1c037117-1e04-4cd7-a544-c3e10b3639bb', 'keypoint': {'x': 295.5, 'y': 466.9811}, 'name': 'green'}, {'id': 'a8f28480-33ff-4a0a-883f-f1f226ced914', 'keypoint': {'x': 299.3915, 'y': 501.2264}, 'name': 'green'}, {'id': 'b9578593-4dcd-4808-9119-c6439b45d587', 'keypoint': {'x': 427.033, 'y': 534.6934}, 'name': 'green'}, {'id': '70024c2f-19b2-412a-a76d-d9e741635b7e', 'keypoint': {'x': 380.3349, 'y': 33.467}, 'name': 'you'}]}
[/INST]
{'x': 345.3113, 'y': 424.9528}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:There are two blue blocks near the top of the table - pick up the one that is NOT furthest away. It should be to the left of the other blue block.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'c7519096-f7ea-4ab1-a66d-97c708976599', 'keypoint': {'x': 380.3349, 'y': 34.2453}, 'name': 'you'}, {'id': 'a95968ee-51d4-482d-ae79-97cb54663aed', 'keypoint': {'x': 293.9434, 'y': 479.434}, 'name': 'blue'}, {'id': '2fc8b299-6074-43eb-92bc-7ed758a958fc', 'keypoint': {'x': 329.7453, 'y': 557.2642}, 'name': 'blue'}, {'id': '11dbc045-de4f-4d3a-bc50-a1c2ba279bd1', 'keypoint': {'x': 366.3255, 'y': 546.3679}, 'name': 'blue'}, {'id': 'a3036737-fb7e-4f86-88c8-b0c5262cc051', 'keypoint': {'x': 379.5566, 'y': 438.9623}, 'name': 'blue'}, {'id': '5370f3f3-ef52-45e8-bdb4-3d96d54f4bd9', 'keypoint': {'x': 455.8302, 'y': 449.0802}, 'name': 'blue'}, {'id': '95294165-9354-41e7-b1be-60d4033bd1a3', 'keypoint': {'x': 318.0708, 'y': 454.5283}, 'name': 'yellow'}, {'id': '71f49ac9-1905-457f-9219-701cac7c42e4', 'keypoint': {'x': 318.0708, 'y': 530.8019}, 'name': 'yellow'}, {'id': '6f7a8a19-794f-4ffc-80ed-8a42dedeab0f', 'keypoint': {'x': 463.6132, 'y': 524.5755}, 'name': 'yellow'}, {'id': '407068c6-6a1e-4a14-b9d0-c0a4ed13b9c2', 'keypoint': {'x': 395.1226, 'y': 466.2028}, 'name': 'green'}, {'id': '847fb0f9-02cc-42df-8984-2b193e2abb8d', 'keypoint': {'x': 403.684, 'y': 523.0189}, 'name': 'green'}, {'id': 'bd78738b-2c39-4b85-9575-960d46ecd66c', 'keypoint': {'x': 325.0755, 'y': 500.4481}, 'name': 'orange'}, {'id': 'dafb3b80-4c1e-48a7-b398-e3aa56c5dcb0', 'keypoint': {'x': 374.1085, 'y': 515.2358}, 'name': 'orange'}, {'id': '36cf3f26-cac5-4368-91eb-ae8937ec2f91', 'keypoint': {'x': 412.2453, 'y': 427.2877}, 'name': 'orange'}, {'id': 'f93ff614-2250-4f4d-8ec0-ac87ec62305c', 'keypoint': {'x': 436.3726, 'y': 437.4057}, 'name': 'orange'}, {'id': '09066da9-88d0-4617-8cf6-7d604ee0190d', 'keypoint': {'x': 485.4057, 'y': 510.566}, 'name': 'orange'}]}
[/INST]
{'x': 366.3255, 'y': 546.3679}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the orange block that is the furthermost away from you.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'b1c926ae-92c1-43eb-96c5-76c41d0fab06', 'keypoint': {'x': 342.1981, 'y': 437.4057}, 'name': 'orange'}, {'id': '76267614-95a1-4ded-b4e9-354b821f62bb', 'keypoint': {'x': 314.1792, 'y': 499.6698}, 'name': 'orange'}, {'id': '3124e476-176a-4feb-8844-b3973d48973a', 'keypoint': {'x': 453.4953, 'y': 463.8679}, 'name': 'orange'}, {'id': '728bd5ee-ece2-45c9-b0d7-b887692bb453', 'keypoint': {'x': 494.7453, 'y': 534.6934}, 'name': 'orange'}, {'id': 'da0db0c2-fdee-4550-8e2a-11f1aace764f', 'keypoint': {'x': 376.4434, 'y': 419.5047}, 'name': 'yellow'}, {'id': 'bd27285d-35e5-47b5-9b1e-78bd5fb9e595', 'keypoint': {'x': 360.0991, 'y': 516.7925}, 'name': 'yellow'}, {'id': '5a97f3d8-463a-4c65-9906-b8cf5f772e2b', 'keypoint': {'x': 408.3538, 'y': 526.9104}, 'name': 'yellow'}, {'id': 'f2444027-7c76-40d2-9424-36c46342a49f', 'keypoint': {'x': 318.8491, 'y': 449.0802}, 'name': 'blue'}, {'id': '0b2e3fca-8a71-43d2-a83e-f02af09891b9', 'keypoint': {'x': 315.7358, 'y': 480.2123}, 'name': 'blue'}, {'id': '897e21a9-0e62-42b6-8c01-826ad479026c', 'keypoint': {'x': 378.0, 'y': 490.3302}, 'name': 'blue'}, {'id': 'ca6c0f72-0600-47ba-b4b1-05dc3566a83e', 'keypoint': {'x': 402.1274, 'y': 463.0896}, 'name': 'blue'}, {'id': '41397d83-cab1-4c4d-ba20-b13a1c93f043', 'keypoint': {'x': 419.25, 'y': 432.7358}, 'name': 'blue'}, {'id': '051bcea0-383b-4791-9652-d84f6ef3fe4c', 'keypoint': {'x': 348.4245, 'y': 488.7736}, 'name': 'green'}, {'id': '58f619eb-2384-4a80-97f9-10520e5226a9', 'keypoint': {'x': 311.8443, 'y': 539.3632}, 'name': 'green'}, {'id': 'b9c15fa4-b07d-4f16-9f7d-f007698c83c2', 'keypoint': {'x': 417.6934, 'y': 502.0047}, 'name': 'green'}, {'id': '5c0026dd-e833-4ad7-8339-361bad3686e0', 'keypoint': {'x': 378.0, 'y': 59.1509}, 'name': 'you'}]}
[/INST]
{'x': 494.7453, 'y': 534.6934}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick the blue block that is farthest from you and right above the orange block
 {'width': 756, 'height': 660, 'annotations': [{'id': 'c7519096-f7ea-4ab1-a66d-97c708976599', 'keypoint': {'x': 380.3349, 'y': 34.2453}, 'name': 'you'}, {'id': 'a95968ee-51d4-482d-ae79-97cb54663aed', 'keypoint': {'x': 293.9434, 'y': 479.434}, 'name': 'blue'}, {'id': '2fc8b299-6074-43eb-92bc-7ed758a958fc', 'keypoint': {'x': 329.7453, 'y': 557.2642}, 'name': 'blue'}, {'id': '11dbc045-de4f-4d3a-bc50-a1c2ba279bd1', 'keypoint': {'x': 366.3255, 'y': 546.3679}, 'name': 'blue'}, {'id': 'a3036737-fb7e-4f86-88c8-b0c5262cc051', 'keypoint': {'x': 379.5566, 'y': 438.9623}, 'name': 'blue'}, {'id': '5370f3f3-ef52-45e8-bdb4-3d96d54f4bd9', 'keypoint': {'x': 455.8302, 'y': 449.0802}, 'name': 'blue'}, {'id': '95294165-9354-41e7-b1be-60d4033bd1a3', 'keypoint': {'x': 318.0708, 'y': 454.5283}, 'name': 'yellow'}, {'id': '71f49ac9-1905-457f-9219-701cac7c42e4', 'keypoint': {'x': 318.0708, 'y': 530.8019}, 'name': 'yellow'}, {'id': '6f7a8a19-794f-4ffc-80ed-8a42dedeab0f', 'keypoint': {'x': 463.6132, 'y': 524.5755}, 'name': 'yellow'}, {'id': '407068c6-6a1e-4a14-b9d0-c0a4ed13b9c2', 'keypoint': {'x': 395.1226, 'y': 466.2028}, 'name': 'green'}, {'id': '847fb0f9-02cc-42df-8984-2b193e2abb8d', 'keypoint': {'x': 403.684, 'y': 523.0189}, 'name': 'green'}, {'id': 'bd78738b-2c39-4b85-9575-960d46ecd66c', 'keypoint': {'x': 325.0755, 'y': 500.4481}, 'name': 'orange'}, {'id': 'dafb3b80-4c1e-48a7-b398-e3aa56c5dcb0', 'keypoint': {'x': 374.1085, 'y': 515.2358}, 'name': 'orange'}, {'id': '36cf3f26-cac5-4368-91eb-ae8937ec2f91', 'keypoint': {'x': 412.2453, 'y': 427.2877}, 'name': 'orange'}, {'id': 'f93ff614-2250-4f4d-8ec0-ac87ec62305c', 'keypoint': {'x': 436.3726, 'y': 437.4057}, 'name': 'orange'}, {'id': '09066da9-88d0-4617-8cf6-7d604ee0190d', 'keypoint': {'x': 485.4057, 'y': 510.566}, 'name': 'orange'}]}
[/INST]
{'x': 366.3255, 'y': 546.3679}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the first green block next to a blue block that is completely to your left.
 {'width': 756, 'height': 660, 'annotations': [{'id': '8f941e4d-8e6b-4e37-a2cd-f19b3dbb1dda', 'keypoint': {'x': 293.1651, 'y': 452.9717}, 'name': 'orange'}, {'id': '77f8b560-0c54-4923-9c65-8e8973d5cc1a', 'keypoint': {'x': 293.9434, 'y': 485.6604}, 'name': 'orange'}, {'id': 'c2fdf254-12fa-43ee-9d04-fd097d6a2fad', 'keypoint': {'x': 332.8585, 'y': 517.5708}, 'name': 'orange'}, {'id': 'ee022624-310e-4867-8360-cd61e6a5e8ec', 'keypoint': {'x': 325.0755, 'y': 557.2642}, 'name': 'orange'}, {'id': 'ca78545f-1817-447e-a767-1ec186e40025', 'keypoint': {'x': 377.2217, 'y': 478.6557}, 'name': 'orange'}, {'id': 'b7ea891b-3095-4df8-a718-b193f34c851c', 'keypoint': {'x': 386.5613, 'y': 492.6651}, 'name': 'orange'}, {'id': '54b85951-9b7d-4a22-9b94-5856c8f4cc7e', 'keypoint': {'x': 441.8208, 'y': 499.6698}, 'name': 'orange'}, {'id': 'ef2b7cbb-fa7e-4894-8e50-310d43a6f3eb', 'keypoint': {'x': 385.0047, 'y': 448.3019}, 'name': 'yellow'}, {'id': 'ab7d8b08-4eaf-41cf-8066-9aba376bb546', 'keypoint': {'x': 392.0094, 'y': 523.0189}, 'name': 'yellow'}, {'id': 'aec059f3-2def-4458-9ea4-3fa5916cab5b', 'keypoint': {'x': 466.7264, 'y': 442.8538}, 'name': 'yellow'}, {'id': '39fe094d-1e93-4828-ae7e-d672aeb88451', 'keypoint': {'x': 356.2075, 'y': 533.9151}, 'name': 'blue'}, {'id': '57cde523-a685-45a5-b623-1b11057840da', 'keypoint': {'x': 487.7406, 'y': 488.7736}, 'name': 'blue'}, {'id': 'da91e8aa-4087-4689-95ed-80e4ba9a93ec', 'keypoint': {'x': 327.4104, 'y': 472.4292}, 'name': 'green'}, {'id': '9406ceb5-4de6-4915-8da6-132b32721d7c', 'keypoint': {'x': 508.7547, 'y': 511.3443}, 'name': 'green'}, {'id': 'fd93334e-be71-4598-9c11-abb001661b87', 'keypoint': {'x': 497.0802, 'y': 560.3774}, 'name': 'green'}, {'id': '3067ae97-d698-4a21-82f6-e94a33ed51a0', 'keypoint': {'x': 380.3349, 'y': 59.1509}, 'name': 'you'}]}
[/INST]
{'x': 508.7547, 'y': 511.3443}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the blue block second from your left.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'a7c6f9ab-55b7-422b-8e7f-70902f2e94d6', 'keypoint': {'x': 320.4057, 'y': 488.7736}, 'name': 'orange'}, {'id': 'acc187bd-2783-4707-be6a-314a58518228', 'keypoint': {'x': 423.9198, 'y': 451.4151}, 'name': 'orange'}, {'id': '7b9652c2-a499-48cc-ab07-68e686e885ac', 'keypoint': {'x': 358.5425, 'y': 462.3113}, 'name': 'yellow'}, {'id': '6e4e6633-28b0-4e37-a08b-d90cdb596a0d', 'keypoint': {'x': 334.4151, 'y': 548.7028}, 'name': 'yellow'}, {'id': 'dc28f152-a36a-458c-9f8a-21f4093b66bd', 'keypoint': {'x': 418.4717, 'y': 547.9245}, 'name': 'yellow'}, {'id': '34029874-d3f4-4998-a6fa-85ee314b35b5', 'keypoint': {'x': 479.1792, 'y': 484.8821}, 'name': 'yellow'}, {'id': '3d4114df-2e6e-4d36-8e12-47bbffbd6150', 'keypoint': {'x': 314.1792, 'y': 452.1934}, 'name': 'blue'}, {'id': '021cc2a3-2e83-4079-9777-64d3bc57e829', 'keypoint': {'x': 293.1651, 'y': 557.2642}, 'name': 'blue'}, {'id': '49911b5f-df0a-4694-9ff5-d8165f0f587a', 'keypoint': {'x': 390.4528, 'y': 513.6792}, 'name': 'blue'}, {'id': '95b20075-c6a2-46e0-b203-f8004a3ee4a3', 'keypoint': {'x': 440.2642, 'y': 484.8821}, 'name': 'blue'}, {'id': '031e7d06-9b93-4602-a1ef-c48bce3518c4', 'keypoint': {'x': 462.0566, 'y': 469.316}, 'name': 'blue'}, {'id': '3f2fdd03-f2f9-4892-999b-713f9e988be6', 'keypoint': {'x': 493.1887, 'y': 465.4245}, 'name': 'blue'}, {'id': '7ed0f47f-3bdf-429a-ab98-1d1a9d3058fa', 'keypoint': {'x': 482.2925, 'y': 507.4528}, 'name': 'green'}, {'id': '1fb5251e-aa36-4074-87f5-edef1e5a25c7', 'keypoint': {'x': 491.6321, 'y': 542.4764}, 'name': 'green'}, {'id': 'e7cf6134-82a1-4efe-99a5-a098dcdb3a3e', 'keypoint': {'x': 334.4151, 'y': 516.0142}, 'name': 'green'}, {'id': '0aeb291a-183c-47ce-9552-8b4d4860c05f', 'keypoint': {'x': 376.4434, 'y': 44.3632}, 'name': 'you'}]}
[/INST]
{'x': 462.0566, 'y': 469.316}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the green block that is closest to you.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'cad61a18-85b8-4f6e-93b3-c3ce9e28cda2', 'keypoint': {'x': 316.5142, 'y': 554.9292}, 'name': 'orange'}, {'id': '17682384-25d2-46d1-ae87-f04077458497', 'keypoint': {'x': 330.5236, 'y': 516.7925}, 'name': 'orange'}, {'id': '5d36ae41-004b-494e-b32d-33279b1dbfee', 'keypoint': {'x': 447.2689, 'y': 457.6415}, 'name': 'orange'}, {'id': 'fee58518-97d8-4c98-9b1b-ebeaa7ef3967', 'keypoint': {'x': 488.5189, 'y': 530.8019}, 'name': 'orange'}, {'id': '4fa5b6b4-f98b-4588-95e6-8255a4bef84b', 'keypoint': {'x': 380.3349, 'y': 446.7453}, 'name': 'yellow'}, {'id': '22f0cd3c-99be-40d9-9b3f-a7bc782da1f8', 'keypoint': {'x': 297.8349, 'y': 529.2453}, 'name': 'yellow'}, {'id': 'd5f859c2-a998-4ee4-bff1-d6025abd4883', 'keypoint': {'x': 375.6651, 'y': 536.25}, 'name': 'yellow'}, {'id': '96691580-8803-4b4b-b31f-4920d402e03e', 'keypoint': {'x': 473.7311, 'y': 435.0708}, 'name': 'yellow'}, {'id': '86653055-abe8-4a82-8b79-fb7f461438c0', 'keypoint': {'x': 474.5094, 'y': 487.217}, 'name': 'yellow'}, {'id': '34e55929-db77-4549-8f50-5adc4da9bd5d', 'keypoint': {'x': 370.9953, 'y': 492.6651}, 'name': 'blue'}, {'id': '500fbe66-22dd-4552-a43c-63bf2efeb1c9', 'keypoint': {'x': 417.6934, 'y': 477.0991}, 'name': 'blue'}, {'id': '85cb36b6-c443-488a-85b6-363a2284870a', 'keypoint': {'x': 345.3113, 'y': 424.9528}, 'name': 'green'}, {'id': '1c037117-1e04-4cd7-a544-c3e10b3639bb', 'keypoint': {'x': 295.5, 'y': 466.9811}, 'name': 'green'}, {'id': 'a8f28480-33ff-4a0a-883f-f1f226ced914', 'keypoint': {'x': 299.3915, 'y': 501.2264}, 'name': 'green'}, {'id': 'b9578593-4dcd-4808-9119-c6439b45d587', 'keypoint': {'x': 427.033, 'y': 534.6934}, 'name': 'green'}, {'id': '70024c2f-19b2-412a-a76d-d9e741635b7e', 'keypoint': {'x': 380.3349, 'y': 33.467}, 'name': 'you'}]}
[/INST]
{'x': 345.3113, 'y': 424.9528}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the blue block furthest from you, on your left.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'c7519096-f7ea-4ab1-a66d-97c708976599', 'keypoint': {'x': 380.3349, 'y': 34.2453}, 'name': 'you'}, {'id': 'a95968ee-51d4-482d-ae79-97cb54663aed', 'keypoint': {'x': 293.9434, 'y': 479.434}, 'name': 'blue'}, {'id': '2fc8b299-6074-43eb-92bc-7ed758a958fc', 'keypoint': {'x': 329.7453, 'y': 557.2642}, 'name': 'blue'}, {'id': '11dbc045-de4f-4d3a-bc50-a1c2ba279bd1', 'keypoint': {'x': 366.3255, 'y': 546.3679}, 'name': 'blue'}, {'id': 'a3036737-fb7e-4f86-88c8-b0c5262cc051', 'keypoint': {'x': 379.5566, 'y': 438.9623}, 'name': 'blue'}, {'id': '5370f3f3-ef52-45e8-bdb4-3d96d54f4bd9', 'keypoint': {'x': 455.8302, 'y': 449.0802}, 'name': 'blue'}, {'id': '95294165-9354-41e7-b1be-60d4033bd1a3', 'keypoint': {'x': 318.0708, 'y': 454.5283}, 'name': 'yellow'}, {'id': '71f49ac9-1905-457f-9219-701cac7c42e4', 'keypoint': {'x': 318.0708, 'y': 530.8019}, 'name': 'yellow'}, {'id': '6f7a8a19-794f-4ffc-80ed-8a42dedeab0f', 'keypoint': {'x': 463.6132, 'y': 524.5755}, 'name': 'yellow'}, {'id': '407068c6-6a1e-4a14-b9d0-c0a4ed13b9c2', 'keypoint': {'x': 395.1226, 'y': 466.2028}, 'name': 'green'}, {'id': '847fb0f9-02cc-42df-8984-2b193e2abb8d', 'keypoint': {'x': 403.684, 'y': 523.0189}, 'name': 'green'}, {'id': 'bd78738b-2c39-4b85-9575-960d46ecd66c', 'keypoint': {'x': 325.0755, 'y': 500.4481}, 'name': 'orange'}, {'id': 'dafb3b80-4c1e-48a7-b398-e3aa56c5dcb0', 'keypoint': {'x': 374.1085, 'y': 515.2358}, 'name': 'orange'}, {'id': '36cf3f26-cac5-4368-91eb-ae8937ec2f91', 'keypoint': {'x': 412.2453, 'y': 427.2877}, 'name': 'orange'}, {'id': 'f93ff614-2250-4f4d-8ec0-ac87ec62305c', 'keypoint': {'x': 436.3726, 'y': 437.4057}, 'name': 'orange'}, {'id': '09066da9-88d0-4617-8cf6-7d604ee0190d', 'keypoint': {'x': 485.4057, 'y': 510.566}, 'name': 'orange'}]}
[/INST]
{'x': 366.3255, 'y': 546.3679}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the yellow block that is exactly 3rd in order of distance from you
 {'width': 756, 'height': 660, 'annotations': [{'id': '53de559f-b2c6-46fa-821d-b5a4e98c67c3', 'keypoint': {'x': 332.0802, 'y': 517.5708}, 'name': 'orange'}, {'id': 'fc5cb394-ebdb-4064-bb12-1d3da5b0cb6f', 'keypoint': {'x': 316.5142, 'y': 540.1415}, 'name': 'orange'}, {'id': 'b8f5d815-90be-4cd5-951f-44207215b594', 'keypoint': {'x': 373.3302, 'y': 537.8066}, 'name': 'orange'}, {'id': '78d88ded-7431-4260-a864-725e2dfef6ab', 'keypoint': {'x': 309.5094, 'y': 498.1132}, 'name': 'yellow'}, {'id': 'e5215008-d111-4bda-9009-f2c0795ffb0d', 'keypoint': {'x': 388.1179, 'y': 471.6509}, 'name': 'yellow'}, {'id': 'b722e1f1-0a3c-47ef-b35f-e9cd85898169', 'keypoint': {'x': 388.1179, 'y': 501.2264}, 'name': 'yellow'}, {'id': '27ad7266-481b-40f9-aaa0-e71dabd71d5e', 'keypoint': {'x': 413.8019, 'y': 491.1085}, 'name': 'yellow'}, {'id': '71348b8e-08dc-4a76-be0e-68cbee1a56a4', 'keypoint': {'x': 416.1368, 'y': 449.8585}, 'name': 'yellow'}, {'id': 'e46d6178-4284-433d-a81f-bc6cbbf7cdbb', 'keypoint': {'x': 490.8538, 'y': 505.1179}, 'name': 'yellow'}, {'id': '272a554a-aad9-4bba-9062-941465beeaa5', 'keypoint': {'x': 465.1698, 'y': 554.1509}, 'name': 'yellow'}, {'id': 'c6ba259b-1c8e-43d4-8bdb-404594f1c175', 'keypoint': {'x': 310.2877, 'y': 438.9623}, 'name': 'blue'}, {'id': '693a6454-810f-447d-8417-a2c57867901d', 'keypoint': {'x': 354.6509, 'y': 477.8774}, 'name': 'blue'}, {'id': '3ec7fa7f-c792-4c99-a423-b40bbb665d89', 'keypoint': {'x': 427.8113, 'y': 428.8443}, 'name': 'blue'}, {'id': '3e041f2e-9e78-4001-9fa0-3bebee80ac07', 'keypoint': {'x': 440.2642, 'y': 498.8915}, 'name': 'green'}, {'id': 'e2e59b33-f4e7-47cd-b137-77a874c834c7', 'keypoint': {'x': 450.3821, 'y': 519.1274}, 'name': 'green'}, {'id': '707959b0-76f6-463a-bc2f-62cd702815f4', 'keypoint': {'x': 377.2217, 'y': 55.2594}, 'name': 'you'}]}
[/INST]
{'x': 413.8019, 'y': 491.1085}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Narrow search to just orange blocks. Look for the orange block that is closest to you. Pick it up.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'b1c926ae-92c1-43eb-96c5-76c41d0fab06', 'keypoint': {'x': 342.1981, 'y': 437.4057}, 'name': 'orange'}, {'id': '76267614-95a1-4ded-b4e9-354b821f62bb', 'keypoint': {'x': 314.1792, 'y': 499.6698}, 'name': 'orange'}, {'id': '3124e476-176a-4feb-8844-b3973d48973a', 'keypoint': {'x': 453.4953, 'y': 463.8679}, 'name': 'orange'}, {'id': '728bd5ee-ece2-45c9-b0d7-b887692bb453', 'keypoint': {'x': 494.7453, 'y': 534.6934}, 'name': 'orange'}, {'id': 'da0db0c2-fdee-4550-8e2a-11f1aace764f', 'keypoint': {'x': 376.4434, 'y': 419.5047}, 'name': 'yellow'}, {'id': 'bd27285d-35e5-47b5-9b1e-78bd5fb9e595', 'keypoint': {'x': 360.0991, 'y': 516.7925}, 'name': 'yellow'}, {'id': '5a97f3d8-463a-4c65-9906-b8cf5f772e2b', 'keypoint': {'x': 408.3538, 'y': 526.9104}, 'name': 'yellow'}, {'id': 'f2444027-7c76-40d2-9424-36c46342a49f', 'keypoint': {'x': 318.8491, 'y': 449.0802}, 'name': 'blue'}, {'id': '0b2e3fca-8a71-43d2-a83e-f02af09891b9', 'keypoint': {'x': 315.7358, 'y': 480.2123}, 'name': 'blue'}, {'id': '897e21a9-0e62-42b6-8c01-826ad479026c', 'keypoint': {'x': 378.0, 'y': 490.3302}, 'name': 'blue'}, {'id': 'ca6c0f72-0600-47ba-b4b1-05dc3566a83e', 'keypoint': {'x': 402.1274, 'y': 463.0896}, 'name': 'blue'}, {'id': '41397d83-cab1-4c4d-ba20-b13a1c93f043', 'keypoint': {'x': 419.25, 'y': 432.7358}, 'name': 'blue'}, {'id': '051bcea0-383b-4791-9652-d84f6ef3fe4c', 'keypoint': {'x': 348.4245, 'y': 488.7736}, 'name': 'green'}, {'id': '58f619eb-2384-4a80-97f9-10520e5226a9', 'keypoint': {'x': 311.8443, 'y': 539.3632}, 'name': 'green'}, {'id': 'b9c15fa4-b07d-4f16-9f7d-f007698c83c2', 'keypoint': {'x': 417.6934, 'y': 502.0047}, 'name': 'green'}, {'id': '5c0026dd-e833-4ad7-8339-361bad3686e0', 'keypoint': {'x': 378.0, 'y': 59.1509}, 'name': 'you'}]}
[/INST]
{'x': 342.1981, 'y': 437.4057}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the blue block directly to the left of the orange block at the bottom left side of the table.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'a7c6f9ab-55b7-422b-8e7f-70902f2e94d6', 'keypoint': {'x': 320.4057, 'y': 488.7736}, 'name': 'orange'}, {'id': 'acc187bd-2783-4707-be6a-314a58518228', 'keypoint': {'x': 423.9198, 'y': 451.4151}, 'name': 'orange'}, {'id': '7b9652c2-a499-48cc-ab07-68e686e885ac', 'keypoint': {'x': 358.5425, 'y': 462.3113}, 'name': 'yellow'}, {'id': '6e4e6633-28b0-4e37-a08b-d90cdb596a0d', 'keypoint': {'x': 334.4151, 'y': 548.7028}, 'name': 'yellow'}, {'id': 'dc28f152-a36a-458c-9f8a-21f4093b66bd', 'keypoint': {'x': 418.4717, 'y': 547.9245}, 'name': 'yellow'}, {'id': '34029874-d3f4-4998-a6fa-85ee314b35b5', 'keypoint': {'x': 479.1792, 'y': 484.8821}, 'name': 'yellow'}, {'id': '3d4114df-2e6e-4d36-8e12-47bbffbd6150', 'keypoint': {'x': 314.1792, 'y': 452.1934}, 'name': 'blue'}, {'id': '021cc2a3-2e83-4079-9777-64d3bc57e829', 'keypoint': {'x': 293.1651, 'y': 557.2642}, 'name': 'blue'}, {'id': '49911b5f-df0a-4694-9ff5-d8165f0f587a', 'keypoint': {'x': 390.4528, 'y': 513.6792}, 'name': 'blue'}, {'id': '95b20075-c6a2-46e0-b203-f8004a3ee4a3', 'keypoint': {'x': 440.2642, 'y': 484.8821}, 'name': 'blue'}, {'id': '031e7d06-9b93-4602-a1ef-c48bce3518c4', 'keypoint': {'x': 462.0566, 'y': 469.316}, 'name': 'blue'}, {'id': '3f2fdd03-f2f9-4892-999b-713f9e988be6', 'keypoint': {'x': 493.1887, 'y': 465.4245}, 'name': 'blue'}, {'id': '7ed0f47f-3bdf-429a-ab98-1d1a9d3058fa', 'keypoint': {'x': 482.2925, 'y': 507.4528}, 'name': 'green'}, {'id': '1fb5251e-aa36-4074-87f5-edef1e5a25c7', 'keypoint': {'x': 491.6321, 'y': 542.4764}, 'name': 'green'}, {'id': 'e7cf6134-82a1-4efe-99a5-a098dcdb3a3e', 'keypoint': {'x': 334.4151, 'y': 516.0142}, 'name': 'green'}, {'id': '0aeb291a-183c-47ce-9552-8b4d4860c05f', 'keypoint': {'x': 376.4434, 'y': 44.3632}, 'name': 'you'}]}
[/INST]
{'x': 462.0566, 'y': 469.316}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:pick up the green block on your right but not the furthest right, pick up the one next to the far right one.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'd5bcc2b7-ff91-4686-8f50-3a73ffd3beeb', 'keypoint': {'x': 315.7358, 'y': 436.6274}, 'name': 'orange'}, {'id': '70515b86-98d5-4fa6-9118-a0bc484f6de1', 'keypoint': {'x': 313.4009, 'y': 525.3538}, 'name': 'orange'}, {'id': 'b40b0958-f706-45d2-a042-0440eb67121b', 'keypoint': {'x': 380.3349, 'y': 515.2358}, 'name': 'orange'}, {'id': '9dcd051f-bad9-4f84-b653-9a30a72cb545', 'keypoint': {'x': 389.6745, 'y': 552.5943}, 'name': 'orange'}, {'id': 'b0de6329-f4d9-486c-91ed-478f28262cff', 'keypoint': {'x': 363.9906, 'y': 439.7406}, 'name': 'yellow'}, {'id': '10eeedc5-3253-42e7-b401-33c4616e4623', 'keypoint': {'x': 396.6792, 'y': 457.6415}, 'name': 'yellow'}, {'id': 'f7642eee-44ca-4f1d-abf7-a79d43dfbd0d', 'keypoint': {'x': 279.934, 'y': 535.4717}, 'name': 'yellow'}, {'id': 'd87c603d-89da-4c0b-8f61-a496275c1297', 'keypoint': {'x': 311.066, 'y': 487.217}, 'name': 'blue'}, {'id': '2065d0e4-12eb-4ab3-bde4-75688bb3c142', 'keypoint': {'x': 457.3868, 'y': 439.7406}, 'name': 'green'}, {'id': '1550df58-31c4-49f4-8e6d-7db4a57bc294', 'keypoint': {'x': 435.5943, 'y': 445.967}, 'name': 'green'}, {'id': '1990bf50-38ac-43ab-8581-26778cea80ed', 'keypoint': {'x': 458.9434, 'y': 484.8821}, 'name': 'green'}, {'id': '7895d85c-2e57-4008-b9d7-74625608a42c', 'keypoint': {'x': 485.4057, 'y': 515.2358}, 'name': 'green'}, {'id': '991a571a-03aa-4a00-b80d-3d6a0845625b', 'keypoint': {'x': 440.2642, 'y': 514.4575}, 'name': 'green'}, {'id': 'e0e1086c-3a47-42f6-98fd-868d0756cd4d', 'keypoint': {'x': 378.7783, 'y': 495.7783}, 'name': 'green'}, {'id': '3d132917-dd96-4ae9-9302-da261696357f', 'keypoint': {'x': 333.6368, 'y': 505.1179}, 'name': 'green'}, {'id': 'bc7494f1-b049-49c6-bf24-85e942969065', 'keypoint': {'x': 383.4481, 'y': 56.0377}, 'name': 'you'}]}
[/INST]
{'x': 378.7783, 'y': 495.7783}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Please pick up the green block on your left hand side that is closer to you.
 {'width': 756, 'height': 660, 'annotations': [{'id': '8f941e4d-8e6b-4e37-a2cd-f19b3dbb1dda', 'keypoint': {'x': 293.1651, 'y': 452.9717}, 'name': 'orange'}, {'id': '77f8b560-0c54-4923-9c65-8e8973d5cc1a', 'keypoint': {'x': 293.9434, 'y': 485.6604}, 'name': 'orange'}, {'id': 'c2fdf254-12fa-43ee-9d04-fd097d6a2fad', 'keypoint': {'x': 332.8585, 'y': 517.5708}, 'name': 'orange'}, {'id': 'ee022624-310e-4867-8360-cd61e6a5e8ec', 'keypoint': {'x': 325.0755, 'y': 557.2642}, 'name': 'orange'}, {'id': 'ca78545f-1817-447e-a767-1ec186e40025', 'keypoint': {'x': 377.2217, 'y': 478.6557}, 'name': 'orange'}, {'id': 'b7ea891b-3095-4df8-a718-b193f34c851c', 'keypoint': {'x': 386.5613, 'y': 492.6651}, 'name': 'orange'}, {'id': '54b85951-9b7d-4a22-9b94-5856c8f4cc7e', 'keypoint': {'x': 441.8208, 'y': 499.6698}, 'name': 'orange'}, {'id': 'ef2b7cbb-fa7e-4894-8e50-310d43a6f3eb', 'keypoint': {'x': 385.0047, 'y': 448.3019}, 'name': 'yellow'}, {'id': 'ab7d8b08-4eaf-41cf-8066-9aba376bb546', 'keypoint': {'x': 392.0094, 'y': 523.0189}, 'name': 'yellow'}, {'id': 'aec059f3-2def-4458-9ea4-3fa5916cab5b', 'keypoint': {'x': 466.7264, 'y': 442.8538}, 'name': 'yellow'}, {'id': '39fe094d-1e93-4828-ae7e-d672aeb88451', 'keypoint': {'x': 356.2075, 'y': 533.9151}, 'name': 'blue'}, {'id': '57cde523-a685-45a5-b623-1b11057840da', 'keypoint': {'x': 487.7406, 'y': 488.7736}, 'name': 'blue'}, {'id': 'da91e8aa-4087-4689-95ed-80e4ba9a93ec', 'keypoint': {'x': 327.4104, 'y': 472.4292}, 'name': 'green'}, {'id': '9406ceb5-4de6-4915-8da6-132b32721d7c', 'keypoint': {'x': 508.7547, 'y': 511.3443}, 'name': 'green'}, {'id': 'fd93334e-be71-4598-9c11-abb001661b87', 'keypoint': {'x': 497.0802, 'y': 560.3774}, 'name': 'green'}, {'id': '3067ae97-d698-4a21-82f6-e94a33ed51a0', 'keypoint': {'x': 380.3349, 'y': 59.1509}, 'name': 'you'}]}
[/INST]
{'x': 508.7547, 'y': 511.3443}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the far left green block towards the front and in-between the yellow and orange blocks in front.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'cad61a18-85b8-4f6e-93b3-c3ce9e28cda2', 'keypoint': {'x': 316.5142, 'y': 554.9292}, 'name': 'orange'}, {'id': '17682384-25d2-46d1-ae87-f04077458497', 'keypoint': {'x': 330.5236, 'y': 516.7925}, 'name': 'orange'}, {'id': '5d36ae41-004b-494e-b32d-33279b1dbfee', 'keypoint': {'x': 447.2689, 'y': 457.6415}, 'name': 'orange'}, {'id': 'fee58518-97d8-4c98-9b1b-ebeaa7ef3967', 'keypoint': {'x': 488.5189, 'y': 530.8019}, 'name': 'orange'}, {'id': '4fa5b6b4-f98b-4588-95e6-8255a4bef84b', 'keypoint': {'x': 380.3349, 'y': 446.7453}, 'name': 'yellow'}, {'id': '22f0cd3c-99be-40d9-9b3f-a7bc782da1f8', 'keypoint': {'x': 297.8349, 'y': 529.2453}, 'name': 'yellow'}, {'id': 'd5f859c2-a998-4ee4-bff1-d6025abd4883', 'keypoint': {'x': 375.6651, 'y': 536.25}, 'name': 'yellow'}, {'id': '96691580-8803-4b4b-b31f-4920d402e03e', 'keypoint': {'x': 473.7311, 'y': 435.0708}, 'name': 'yellow'}, {'id': '86653055-abe8-4a82-8b79-fb7f461438c0', 'keypoint': {'x': 474.5094, 'y': 487.217}, 'name': 'yellow'}, {'id': '34e55929-db77-4549-8f50-5adc4da9bd5d', 'keypoint': {'x': 370.9953, 'y': 492.6651}, 'name': 'blue'}, {'id': '500fbe66-22dd-4552-a43c-63bf2efeb1c9', 'keypoint': {'x': 417.6934, 'y': 477.0991}, 'name': 'blue'}, {'id': '85cb36b6-c443-488a-85b6-363a2284870a', 'keypoint': {'x': 345.3113, 'y': 424.9528}, 'name': 'green'}, {'id': '1c037117-1e04-4cd7-a544-c3e10b3639bb', 'keypoint': {'x': 295.5, 'y': 466.9811}, 'name': 'green'}, {'id': 'a8f28480-33ff-4a0a-883f-f1f226ced914', 'keypoint': {'x': 299.3915, 'y': 501.2264}, 'name': 'green'}, {'id': 'b9578593-4dcd-4808-9119-c6439b45d587', 'keypoint': {'x': 427.033, 'y': 534.6934}, 'name': 'green'}, {'id': '70024c2f-19b2-412a-a76d-d9e741635b7e', 'keypoint': {'x': 380.3349, 'y': 33.467}, 'name': 'you'}]}
[/INST]
{'x': 427.033, 'y': 534.6934}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the blue block nearest you.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'fe98e63d-c226-4a25-9e75-229c9a2d828a', 'keypoint': {'x': 377.2217, 'y': 28.7972}, 'name': 'you'}, {'id': 'c1bfc6fe-9e0d-4152-aab0-9efe03f99ee1', 'keypoint': {'x': 291.6085, 'y': 487.217}, 'name': 'yellow'}, {'id': 'eefd5787-08ef-4674-ae9e-c41f4a2ed951', 'keypoint': {'x': 314.1792, 'y': 539.3632}, 'name': 'yellow'}, {'id': '8987e25d-8eb0-42ab-aa8e-390a7b046e44', 'keypoint': {'x': 363.2123, 'y': 544.8113}, 'name': 'yellow'}, {'id': 'af54d764-7469-4b39-95f2-d25ac6241715', 'keypoint': {'x': 434.0377, 'y': 542.4764}, 'name': 'yellow'}, {'id': 'c426887c-43fa-4c7c-9ee6-447f48c5b353', 'keypoint': {'x': 463.6132, 'y': 487.9953}, 'name': 'yellow'}, {'id': 'f1402f90-fe95-42c0-93e5-75019494beba', 'keypoint': {'x': 497.0802, 'y': 442.0755}, 'name': 'yellow'}, {'id': 'ad7b2c3b-efac-4705-8dc4-076514da9f0d', 'keypoint': {'x': 319.6274, 'y': 463.8679}, 'name': 'blue'}, {'id': '57fd11a6-5352-4e90-8c5e-78e080a9f63b', 'keypoint': {'x': 458.1651, 'y': 516.0142}, 'name': 'blue'}, {'id': 'bffb87fb-647e-44e1-b4c0-9bbf464cc4da', 'keypoint': {'x': 353.8726, 'y': 445.1887}, 'name': 'green'}, {'id': '2be751d9-34f0-411b-89e7-1b20b054e1ee', 'keypoint': {'x': 388.8962, 'y': 498.1132}, 'name': 'green'}, {'id': '98a41430-e570-492e-969a-8ea45857d765', 'keypoint': {'x': 395.9009, 'y': 533.1368}, 'name': 'green'}, {'id': '0327cb90-c17c-4528-8964-e978fbff5999', 'keypoint': {'x': 427.8113, 'y': 484.8821}, 'name': 'green'}, {'id': '97f91dd8-6cd8-4194-8783-e9e3b5786e68', 'keypoint': {'x': 456.6085, 'y': 452.9717}, 'name': 'green'}, {'id': '4ebadfd4-3497-4f80-a4e9-a870c25786bd', 'keypoint': {'x': 504.8632, 'y': 502.0047}, 'name': 'green'}, {'id': 'd2ff782c-210c-4090-91a5-de43a78f1232', 'keypoint': {'x': 331.3019, 'y': 512.9009}, 'name': 'orange'}]}
[/INST]
{'x': 319.6274, 'y': 463.8679}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:pick up the left most green block
 {'width': 756, 'height': 660, 'annotations': [{'id': '8f941e4d-8e6b-4e37-a2cd-f19b3dbb1dda', 'keypoint': {'x': 293.1651, 'y': 452.9717}, 'name': 'orange'}, {'id': '77f8b560-0c54-4923-9c65-8e8973d5cc1a', 'keypoint': {'x': 293.9434, 'y': 485.6604}, 'name': 'orange'}, {'id': 'c2fdf254-12fa-43ee-9d04-fd097d6a2fad', 'keypoint': {'x': 332.8585, 'y': 517.5708}, 'name': 'orange'}, {'id': 'ee022624-310e-4867-8360-cd61e6a5e8ec', 'keypoint': {'x': 325.0755, 'y': 557.2642}, 'name': 'orange'}, {'id': 'ca78545f-1817-447e-a767-1ec186e40025', 'keypoint': {'x': 377.2217, 'y': 478.6557}, 'name': 'orange'}, {'id': 'b7ea891b-3095-4df8-a718-b193f34c851c', 'keypoint': {'x': 386.5613, 'y': 492.6651}, 'name': 'orange'}, {'id': '54b85951-9b7d-4a22-9b94-5856c8f4cc7e', 'keypoint': {'x': 441.8208, 'y': 499.6698}, 'name': 'orange'}, {'id': 'ef2b7cbb-fa7e-4894-8e50-310d43a6f3eb', 'keypoint': {'x': 385.0047, 'y': 448.3019}, 'name': 'yellow'}, {'id': 'ab7d8b08-4eaf-41cf-8066-9aba376bb546', 'keypoint': {'x': 392.0094, 'y': 523.0189}, 'name': 'yellow'}, {'id': 'aec059f3-2def-4458-9ea4-3fa5916cab5b', 'keypoint': {'x': 466.7264, 'y': 442.8538}, 'name': 'yellow'}, {'id': '39fe094d-1e93-4828-ae7e-d672aeb88451', 'keypoint': {'x': 356.2075, 'y': 533.9151}, 'name': 'blue'}, {'id': '57cde523-a685-45a5-b623-1b11057840da', 'keypoint': {'x': 487.7406, 'y': 488.7736}, 'name': 'blue'}, {'id': 'da91e8aa-4087-4689-95ed-80e4ba9a93ec', 'keypoint': {'x': 327.4104, 'y': 472.4292}, 'name': 'green'}, {'id': '9406ceb5-4de6-4915-8da6-132b32721d7c', 'keypoint': {'x': 508.7547, 'y': 511.3443}, 'name': 'green'}, {'id': 'fd93334e-be71-4598-9c11-abb001661b87', 'keypoint': {'x': 497.0802, 'y': 560.3774}, 'name': 'green'}, {'id': '3067ae97-d698-4a21-82f6-e94a33ed51a0', 'keypoint': {'x': 380.3349, 'y': 59.1509}, 'name': 'you'}]}
[/INST]
{'x': 508.7547, 'y': 511.3443}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:All the way to the left and farthest away from you is an orange block. Pick it up.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'b1c926ae-92c1-43eb-96c5-76c41d0fab06', 'keypoint': {'x': 342.1981, 'y': 437.4057}, 'name': 'orange'}, {'id': '76267614-95a1-4ded-b4e9-354b821f62bb', 'keypoint': {'x': 314.1792, 'y': 499.6698}, 'name': 'orange'}, {'id': '3124e476-176a-4feb-8844-b3973d48973a', 'keypoint': {'x': 453.4953, 'y': 463.8679}, 'name': 'orange'}, {'id': '728bd5ee-ece2-45c9-b0d7-b887692bb453', 'keypoint': {'x': 494.7453, 'y': 534.6934}, 'name': 'orange'}, {'id': 'da0db0c2-fdee-4550-8e2a-11f1aace764f', 'keypoint': {'x': 376.4434, 'y': 419.5047}, 'name': 'yellow'}, {'id': 'bd27285d-35e5-47b5-9b1e-78bd5fb9e595', 'keypoint': {'x': 360.0991, 'y': 516.7925}, 'name': 'yellow'}, {'id': '5a97f3d8-463a-4c65-9906-b8cf5f772e2b', 'keypoint': {'x': 408.3538, 'y': 526.9104}, 'name': 'yellow'}, {'id': 'f2444027-7c76-40d2-9424-36c46342a49f', 'keypoint': {'x': 318.8491, 'y': 449.0802}, 'name': 'blue'}, {'id': '0b2e3fca-8a71-43d2-a83e-f02af09891b9', 'keypoint': {'x': 315.7358, 'y': 480.2123}, 'name': 'blue'}, {'id': '897e21a9-0e62-42b6-8c01-826ad479026c', 'keypoint': {'x': 378.0, 'y': 490.3302}, 'name': 'blue'}, {'id': 'ca6c0f72-0600-47ba-b4b1-05dc3566a83e', 'keypoint': {'x': 402.1274, 'y': 463.0896}, 'name': 'blue'}, {'id': '41397d83-cab1-4c4d-ba20-b13a1c93f043', 'keypoint': {'x': 419.25, 'y': 432.7358}, 'name': 'blue'}, {'id': '051bcea0-383b-4791-9652-d84f6ef3fe4c', 'keypoint': {'x': 348.4245, 'y': 488.7736}, 'name': 'green'}, {'id': '58f619eb-2384-4a80-97f9-10520e5226a9', 'keypoint': {'x': 311.8443, 'y': 539.3632}, 'name': 'green'}, {'id': 'b9c15fa4-b07d-4f16-9f7d-f007698c83c2', 'keypoint': {'x': 417.6934, 'y': 502.0047}, 'name': 'green'}, {'id': '5c0026dd-e833-4ad7-8339-361bad3686e0', 'keypoint': {'x': 378.0, 'y': 59.1509}, 'name': 'you'}]}
[/INST]
{'x': 494.7453, 'y': 534.6934}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Please pick up the yellow block in line with the blue block on your left hand side located nearer towards me.
 {'width': 756, 'height': 660, 'annotations': [{'id': '53de559f-b2c6-46fa-821d-b5a4e98c67c3', 'keypoint': {'x': 332.0802, 'y': 517.5708}, 'name': 'orange'}, {'id': 'fc5cb394-ebdb-4064-bb12-1d3da5b0cb6f', 'keypoint': {'x': 316.5142, 'y': 540.1415}, 'name': 'orange'}, {'id': 'b8f5d815-90be-4cd5-951f-44207215b594', 'keypoint': {'x': 373.3302, 'y': 537.8066}, 'name': 'orange'}, {'id': '78d88ded-7431-4260-a864-725e2dfef6ab', 'keypoint': {'x': 309.5094, 'y': 498.1132}, 'name': 'yellow'}, {'id': 'e5215008-d111-4bda-9009-f2c0795ffb0d', 'keypoint': {'x': 388.1179, 'y': 471.6509}, 'name': 'yellow'}, {'id': 'b722e1f1-0a3c-47ef-b35f-e9cd85898169', 'keypoint': {'x': 388.1179, 'y': 501.2264}, 'name': 'yellow'}, {'id': '27ad7266-481b-40f9-aaa0-e71dabd71d5e', 'keypoint': {'x': 413.8019, 'y': 491.1085}, 'name': 'yellow'}, {'id': '71348b8e-08dc-4a76-be0e-68cbee1a56a4', 'keypoint': {'x': 416.1368, 'y': 449.8585}, 'name': 'yellow'}, {'id': 'e46d6178-4284-433d-a81f-bc6cbbf7cdbb', 'keypoint': {'x': 490.8538, 'y': 505.1179}, 'name': 'yellow'}, {'id': '272a554a-aad9-4bba-9062-941465beeaa5', 'keypoint': {'x': 465.1698, 'y': 554.1509}, 'name': 'yellow'}, {'id': 'c6ba259b-1c8e-43d4-8bdb-404594f1c175', 'keypoint': {'x': 310.2877, 'y': 438.9623}, 'name': 'blue'}, {'id': '693a6454-810f-447d-8417-a2c57867901d', 'keypoint': {'x': 354.6509, 'y': 477.8774}, 'name': 'blue'}, {'id': '3ec7fa7f-c792-4c99-a423-b40bbb665d89', 'keypoint': {'x': 427.8113, 'y': 428.8443}, 'name': 'blue'}, {'id': '3e041f2e-9e78-4001-9fa0-3bebee80ac07', 'keypoint': {'x': 440.2642, 'y': 498.8915}, 'name': 'green'}, {'id': 'e2e59b33-f4e7-47cd-b137-77a874c834c7', 'keypoint': {'x': 450.3821, 'y': 519.1274}, 'name': 'green'}, {'id': '707959b0-76f6-463a-bc2f-62cd702815f4', 'keypoint': {'x': 377.2217, 'y': 55.2594}, 'name': 'you'}]}
[/INST]
{'x': 413.8019, 'y': 491.1085}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:pick up the closest orange block
 {'width': 756, 'height': 660, 'annotations': [{'id': 'b1c926ae-92c1-43eb-96c5-76c41d0fab06', 'keypoint': {'x': 342.1981, 'y': 437.4057}, 'name': 'orange'}, {'id': '76267614-95a1-4ded-b4e9-354b821f62bb', 'keypoint': {'x': 314.1792, 'y': 499.6698}, 'name': 'orange'}, {'id': '3124e476-176a-4feb-8844-b3973d48973a', 'keypoint': {'x': 453.4953, 'y': 463.8679}, 'name': 'orange'}, {'id': '728bd5ee-ece2-45c9-b0d7-b887692bb453', 'keypoint': {'x': 494.7453, 'y': 534.6934}, 'name': 'orange'}, {'id': 'da0db0c2-fdee-4550-8e2a-11f1aace764f', 'keypoint': {'x': 376.4434, 'y': 419.5047}, 'name': 'yellow'}, {'id': 'bd27285d-35e5-47b5-9b1e-78bd5fb9e595', 'keypoint': {'x': 360.0991, 'y': 516.7925}, 'name': 'yellow'}, {'id': '5a97f3d8-463a-4c65-9906-b8cf5f772e2b', 'keypoint': {'x': 408.3538, 'y': 526.9104}, 'name': 'yellow'}, {'id': 'f2444027-7c76-40d2-9424-36c46342a49f', 'keypoint': {'x': 318.8491, 'y': 449.0802}, 'name': 'blue'}, {'id': '0b2e3fca-8a71-43d2-a83e-f02af09891b9', 'keypoint': {'x': 315.7358, 'y': 480.2123}, 'name': 'blue'}, {'id': '897e21a9-0e62-42b6-8c01-826ad479026c', 'keypoint': {'x': 378.0, 'y': 490.3302}, 'name': 'blue'}, {'id': 'ca6c0f72-0600-47ba-b4b1-05dc3566a83e', 'keypoint': {'x': 402.1274, 'y': 463.0896}, 'name': 'blue'}, {'id': '41397d83-cab1-4c4d-ba20-b13a1c93f043', 'keypoint': {'x': 419.25, 'y': 432.7358}, 'name': 'blue'}, {'id': '051bcea0-383b-4791-9652-d84f6ef3fe4c', 'keypoint': {'x': 348.4245, 'y': 488.7736}, 'name': 'green'}, {'id': '58f619eb-2384-4a80-97f9-10520e5226a9', 'keypoint': {'x': 311.8443, 'y': 539.3632}, 'name': 'green'}, {'id': 'b9c15fa4-b07d-4f16-9f7d-f007698c83c2', 'keypoint': {'x': 417.6934, 'y': 502.0047}, 'name': 'green'}, {'id': '5c0026dd-e833-4ad7-8339-361bad3686e0', 'keypoint': {'x': 378.0, 'y': 59.1509}, 'name': 'you'}]}
[/INST]
{'x': 342.1981, 'y': 437.4057}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Grab the blue block that is closest to you.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'c7519096-f7ea-4ab1-a66d-97c708976599', 'keypoint': {'x': 380.3349, 'y': 34.2453}, 'name': 'you'}, {'id': 'a95968ee-51d4-482d-ae79-97cb54663aed', 'keypoint': {'x': 293.9434, 'y': 479.434}, 'name': 'blue'}, {'id': '2fc8b299-6074-43eb-92bc-7ed758a958fc', 'keypoint': {'x': 329.7453, 'y': 557.2642}, 'name': 'blue'}, {'id': '11dbc045-de4f-4d3a-bc50-a1c2ba279bd1', 'keypoint': {'x': 366.3255, 'y': 546.3679}, 'name': 'blue'}, {'id': 'a3036737-fb7e-4f86-88c8-b0c5262cc051', 'keypoint': {'x': 379.5566, 'y': 438.9623}, 'name': 'blue'}, {'id': '5370f3f3-ef52-45e8-bdb4-3d96d54f4bd9', 'keypoint': {'x': 455.8302, 'y': 449.0802}, 'name': 'blue'}, {'id': '95294165-9354-41e7-b1be-60d4033bd1a3', 'keypoint': {'x': 318.0708, 'y': 454.5283}, 'name': 'yellow'}, {'id': '71f49ac9-1905-457f-9219-701cac7c42e4', 'keypoint': {'x': 318.0708, 'y': 530.8019}, 'name': 'yellow'}, {'id': '6f7a8a19-794f-4ffc-80ed-8a42dedeab0f', 'keypoint': {'x': 463.6132, 'y': 524.5755}, 'name': 'yellow'}, {'id': '407068c6-6a1e-4a14-b9d0-c0a4ed13b9c2', 'keypoint': {'x': 395.1226, 'y': 466.2028}, 'name': 'green'}, {'id': '847fb0f9-02cc-42df-8984-2b193e2abb8d', 'keypoint': {'x': 403.684, 'y': 523.0189}, 'name': 'green'}, {'id': 'bd78738b-2c39-4b85-9575-960d46ecd66c', 'keypoint': {'x': 325.0755, 'y': 500.4481}, 'name': 'orange'}, {'id': 'dafb3b80-4c1e-48a7-b398-e3aa56c5dcb0', 'keypoint': {'x': 374.1085, 'y': 515.2358}, 'name': 'orange'}, {'id': '36cf3f26-cac5-4368-91eb-ae8937ec2f91', 'keypoint': {'x': 412.2453, 'y': 427.2877}, 'name': 'orange'}, {'id': 'f93ff614-2250-4f4d-8ec0-ac87ec62305c', 'keypoint': {'x': 436.3726, 'y': 437.4057}, 'name': 'orange'}, {'id': '09066da9-88d0-4617-8cf6-7d604ee0190d', 'keypoint': {'x': 485.4057, 'y': 510.566}, 'name': 'orange'}]}
[/INST]
{'x': 329.7453, 'y': 557.2642}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the green block directly in front of you
 {'width': 756, 'height': 660, 'annotations': [{'id': 'cad61a18-85b8-4f6e-93b3-c3ce9e28cda2', 'keypoint': {'x': 316.5142, 'y': 554.9292}, 'name': 'orange'}, {'id': '17682384-25d2-46d1-ae87-f04077458497', 'keypoint': {'x': 330.5236, 'y': 516.7925}, 'name': 'orange'}, {'id': '5d36ae41-004b-494e-b32d-33279b1dbfee', 'keypoint': {'x': 447.2689, 'y': 457.6415}, 'name': 'orange'}, {'id': 'fee58518-97d8-4c98-9b1b-ebeaa7ef3967', 'keypoint': {'x': 488.5189, 'y': 530.8019}, 'name': 'orange'}, {'id': '4fa5b6b4-f98b-4588-95e6-8255a4bef84b', 'keypoint': {'x': 380.3349, 'y': 446.7453}, 'name': 'yellow'}, {'id': '22f0cd3c-99be-40d9-9b3f-a7bc782da1f8', 'keypoint': {'x': 297.8349, 'y': 529.2453}, 'name': 'yellow'}, {'id': 'd5f859c2-a998-4ee4-bff1-d6025abd4883', 'keypoint': {'x': 375.6651, 'y': 536.25}, 'name': 'yellow'}, {'id': '96691580-8803-4b4b-b31f-4920d402e03e', 'keypoint': {'x': 473.7311, 'y': 435.0708}, 'name': 'yellow'}, {'id': '86653055-abe8-4a82-8b79-fb7f461438c0', 'keypoint': {'x': 474.5094, 'y': 487.217}, 'name': 'yellow'}, {'id': '34e55929-db77-4549-8f50-5adc4da9bd5d', 'keypoint': {'x': 370.9953, 'y': 492.6651}, 'name': 'blue'}, {'id': '500fbe66-22dd-4552-a43c-63bf2efeb1c9', 'keypoint': {'x': 417.6934, 'y': 477.0991}, 'name': 'blue'}, {'id': '85cb36b6-c443-488a-85b6-363a2284870a', 'keypoint': {'x': 345.3113, 'y': 424.9528}, 'name': 'green'}, {'id': '1c037117-1e04-4cd7-a544-c3e10b3639bb', 'keypoint': {'x': 295.5, 'y': 466.9811}, 'name': 'green'}, {'id': 'a8f28480-33ff-4a0a-883f-f1f226ced914', 'keypoint': {'x': 299.3915, 'y': 501.2264}, 'name': 'green'}, {'id': 'b9578593-4dcd-4808-9119-c6439b45d587', 'keypoint': {'x': 427.033, 'y': 534.6934}, 'name': 'green'}, {'id': '70024c2f-19b2-412a-a76d-d9e741635b7e', 'keypoint': {'x': 380.3349, 'y': 33.467}, 'name': 'you'}]}
[/INST]
{'x': 345.3113, 'y': 424.9528}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:pick up the green block closest to me
 {'width': 756, 'height': 660, 'annotations': [{'id': 'cad61a18-85b8-4f6e-93b3-c3ce9e28cda2', 'keypoint': {'x': 316.5142, 'y': 554.9292}, 'name': 'orange'}, {'id': '17682384-25d2-46d1-ae87-f04077458497', 'keypoint': {'x': 330.5236, 'y': 516.7925}, 'name': 'orange'}, {'id': '5d36ae41-004b-494e-b32d-33279b1dbfee', 'keypoint': {'x': 447.2689, 'y': 457.6415}, 'name': 'orange'}, {'id': 'fee58518-97d8-4c98-9b1b-ebeaa7ef3967', 'keypoint': {'x': 488.5189, 'y': 530.8019}, 'name': 'orange'}, {'id': '4fa5b6b4-f98b-4588-95e6-8255a4bef84b', 'keypoint': {'x': 380.3349, 'y': 446.7453}, 'name': 'yellow'}, {'id': '22f0cd3c-99be-40d9-9b3f-a7bc782da1f8', 'keypoint': {'x': 297.8349, 'y': 529.2453}, 'name': 'yellow'}, {'id': 'd5f859c2-a998-4ee4-bff1-d6025abd4883', 'keypoint': {'x': 375.6651, 'y': 536.25}, 'name': 'yellow'}, {'id': '96691580-8803-4b4b-b31f-4920d402e03e', 'keypoint': {'x': 473.7311, 'y': 435.0708}, 'name': 'yellow'}, {'id': '86653055-abe8-4a82-8b79-fb7f461438c0', 'keypoint': {'x': 474.5094, 'y': 487.217}, 'name': 'yellow'}, {'id': '34e55929-db77-4549-8f50-5adc4da9bd5d', 'keypoint': {'x': 370.9953, 'y': 492.6651}, 'name': 'blue'}, {'id': '500fbe66-22dd-4552-a43c-63bf2efeb1c9', 'keypoint': {'x': 417.6934, 'y': 477.0991}, 'name': 'blue'}, {'id': '85cb36b6-c443-488a-85b6-363a2284870a', 'keypoint': {'x': 345.3113, 'y': 424.9528}, 'name': 'green'}, {'id': '1c037117-1e04-4cd7-a544-c3e10b3639bb', 'keypoint': {'x': 295.5, 'y': 466.9811}, 'name': 'green'}, {'id': 'a8f28480-33ff-4a0a-883f-f1f226ced914', 'keypoint': {'x': 299.3915, 'y': 501.2264}, 'name': 'green'}, {'id': 'b9578593-4dcd-4808-9119-c6439b45d587', 'keypoint': {'x': 427.033, 'y': 534.6934}, 'name': 'green'}, {'id': '70024c2f-19b2-412a-a76d-d9e741635b7e', 'keypoint': {'x': 380.3349, 'y': 33.467}, 'name': 'you'}]}
[/INST]
{'x': 427.033, 'y': 534.6934}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Reach forward in the center and pick up the green block that is immediately before the yellow block.
 {'width': 756, 'height': 660, 'annotations': [{'id': '8beb9a9e-a451-450a-b594-7a9ae18e74f9', 'keypoint': {'x': 458.9434, 'y': 531.5802}, 'name': 'orange'}, {'id': 'ee956837-2183-4b52-8aa2-dfa990f1d376', 'keypoint': {'x': 386.5613, 'y': 529.2453}, 'name': 'yellow'}, {'id': '427a5df9-dc11-494e-8632-81ffcd5e66c8', 'keypoint': {'x': 471.3962, 'y': 453.75}, 'name': 'yellow'}, {'id': 'bcc302a4-c191-4bc2-82e0-1ceedd4a2928', 'keypoint': {'x': 329.7453, 'y': 453.75}, 'name': 'blue'}, {'id': '8584b004-32b4-47bd-9100-a2eab16c5301', 'keypoint': {'x': 294.7217, 'y': 501.2264}, 'name': 'blue'}, {'id': 'ba4430e4-6293-47e3-b512-0e78897c96fb', 'keypoint': {'x': 317.2925, 'y': 520.684}, 'name': 'blue'}, {'id': '9c57d65f-59e8-4d45-a882-721a56841969', 'keypoint': {'x': 433.2594, 'y': 498.1132}, 'name': 'blue'}, {'id': '0e241c7c-e225-4266-9044-a65451aceb68', 'keypoint': {'x': 361.6557, 'y': 428.8443}, 'name': 'green'}, {'id': '050252d9-484a-4a87-a46c-1f69f5d7ea14', 'keypoint': {'x': 416.9151, 'y': 435.8491}, 'name': 'green'}, {'id': '5b9ab273-0e91-4589-9a86-d2969adc4e30', 'keypoint': {'x': 435.5943, 'y': 445.967}, 'name': 'green'}, {'id': '4b452968-0027-4b94-92e6-e8dcd32d1e25', 'keypoint': {'x': 468.283, 'y': 489.5519}, 'name': 'green'}, {'id': '3ab98d12-5e5c-4bdf-931d-55f2ee2d23b2', 'keypoint': {'x': 386.5613, 'y': 498.8915}, 'name': 'green'}, {'id': '416f60f7-59aa-46b3-858c-7dfde5e23875', 'keypoint': {'x': 353.0943, 'y': 497.3349}, 'name': 'green'}, {'id': 'ce8faeb7-52eb-4dc8-8db9-4295f1b077e3', 'keypoint': {'x': 329.7453, 'y': 491.8868}, 'name': 'green'}, {'id': '6ea4f70c-aa80-4f14-a1ac-d26081a22a0d', 'keypoint': {'x': 379.5566, 'y': 52.1462}, 'name': 'you'}]}
[/INST]
{'x': 386.5613, 'y': 498.8915}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the green block that is closest to you.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'cad61a18-85b8-4f6e-93b3-c3ce9e28cda2', 'keypoint': {'x': 316.5142, 'y': 554.9292}, 'name': 'orange'}, {'id': '17682384-25d2-46d1-ae87-f04077458497', 'keypoint': {'x': 330.5236, 'y': 516.7925}, 'name': 'orange'}, {'id': '5d36ae41-004b-494e-b32d-33279b1dbfee', 'keypoint': {'x': 447.2689, 'y': 457.6415}, 'name': 'orange'}, {'id': 'fee58518-97d8-4c98-9b1b-ebeaa7ef3967', 'keypoint': {'x': 488.5189, 'y': 530.8019}, 'name': 'orange'}, {'id': '4fa5b6b4-f98b-4588-95e6-8255a4bef84b', 'keypoint': {'x': 380.3349, 'y': 446.7453}, 'name': 'yellow'}, {'id': '22f0cd3c-99be-40d9-9b3f-a7bc782da1f8', 'keypoint': {'x': 297.8349, 'y': 529.2453}, 'name': 'yellow'}, {'id': 'd5f859c2-a998-4ee4-bff1-d6025abd4883', 'keypoint': {'x': 375.6651, 'y': 536.25}, 'name': 'yellow'}, {'id': '96691580-8803-4b4b-b31f-4920d402e03e', 'keypoint': {'x': 473.7311, 'y': 435.0708}, 'name': 'yellow'}, {'id': '86653055-abe8-4a82-8b79-fb7f461438c0', 'keypoint': {'x': 474.5094, 'y': 487.217}, 'name': 'yellow'}, {'id': '34e55929-db77-4549-8f50-5adc4da9bd5d', 'keypoint': {'x': 370.9953, 'y': 492.6651}, 'name': 'blue'}, {'id': '500fbe66-22dd-4552-a43c-63bf2efeb1c9', 'keypoint': {'x': 417.6934, 'y': 477.0991}, 'name': 'blue'}, {'id': '85cb36b6-c443-488a-85b6-363a2284870a', 'keypoint': {'x': 345.3113, 'y': 424.9528}, 'name': 'green'}, {'id': '1c037117-1e04-4cd7-a544-c3e10b3639bb', 'keypoint': {'x': 295.5, 'y': 466.9811}, 'name': 'green'}, {'id': 'a8f28480-33ff-4a0a-883f-f1f226ced914', 'keypoint': {'x': 299.3915, 'y': 501.2264}, 'name': 'green'}, {'id': 'b9578593-4dcd-4808-9119-c6439b45d587', 'keypoint': {'x': 427.033, 'y': 534.6934}, 'name': 'green'}, {'id': '70024c2f-19b2-412a-a76d-d9e741635b7e', 'keypoint': {'x': 380.3349, 'y': 33.467}, 'name': 'you'}]}
[/INST]
{'x': 345.3113, 'y': 424.9528}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Find the orange block closest to you. Pick up the yellow block just behind the yellow block behind that orange block.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'b328ac50-08b5-486e-a937-2cddbc04245c', 'keypoint': {'x': 304.8396, 'y': 434.2925}, 'name': 'orange'}, {'id': '7344b92c-5683-4670-9cef-33f4cf8e73fb', 'keypoint': {'x': 472.9528, 'y': 456.0849}, 'name': 'orange'}, {'id': '6958bbb7-3dce-4cfe-9049-02ebc5a32e43', 'keypoint': {'x': 306.3962, 'y': 460.7547}, 'name': 'yellow'}, {'id': '62285f29-522c-498a-9fdb-53e2e8421ea5', 'keypoint': {'x': 330.5236, 'y': 470.8726}, 'name': 'yellow'}, {'id': 'e4402e53-f01e-4254-a874-f6ffd3892ac4', 'keypoint': {'x': 381.8915, 'y': 475.5425}, 'name': 'yellow'}, {'id': '3f3ed2fa-5312-418f-9f29-f4750a72aa9e', 'keypoint': {'x': 402.9057, 'y': 435.0708}, 'name': 'yellow'}, {'id': 'fe5ba8f4-1f70-4712-ae25-cd20ac9670d6', 'keypoint': {'x': 455.8302, 'y': 481.7689}, 'name': 'yellow'}, {'id': 'bd1c4ddb-8345-4f24-a982-f6cd6a64fb0e', 'keypoint': {'x': 501.75, 'y': 505.8962}, 'name': 'yellow'}, {'id': '8f332be0-ad10-4130-9840-f3a1dc330890', 'keypoint': {'x': 352.316, 'y': 513.6792}, 'name': 'yellow'}, {'id': '5b19adbf-7efb-4a90-9f40-4c0bf2f3b9a8', 'keypoint': {'x': 324.2972, 'y': 499.6698}, 'name': 'yellow'}, {'id': '7f124547-f8b7-4280-b7e1-401876ce2bdb', 'keypoint': {'x': 308.7311, 'y': 552.5943}, 'name': 'yellow'}, {'id': '8a1fd773-bc7c-4be5-a292-c1c53d5c77df', 'keypoint': {'x': 298.6132, 'y': 491.8868}, 'name': 'blue'}, {'id': '7be8714d-676d-466e-a70a-e9d078b757f4', 'keypoint': {'x': 381.1132, 'y': 497.3349}, 'name': 'blue'}, {'id': 'b1ee1721-f1d2-4cc8-828b-2e6c5072d54f', 'keypoint': {'x': 451.9387, 'y': 432.7358}, 'name': 'blue'}, {'id': '0416eebd-320e-40d8-bbc7-a94a6333aa95', 'keypoint': {'x': 497.0802, 'y': 485.6604}, 'name': 'green'}, {'id': '2240b1ab-d515-4872-a587-09ca836dc397', 'keypoint': {'x': 378.7783, 'y': 28.7972}, 'name': 'you'}]}
[/INST]
{'x': 330.5236, 'y': 470.8726}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:I would tell the robot to pick up the blue block directly in front of him that is to the left of another blue block
 {'width': 756, 'height': 660, 'annotations': [{'id': 'c7519096-f7ea-4ab1-a66d-97c708976599', 'keypoint': {'x': 380.3349, 'y': 34.2453}, 'name': 'you'}, {'id': 'a95968ee-51d4-482d-ae79-97cb54663aed', 'keypoint': {'x': 293.9434, 'y': 479.434}, 'name': 'blue'}, {'id': '2fc8b299-6074-43eb-92bc-7ed758a958fc', 'keypoint': {'x': 329.7453, 'y': 557.2642}, 'name': 'blue'}, {'id': '11dbc045-de4f-4d3a-bc50-a1c2ba279bd1', 'keypoint': {'x': 366.3255, 'y': 546.3679}, 'name': 'blue'}, {'id': 'a3036737-fb7e-4f86-88c8-b0c5262cc051', 'keypoint': {'x': 379.5566, 'y': 438.9623}, 'name': 'blue'}, {'id': '5370f3f3-ef52-45e8-bdb4-3d96d54f4bd9', 'keypoint': {'x': 455.8302, 'y': 449.0802}, 'name': 'blue'}, {'id': '95294165-9354-41e7-b1be-60d4033bd1a3', 'keypoint': {'x': 318.0708, 'y': 454.5283}, 'name': 'yellow'}, {'id': '71f49ac9-1905-457f-9219-701cac7c42e4', 'keypoint': {'x': 318.0708, 'y': 530.8019}, 'name': 'yellow'}, {'id': '6f7a8a19-794f-4ffc-80ed-8a42dedeab0f', 'keypoint': {'x': 463.6132, 'y': 524.5755}, 'name': 'yellow'}, {'id': '407068c6-6a1e-4a14-b9d0-c0a4ed13b9c2', 'keypoint': {'x': 395.1226, 'y': 466.2028}, 'name': 'green'}, {'id': '847fb0f9-02cc-42df-8984-2b193e2abb8d', 'keypoint': {'x': 403.684, 'y': 523.0189}, 'name': 'green'}, {'id': 'bd78738b-2c39-4b85-9575-960d46ecd66c', 'keypoint': {'x': 325.0755, 'y': 500.4481}, 'name': 'orange'}, {'id': 'dafb3b80-4c1e-48a7-b398-e3aa56c5dcb0', 'keypoint': {'x': 374.1085, 'y': 515.2358}, 'name': 'orange'}, {'id': '36cf3f26-cac5-4368-91eb-ae8937ec2f91', 'keypoint': {'x': 412.2453, 'y': 427.2877}, 'name': 'orange'}, {'id': 'f93ff614-2250-4f4d-8ec0-ac87ec62305c', 'keypoint': {'x': 436.3726, 'y': 437.4057}, 'name': 'orange'}, {'id': '09066da9-88d0-4617-8cf6-7d604ee0190d', 'keypoint': {'x': 485.4057, 'y': 510.566}, 'name': 'orange'}]}
[/INST]
{'x': 329.7453, 'y': 557.2642}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Narrow search to only blue blocks. Find the blue block that is closest to you. Pick it up.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'fe98e63d-c226-4a25-9e75-229c9a2d828a', 'keypoint': {'x': 377.2217, 'y': 28.7972}, 'name': 'you'}, {'id': 'c1bfc6fe-9e0d-4152-aab0-9efe03f99ee1', 'keypoint': {'x': 291.6085, 'y': 487.217}, 'name': 'yellow'}, {'id': 'eefd5787-08ef-4674-ae9e-c41f4a2ed951', 'keypoint': {'x': 314.1792, 'y': 539.3632}, 'name': 'yellow'}, {'id': '8987e25d-8eb0-42ab-aa8e-390a7b046e44', 'keypoint': {'x': 363.2123, 'y': 544.8113}, 'name': 'yellow'}, {'id': 'af54d764-7469-4b39-95f2-d25ac6241715', 'keypoint': {'x': 434.0377, 'y': 542.4764}, 'name': 'yellow'}, {'id': 'c426887c-43fa-4c7c-9ee6-447f48c5b353', 'keypoint': {'x': 463.6132, 'y': 487.9953}, 'name': 'yellow'}, {'id': 'f1402f90-fe95-42c0-93e5-75019494beba', 'keypoint': {'x': 497.0802, 'y': 442.0755}, 'name': 'yellow'}, {'id': 'ad7b2c3b-efac-4705-8dc4-076514da9f0d', 'keypoint': {'x': 319.6274, 'y': 463.8679}, 'name': 'blue'}, {'id': '57fd11a6-5352-4e90-8c5e-78e080a9f63b', 'keypoint': {'x': 458.1651, 'y': 516.0142}, 'name': 'blue'}, {'id': 'bffb87fb-647e-44e1-b4c0-9bbf464cc4da', 'keypoint': {'x': 353.8726, 'y': 445.1887}, 'name': 'green'}, {'id': '2be751d9-34f0-411b-89e7-1b20b054e1ee', 'keypoint': {'x': 388.8962, 'y': 498.1132}, 'name': 'green'}, {'id': '98a41430-e570-492e-969a-8ea45857d765', 'keypoint': {'x': 395.9009, 'y': 533.1368}, 'name': 'green'}, {'id': '0327cb90-c17c-4528-8964-e978fbff5999', 'keypoint': {'x': 427.8113, 'y': 484.8821}, 'name': 'green'}, {'id': '97f91dd8-6cd8-4194-8783-e9e3b5786e68', 'keypoint': {'x': 456.6085, 'y': 452.9717}, 'name': 'green'}, {'id': '4ebadfd4-3497-4f80-a4e9-a870c25786bd', 'keypoint': {'x': 504.8632, 'y': 502.0047}, 'name': 'green'}, {'id': 'd2ff782c-210c-4090-91a5-de43a78f1232', 'keypoint': {'x': 331.3019, 'y': 512.9009}, 'name': 'orange'}]}
[/INST]
{'x': 319.6274, 'y': 463.8679}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:There is a group of green blocks on your left and one green block on your right. Ignore those, and grab the green block that falls in the center, almost touching the orange block.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'd5bcc2b7-ff91-4686-8f50-3a73ffd3beeb', 'keypoint': {'x': 315.7358, 'y': 436.6274}, 'name': 'orange'}, {'id': '70515b86-98d5-4fa6-9118-a0bc484f6de1', 'keypoint': {'x': 313.4009, 'y': 525.3538}, 'name': 'orange'}, {'id': 'b40b0958-f706-45d2-a042-0440eb67121b', 'keypoint': {'x': 380.3349, 'y': 515.2358}, 'name': 'orange'}, {'id': '9dcd051f-bad9-4f84-b653-9a30a72cb545', 'keypoint': {'x': 389.6745, 'y': 552.5943}, 'name': 'orange'}, {'id': 'b0de6329-f4d9-486c-91ed-478f28262cff', 'keypoint': {'x': 363.9906, 'y': 439.7406}, 'name': 'yellow'}, {'id': '10eeedc5-3253-42e7-b401-33c4616e4623', 'keypoint': {'x': 396.6792, 'y': 457.6415}, 'name': 'yellow'}, {'id': 'f7642eee-44ca-4f1d-abf7-a79d43dfbd0d', 'keypoint': {'x': 279.934, 'y': 535.4717}, 'name': 'yellow'}, {'id': 'd87c603d-89da-4c0b-8f61-a496275c1297', 'keypoint': {'x': 311.066, 'y': 487.217}, 'name': 'blue'}, {'id': '2065d0e4-12eb-4ab3-bde4-75688bb3c142', 'keypoint': {'x': 457.3868, 'y': 439.7406}, 'name': 'green'}, {'id': '1550df58-31c4-49f4-8e6d-7db4a57bc294', 'keypoint': {'x': 435.5943, 'y': 445.967}, 'name': 'green'}, {'id': '1990bf50-38ac-43ab-8581-26778cea80ed', 'keypoint': {'x': 458.9434, 'y': 484.8821}, 'name': 'green'}, {'id': '7895d85c-2e57-4008-b9d7-74625608a42c', 'keypoint': {'x': 485.4057, 'y': 515.2358}, 'name': 'green'}, {'id': '991a571a-03aa-4a00-b80d-3d6a0845625b', 'keypoint': {'x': 440.2642, 'y': 514.4575}, 'name': 'green'}, {'id': 'e0e1086c-3a47-42f6-98fd-868d0756cd4d', 'keypoint': {'x': 378.7783, 'y': 495.7783}, 'name': 'green'}, {'id': '3d132917-dd96-4ae9-9302-da261696357f', 'keypoint': {'x': 333.6368, 'y': 505.1179}, 'name': 'green'}, {'id': 'bc7494f1-b049-49c6-bf24-85e942969065', 'keypoint': {'x': 383.4481, 'y': 56.0377}, 'name': 'you'}]}
[/INST]
{'x': 378.7783, 'y': 495.7783}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the 3rd blue block from your right.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'a7c6f9ab-55b7-422b-8e7f-70902f2e94d6', 'keypoint': {'x': 320.4057, 'y': 488.7736}, 'name': 'orange'}, {'id': 'acc187bd-2783-4707-be6a-314a58518228', 'keypoint': {'x': 423.9198, 'y': 451.4151}, 'name': 'orange'}, {'id': '7b9652c2-a499-48cc-ab07-68e686e885ac', 'keypoint': {'x': 358.5425, 'y': 462.3113}, 'name': 'yellow'}, {'id': '6e4e6633-28b0-4e37-a08b-d90cdb596a0d', 'keypoint': {'x': 334.4151, 'y': 548.7028}, 'name': 'yellow'}, {'id': 'dc28f152-a36a-458c-9f8a-21f4093b66bd', 'keypoint': {'x': 418.4717, 'y': 547.9245}, 'name': 'yellow'}, {'id': '34029874-d3f4-4998-a6fa-85ee314b35b5', 'keypoint': {'x': 479.1792, 'y': 484.8821}, 'name': 'yellow'}, {'id': '3d4114df-2e6e-4d36-8e12-47bbffbd6150', 'keypoint': {'x': 314.1792, 'y': 452.1934}, 'name': 'blue'}, {'id': '021cc2a3-2e83-4079-9777-64d3bc57e829', 'keypoint': {'x': 293.1651, 'y': 557.2642}, 'name': 'blue'}, {'id': '49911b5f-df0a-4694-9ff5-d8165f0f587a', 'keypoint': {'x': 390.4528, 'y': 513.6792}, 'name': 'blue'}, {'id': '95b20075-c6a2-46e0-b203-f8004a3ee4a3', 'keypoint': {'x': 440.2642, 'y': 484.8821}, 'name': 'blue'}, {'id': '031e7d06-9b93-4602-a1ef-c48bce3518c4', 'keypoint': {'x': 462.0566, 'y': 469.316}, 'name': 'blue'}, {'id': '3f2fdd03-f2f9-4892-999b-713f9e988be6', 'keypoint': {'x': 493.1887, 'y': 465.4245}, 'name': 'blue'}, {'id': '7ed0f47f-3bdf-429a-ab98-1d1a9d3058fa', 'keypoint': {'x': 482.2925, 'y': 507.4528}, 'name': 'green'}, {'id': '1fb5251e-aa36-4074-87f5-edef1e5a25c7', 'keypoint': {'x': 491.6321, 'y': 542.4764}, 'name': 'green'}, {'id': 'e7cf6134-82a1-4efe-99a5-a098dcdb3a3e', 'keypoint': {'x': 334.4151, 'y': 516.0142}, 'name': 'green'}, {'id': '0aeb291a-183c-47ce-9552-8b4d4860c05f', 'keypoint': {'x': 376.4434, 'y': 44.3632}, 'name': 'you'}]}
[/INST]
{'x': 440.2642, 'y': 484.8821}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the blue block farthest away from you.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'c7519096-f7ea-4ab1-a66d-97c708976599', 'keypoint': {'x': 380.3349, 'y': 34.2453}, 'name': 'you'}, {'id': 'a95968ee-51d4-482d-ae79-97cb54663aed', 'keypoint': {'x': 293.9434, 'y': 479.434}, 'name': 'blue'}, {'id': '2fc8b299-6074-43eb-92bc-7ed758a958fc', 'keypoint': {'x': 329.7453, 'y': 557.2642}, 'name': 'blue'}, {'id': '11dbc045-de4f-4d3a-bc50-a1c2ba279bd1', 'keypoint': {'x': 366.3255, 'y': 546.3679}, 'name': 'blue'}, {'id': 'a3036737-fb7e-4f86-88c8-b0c5262cc051', 'keypoint': {'x': 379.5566, 'y': 438.9623}, 'name': 'blue'}, {'id': '5370f3f3-ef52-45e8-bdb4-3d96d54f4bd9', 'keypoint': {'x': 455.8302, 'y': 449.0802}, 'name': 'blue'}, {'id': '95294165-9354-41e7-b1be-60d4033bd1a3', 'keypoint': {'x': 318.0708, 'y': 454.5283}, 'name': 'yellow'}, {'id': '71f49ac9-1905-457f-9219-701cac7c42e4', 'keypoint': {'x': 318.0708, 'y': 530.8019}, 'name': 'yellow'}, {'id': '6f7a8a19-794f-4ffc-80ed-8a42dedeab0f', 'keypoint': {'x': 463.6132, 'y': 524.5755}, 'name': 'yellow'}, {'id': '407068c6-6a1e-4a14-b9d0-c0a4ed13b9c2', 'keypoint': {'x': 395.1226, 'y': 466.2028}, 'name': 'green'}, {'id': '847fb0f9-02cc-42df-8984-2b193e2abb8d', 'keypoint': {'x': 403.684, 'y': 523.0189}, 'name': 'green'}, {'id': 'bd78738b-2c39-4b85-9575-960d46ecd66c', 'keypoint': {'x': 325.0755, 'y': 500.4481}, 'name': 'orange'}, {'id': 'dafb3b80-4c1e-48a7-b398-e3aa56c5dcb0', 'keypoint': {'x': 374.1085, 'y': 515.2358}, 'name': 'orange'}, {'id': '36cf3f26-cac5-4368-91eb-ae8937ec2f91', 'keypoint': {'x': 412.2453, 'y': 427.2877}, 'name': 'orange'}, {'id': 'f93ff614-2250-4f4d-8ec0-ac87ec62305c', 'keypoint': {'x': 436.3726, 'y': 437.4057}, 'name': 'orange'}, {'id': '09066da9-88d0-4617-8cf6-7d604ee0190d', 'keypoint': {'x': 485.4057, 'y': 510.566}, 'name': 'orange'}]}
[/INST]
{'x': 329.7453, 'y': 557.2642}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the green block on your left that is closest to you.
 {'width': 756, 'height': 660, 'annotations': [{'id': '8f941e4d-8e6b-4e37-a2cd-f19b3dbb1dda', 'keypoint': {'x': 293.1651, 'y': 452.9717}, 'name': 'orange'}, {'id': '77f8b560-0c54-4923-9c65-8e8973d5cc1a', 'keypoint': {'x': 293.9434, 'y': 485.6604}, 'name': 'orange'}, {'id': 'c2fdf254-12fa-43ee-9d04-fd097d6a2fad', 'keypoint': {'x': 332.8585, 'y': 517.5708}, 'name': 'orange'}, {'id': 'ee022624-310e-4867-8360-cd61e6a5e8ec', 'keypoint': {'x': 325.0755, 'y': 557.2642}, 'name': 'orange'}, {'id': 'ca78545f-1817-447e-a767-1ec186e40025', 'keypoint': {'x': 377.2217, 'y': 478.6557}, 'name': 'orange'}, {'id': 'b7ea891b-3095-4df8-a718-b193f34c851c', 'keypoint': {'x': 386.5613, 'y': 492.6651}, 'name': 'orange'}, {'id': '54b85951-9b7d-4a22-9b94-5856c8f4cc7e', 'keypoint': {'x': 441.8208, 'y': 499.6698}, 'name': 'orange'}, {'id': 'ef2b7cbb-fa7e-4894-8e50-310d43a6f3eb', 'keypoint': {'x': 385.0047, 'y': 448.3019}, 'name': 'yellow'}, {'id': 'ab7d8b08-4eaf-41cf-8066-9aba376bb546', 'keypoint': {'x': 392.0094, 'y': 523.0189}, 'name': 'yellow'}, {'id': 'aec059f3-2def-4458-9ea4-3fa5916cab5b', 'keypoint': {'x': 466.7264, 'y': 442.8538}, 'name': 'yellow'}, {'id': '39fe094d-1e93-4828-ae7e-d672aeb88451', 'keypoint': {'x': 356.2075, 'y': 533.9151}, 'name': 'blue'}, {'id': '57cde523-a685-45a5-b623-1b11057840da', 'keypoint': {'x': 487.7406, 'y': 488.7736}, 'name': 'blue'}, {'id': 'da91e8aa-4087-4689-95ed-80e4ba9a93ec', 'keypoint': {'x': 327.4104, 'y': 472.4292}, 'name': 'green'}, {'id': '9406ceb5-4de6-4915-8da6-132b32721d7c', 'keypoint': {'x': 508.7547, 'y': 511.3443}, 'name': 'green'}, {'id': 'fd93334e-be71-4598-9c11-abb001661b87', 'keypoint': {'x': 497.0802, 'y': 560.3774}, 'name': 'green'}, {'id': '3067ae97-d698-4a21-82f6-e94a33ed51a0', 'keypoint': {'x': 380.3349, 'y': 59.1509}, 'name': 'you'}]}
[/INST]
{'x': 508.7547, 'y': 511.3443}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:pick up the second green block from the right
 {'width': 756, 'height': 660, 'annotations': [{'id': 'd5bcc2b7-ff91-4686-8f50-3a73ffd3beeb', 'keypoint': {'x': 315.7358, 'y': 436.6274}, 'name': 'orange'}, {'id': '70515b86-98d5-4fa6-9118-a0bc484f6de1', 'keypoint': {'x': 313.4009, 'y': 525.3538}, 'name': 'orange'}, {'id': 'b40b0958-f706-45d2-a042-0440eb67121b', 'keypoint': {'x': 380.3349, 'y': 515.2358}, 'name': 'orange'}, {'id': '9dcd051f-bad9-4f84-b653-9a30a72cb545', 'keypoint': {'x': 389.6745, 'y': 552.5943}, 'name': 'orange'}, {'id': 'b0de6329-f4d9-486c-91ed-478f28262cff', 'keypoint': {'x': 363.9906, 'y': 439.7406}, 'name': 'yellow'}, {'id': '10eeedc5-3253-42e7-b401-33c4616e4623', 'keypoint': {'x': 396.6792, 'y': 457.6415}, 'name': 'yellow'}, {'id': 'f7642eee-44ca-4f1d-abf7-a79d43dfbd0d', 'keypoint': {'x': 279.934, 'y': 535.4717}, 'name': 'yellow'}, {'id': 'd87c603d-89da-4c0b-8f61-a496275c1297', 'keypoint': {'x': 311.066, 'y': 487.217}, 'name': 'blue'}, {'id': '2065d0e4-12eb-4ab3-bde4-75688bb3c142', 'keypoint': {'x': 457.3868, 'y': 439.7406}, 'name': 'green'}, {'id': '1550df58-31c4-49f4-8e6d-7db4a57bc294', 'keypoint': {'x': 435.5943, 'y': 445.967}, 'name': 'green'}, {'id': '1990bf50-38ac-43ab-8581-26778cea80ed', 'keypoint': {'x': 458.9434, 'y': 484.8821}, 'name': 'green'}, {'id': '7895d85c-2e57-4008-b9d7-74625608a42c', 'keypoint': {'x': 485.4057, 'y': 515.2358}, 'name': 'green'}, {'id': '991a571a-03aa-4a00-b80d-3d6a0845625b', 'keypoint': {'x': 440.2642, 'y': 514.4575}, 'name': 'green'}, {'id': 'e0e1086c-3a47-42f6-98fd-868d0756cd4d', 'keypoint': {'x': 378.7783, 'y': 495.7783}, 'name': 'green'}, {'id': '3d132917-dd96-4ae9-9302-da261696357f', 'keypoint': {'x': 333.6368, 'y': 505.1179}, 'name': 'green'}, {'id': 'bc7494f1-b049-49c6-bf24-85e942969065', 'keypoint': {'x': 383.4481, 'y': 56.0377}, 'name': 'you'}]}
[/INST]
{'x': 378.7783, 'y': 495.7783}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:I would tell the robot to pick up the orange block that is group together with other orange blocks but is closer to me.
 {'width': 756, 'height': 660, 'annotations': [{'id': '53de559f-b2c6-46fa-821d-b5a4e98c67c3', 'keypoint': {'x': 332.0802, 'y': 517.5708}, 'name': 'orange'}, {'id': 'fc5cb394-ebdb-4064-bb12-1d3da5b0cb6f', 'keypoint': {'x': 316.5142, 'y': 540.1415}, 'name': 'orange'}, {'id': 'b8f5d815-90be-4cd5-951f-44207215b594', 'keypoint': {'x': 373.3302, 'y': 537.8066}, 'name': 'orange'}, {'id': '78d88ded-7431-4260-a864-725e2dfef6ab', 'keypoint': {'x': 309.5094, 'y': 498.1132}, 'name': 'yellow'}, {'id': 'e5215008-d111-4bda-9009-f2c0795ffb0d', 'keypoint': {'x': 388.1179, 'y': 471.6509}, 'name': 'yellow'}, {'id': 'b722e1f1-0a3c-47ef-b35f-e9cd85898169', 'keypoint': {'x': 388.1179, 'y': 501.2264}, 'name': 'yellow'}, {'id': '27ad7266-481b-40f9-aaa0-e71dabd71d5e', 'keypoint': {'x': 413.8019, 'y': 491.1085}, 'name': 'yellow'}, {'id': '71348b8e-08dc-4a76-be0e-68cbee1a56a4', 'keypoint': {'x': 416.1368, 'y': 449.8585}, 'name': 'yellow'}, {'id': 'e46d6178-4284-433d-a81f-bc6cbbf7cdbb', 'keypoint': {'x': 490.8538, 'y': 505.1179}, 'name': 'yellow'}, {'id': '272a554a-aad9-4bba-9062-941465beeaa5', 'keypoint': {'x': 465.1698, 'y': 554.1509}, 'name': 'yellow'}, {'id': 'c6ba259b-1c8e-43d4-8bdb-404594f1c175', 'keypoint': {'x': 310.2877, 'y': 438.9623}, 'name': 'blue'}, {'id': '693a6454-810f-447d-8417-a2c57867901d', 'keypoint': {'x': 354.6509, 'y': 477.8774}, 'name': 'blue'}, {'id': '3ec7fa7f-c792-4c99-a423-b40bbb665d89', 'keypoint': {'x': 427.8113, 'y': 428.8443}, 'name': 'blue'}, {'id': '3e041f2e-9e78-4001-9fa0-3bebee80ac07', 'keypoint': {'x': 440.2642, 'y': 498.8915}, 'name': 'green'}, {'id': 'e2e59b33-f4e7-47cd-b137-77a874c834c7', 'keypoint': {'x': 450.3821, 'y': 519.1274}, 'name': 'green'}, {'id': '707959b0-76f6-463a-bc2f-62cd702815f4', 'keypoint': {'x': 377.2217, 'y': 55.2594}, 'name': 'you'}]}
[/INST]
{'x': 332.0802, 'y': 517.5708}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the green block right in front of the two blue blocks that are furthest away from you.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'f8ce6e33-d9e3-4687-bd11-4537937b3def', 'keypoint': {'x': 313.4009, 'y': 480.9906}, 'name': 'orange'}, {'id': '8758b8e0-78a7-4dc2-b595-9eec97d96b3c', 'keypoint': {'x': 403.684, 'y': 466.9811}, 'name': 'orange'}, {'id': '9024f068-b46e-400e-bf69-507f61b74f80', 'keypoint': {'x': 459.7217, 'y': 452.9717}, 'name': 'orange'}, {'id': 'd7ab991d-e3d6-4ffb-8d9b-9cffbb59b9c6', 'keypoint': {'x': 502.5283, 'y': 457.6415}, 'name': 'orange'}, {'id': 'b9fb860f-d2d0-4b4a-a275-416d46a99ab8', 'keypoint': {'x': 335.9717, 'y': 450.6368}, 'name': 'yellow'}, {'id': '7404a736-3ae3-4f5c-b40d-5d5af4635c6e', 'keypoint': {'x': 316.5142, 'y': 531.5802}, 'name': 'yellow'}, {'id': 'd2d6b9ff-fd85-48a1-b368-3f0e1d09cc6f', 'keypoint': {'x': 396.6792, 'y': 491.8868}, 'name': 'yellow'}, {'id': 'c4551bcd-78f8-4205-b828-5b040d49a2c3', 'keypoint': {'x': 444.1557, 'y': 433.5142}, 'name': 'yellow'}, {'id': '3e0cda0a-1e8a-4472-8984-7886c02ff560', 'keypoint': {'x': 347.6462, 'y': 514.4575}, 'name': 'blue'}, {'id': '266b6be1-582b-4c4d-8544-906fad90cbd2', 'keypoint': {'x': 359.3208, 'y': 478.6557}, 'name': 'blue'}, {'id': 'c58796fa-46b4-4231-8589-0612cd52983f', 'keypoint': {'x': 395.9009, 'y': 444.4104}, 'name': 'blue'}, {'id': '3de58fe6-02ae-4e5a-b0eb-ac1e3ac60960', 'keypoint': {'x': 444.934, 'y': 537.8066}, 'name': 'blue'}, {'id': 'a1a96c53-7f31-4121-8983-bf6a69f42935', 'keypoint': {'x': 502.5283, 'y': 541.6981}, 'name': 'blue'}, {'id': 'c2e223f1-9da7-4029-ab29-81edb6be3c94', 'keypoint': {'x': 355.4292, 'y': 424.9528}, 'name': 'green'}, {'id': 'd1f0facf-050a-4a92-bfb1-1367c5fdd80d', 'keypoint': {'x': 466.7264, 'y': 500.4481}, 'name': 'green'}, {'id': '727b2df8-1454-47db-9eff-8cfe072974ff', 'keypoint': {'x': 383.4481, 'y': 44.3632}, 'name': 'you'}]}
[/INST]
{'x': 466.7264, 'y': 500.4481}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Do you see the two blue blocks closest to me? Grab the one on your left side.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'c7519096-f7ea-4ab1-a66d-97c708976599', 'keypoint': {'x': 380.3349, 'y': 34.2453}, 'name': 'you'}, {'id': 'a95968ee-51d4-482d-ae79-97cb54663aed', 'keypoint': {'x': 293.9434, 'y': 479.434}, 'name': 'blue'}, {'id': '2fc8b299-6074-43eb-92bc-7ed758a958fc', 'keypoint': {'x': 329.7453, 'y': 557.2642}, 'name': 'blue'}, {'id': '11dbc045-de4f-4d3a-bc50-a1c2ba279bd1', 'keypoint': {'x': 366.3255, 'y': 546.3679}, 'name': 'blue'}, {'id': 'a3036737-fb7e-4f86-88c8-b0c5262cc051', 'keypoint': {'x': 379.5566, 'y': 438.9623}, 'name': 'blue'}, {'id': '5370f3f3-ef52-45e8-bdb4-3d96d54f4bd9', 'keypoint': {'x': 455.8302, 'y': 449.0802}, 'name': 'blue'}, {'id': '95294165-9354-41e7-b1be-60d4033bd1a3', 'keypoint': {'x': 318.0708, 'y': 454.5283}, 'name': 'yellow'}, {'id': '71f49ac9-1905-457f-9219-701cac7c42e4', 'keypoint': {'x': 318.0708, 'y': 530.8019}, 'name': 'yellow'}, {'id': '6f7a8a19-794f-4ffc-80ed-8a42dedeab0f', 'keypoint': {'x': 463.6132, 'y': 524.5755}, 'name': 'yellow'}, {'id': '407068c6-6a1e-4a14-b9d0-c0a4ed13b9c2', 'keypoint': {'x': 395.1226, 'y': 466.2028}, 'name': 'green'}, {'id': '847fb0f9-02cc-42df-8984-2b193e2abb8d', 'keypoint': {'x': 403.684, 'y': 523.0189}, 'name': 'green'}, {'id': 'bd78738b-2c39-4b85-9575-960d46ecd66c', 'keypoint': {'x': 325.0755, 'y': 500.4481}, 'name': 'orange'}, {'id': 'dafb3b80-4c1e-48a7-b398-e3aa56c5dcb0', 'keypoint': {'x': 374.1085, 'y': 515.2358}, 'name': 'orange'}, {'id': '36cf3f26-cac5-4368-91eb-ae8937ec2f91', 'keypoint': {'x': 412.2453, 'y': 427.2877}, 'name': 'orange'}, {'id': 'f93ff614-2250-4f4d-8ec0-ac87ec62305c', 'keypoint': {'x': 436.3726, 'y': 437.4057}, 'name': 'orange'}, {'id': '09066da9-88d0-4617-8cf6-7d604ee0190d', 'keypoint': {'x': 485.4057, 'y': 510.566}, 'name': 'orange'}]}
[/INST]
{'x': 366.3255, 'y': 546.3679}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:you need to pick up the blue block sitting right behind the green block closest to you
 {'width': 756, 'height': 660, 'annotations': [{'id': 'bb2508d6-bab0-4991-a4b4-1d60addd89be', 'keypoint': {'x': 374.8868, 'y': 32.6887}, 'name': 'you'}, {'id': '80bad823-3c11-42a6-9b7a-8f92fd053db8', 'keypoint': {'x': 286.1604, 'y': 504.3396}, 'name': 'yellow'}, {'id': '74407b61-1d02-4456-a3be-f44e6d0621e4', 'keypoint': {'x': 427.8113, 'y': 511.3443}, 'name': 'yellow'}, {'id': '0c043462-b620-40d3-b17e-3f17c7108248', 'keypoint': {'x': 391.2311, 'y': 435.8491}, 'name': 'green'}, {'id': 'e97e7600-f671-4774-970c-bcd8e011471e', 'keypoint': {'x': 311.066, 'y': 530.8019}, 'name': 'green'}, {'id': '3c45bdee-a533-4cab-8e50-cd3e05ea379a', 'keypoint': {'x': 375.6651, 'y': 518.3491}, 'name': 'green'}, {'id': 'fbe906a6-d83a-45bc-bd1e-7c81ccde8b9b', 'keypoint': {'x': 487.7406, 'y': 473.2075}, 'name': 'green'}, {'id': '662455ed-994d-45d8-b864-0f07ccfe9a1e', 'keypoint': {'x': 309.5094, 'y': 477.8774}, 'name': 'blue'}, {'id': '92e619ee-cc43-4ba6-80a0-fd23cabc2b68', 'keypoint': {'x': 378.7783, 'y': 457.6415}, 'name': 'blue'}, {'id': '4d3a4397-37ee-4ea1-bff7-ea4e40e4ba91', 'keypoint': {'x': 395.1226, 'y': 483.3255}, 'name': 'blue'}, {'id': '83c6c716-2497-4e0e-b233-3cda2dca8049', 'keypoint': {'x': 438.7075, 'y': 448.3019}, 'name': 'blue'}, {'id': '24b23d82-8281-40fd-9942-dc852b76fea4', 'keypoint': {'x': 444.1557, 'y': 484.1038}, 'name': 'blue'}, {'id': '541179c1-e37f-4069-99a8-fe58be3053c6', 'keypoint': {'x': 466.7264, 'y': 502.783}, 'name': 'blue'}, {'id': 'd950b6f4-0fff-425b-bf69-419e3a15863f', 'keypoint': {'x': 343.7547, 'y': 543.2547}, 'name': 'orange'}, {'id': 'a5aa3394-3b02-4613-8a7d-f14c9cd0fa58', 'keypoint': {'x': 431.7028, 'y': 539.3632}, 'name': 'orange'}, {'id': 'd7eb7db6-f69d-4df7-861e-0cc9dbebac2a', 'keypoint': {'x': 484.6274, 'y': 532.3585}, 'name': 'orange'}]}
[/INST]
{'x': 378.7783, 'y': 457.6415}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the yellow block.
 {'width': 756, 'height': 660, 'annotations': [{'id': '4665780d-e410-4494-8512-3557afddb1a4', 'keypoint': {'x': 311.8443, 'y': 504.3396}, 'name': None}, {'id': '565e6ab8-2ba1-4136-96ed-e63f3cdba23d', 'keypoint': {'x': 368.6604, 'y': 505.1179}, 'name': 'orange'}, {'id': '487a86d1-7908-404f-99e9-16ab87ea0d04', 'keypoint': {'x': 359.3208, 'y': 560.3774}, 'name': 'orange'}, {'id': 'ab7b691c-c47b-4c26-8083-096140fac645', 'keypoint': {'x': 413.0236, 'y': 447.5236}, 'name': 'orange'}, {'id': 'fdb6ba1e-6ab3-4d82-862c-1bdaf73940dc', 'keypoint': {'x': 464.3915, 'y': 459.9764}, 'name': 'orange'}, {'id': '7768e90f-d1af-4f66-abc0-01fa10bc5e61', 'keypoint': {'x': 447.2689, 'y': 530.8019}, 'name': 'orange'}, {'id': '698efbba-5197-4d9f-96eb-fdc8c950c48f', 'keypoint': {'x': 290.8302, 'y': 554.1509}, 'name': 'yellow'}, {'id': '1e9d76c9-a045-4417-94ae-129e3daeb548', 'keypoint': {'x': 407.5755, 'y': 519.9057}, 'name': 'blue'}, {'id': 'dc0ba9e0-8eb1-478a-88c5-3eb00d02992d', 'keypoint': {'x': 454.2736, 'y': 505.8962}, 'name': 'blue'}, {'id': '13165129-2db5-428d-ba05-982c1ae9a179', 'keypoint': {'x': 503.3066, 'y': 487.217}, 'name': 'green'}, {'id': 'a2feb4b6-353e-4876-b8e6-d5351bc5f056', 'keypoint': {'x': 459.7217, 'y': 490.3302}, 'name': 'green'}, {'id': 'bd636fd9-8ba5-408a-a286-413978cb55e7', 'keypoint': {'x': 362.434, 'y': 476.3208}, 'name': 'green'}, {'id': '4d316993-cf9a-4115-894b-940d569fafc4', 'keypoint': {'x': 311.066, 'y': 439.7406}, 'name': 'green'}, {'id': 'b6f5df26-1415-40e4-bbf7-01af31da49b9', 'keypoint': {'x': 380.3349, 'y': 544.8113}, 'name': 'green'}, {'id': 'f34f5fb5-74bd-4265-8123-8eed80fffdd5', 'keypoint': {'x': 322.7406, 'y': 536.25}, 'name': 'green'}, {'id': '847a9a8d-0e7a-49f2-b274-f833cdac716e', 'keypoint': {'x': 375.6651, 'y': 49.8113}, 'name': 'you'}]}
[/INST]
{'x': 290.8302, 'y': 554.1509}</s>