<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the green block that has 2 yellow boxes in front of it and 2 orange blocks behind it.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'd5bcc2b7-ff91-4686-8f50-3a73ffd3beeb', 'keypoint': {'x': 315.7358, 'y': 436.6274}, 'name': 'orange'}, {'id': '70515b86-98d5-4fa6-9118-a0bc484f6de1', 'keypoint': {'x': 313.4009, 'y': 525.3538}, 'name': 'orange'}, {'id': 'b40b0958-f706-45d2-a042-0440eb67121b', 'keypoint': {'x': 380.3349, 'y': 515.2358}, 'name': 'orange'}, {'id': '9dcd051f-bad9-4f84-b653-9a30a72cb545', 'keypoint': {'x': 389.6745, 'y': 552.5943}, 'name': 'orange'}, {'id': 'b0de6329-f4d9-486c-91ed-478f28262cff', 'keypoint': {'x': 363.9906, 'y': 439.7406}, 'name': 'yellow'}, {'id': '10eeedc5-3253-42e7-b401-33c4616e4623', 'keypoint': {'x': 396.6792, 'y': 457.6415}, 'name': 'yellow'}, {'id': 'f7642eee-44ca-4f1d-abf7-a79d43dfbd0d', 'keypoint': {'x': 279.934, 'y': 535.4717}, 'name': 'yellow'}, {'id': 'd87c603d-89da-4c0b-8f61-a496275c1297', 'keypoint': {'x': 311.066, 'y': 487.217}, 'name': 'blue'}, {'id': '2065d0e4-12eb-4ab3-bde4-75688bb3c142', 'keypoint': {'x': 457.3868, 'y': 439.7406}, 'name': 'green'}, {'id': '1550df58-31c4-49f4-8e6d-7db4a57bc294', 'keypoint': {'x': 435.5943, 'y': 445.967}, 'name': 'green'}, {'id': '1990bf50-38ac-43ab-8581-26778cea80ed', 'keypoint': {'x': 458.9434, 'y': 484.8821}, 'name': 'green'}, {'id': '7895d85c-2e57-4008-b9d7-74625608a42c', 'keypoint': {'x': 485.4057, 'y': 515.2358}, 'name': 'green'}, {'id': '991a571a-03aa-4a00-b80d-3d6a0845625b', 'keypoint': {'x': 440.2642, 'y': 514.4575}, 'name': 'green'}, {'id': 'e0e1086c-3a47-42f6-98fd-868d0756cd4d', 'keypoint': {'x': 378.7783, 'y': 495.7783}, 'name': 'green'}, {'id': '3d132917-dd96-4ae9-9302-da261696357f', 'keypoint': {'x': 333.6368, 'y': 505.1179}, 'name': 'green'}, {'id': 'bc7494f1-b049-49c6-bf24-85e942969065', 'keypoint': {'x': 383.4481, 'y': 56.0377}, 'name': 'you'}]}
[/INST]
{'x': 378.7783, 'y': 495.7783}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the yellow block furthest away from you.
 {'width': 756, 'height': 660, 'annotations': [{'id': '82770085-a20a-439c-b1a3-f74a8d8ccdd0', 'keypoint': {'x': 322.0293, 'y': 491.9856}, 'name': 'yellow'}, {'id': '975d2226-07a6-46ec-8aa1-01f44597bb5c', 'keypoint': {'x': 376.8281, 'y': 495.1624}, 'name': 'yellow'}, {'id': '9fbd0dd7-292a-46c1-854e-eb8c4bb880cf', 'keypoint': {'x': 402.2421, 'y': 551.5495}, 'name': 'yellow'}, {'id': 'c2d61b6b-0585-4aef-acaf-17bad430192b', 'keypoint': {'x': 464.9827, 'y': 498.3391}, 'name': 'yellow'}, {'id': '13d3414e-a81b-4dad-8963-6fbda2f0d436', 'keypoint': {'x': 376.0339, 'y': 27.387}, 'name': 'you'}, {'id': '9b80a1cd-fed1-49e9-8b69-3cd0088f7fca', 'keypoint': {'x': 302.9689, 'y': 454.6589}, 'name': 'blue'}, {'id': '17a9c4b0-bf5e-45d6-aa6f-94959ad755e2', 'keypoint': {'x': 421.3025, 'y': 473.7194}, 'name': 'blue'}, {'id': '4a1722fe-5d45-4bf9-a201-c0b8f7acc36b', 'keypoint': {'x': 456.2467, 'y': 437.981}, 'name': 'blue'}, {'id': '680351fd-2723-4f71-9103-17bb8a809738', 'keypoint': {'x': 375.2398, 'y': 452.2763}, 'name': 'orange'}, {'id': '188969f7-812f-47f7-a97e-5a0652f78c64', 'keypoint': {'x': 354.5909, 'y': 508.6635}, 'name': 'orange'}, {'id': 'b57e9350-23ed-40aa-96cc-2811f14cd796', 'keypoint': {'x': 396.6828, 'y': 520.5763}, 'name': 'orange'}, {'id': 'f041b6b6-0396-42fe-99f5-46de7b8e6b23', 'keypoint': {'x': 431.6269, 'y': 507.8693}, 'name': 'orange'}, {'id': 'ae76b953-f8b0-4121-9569-ce66605c9783', 'keypoint': {'x': 433.2153, 'y': 530.9007}, 'name': 'orange'}, {'id': '256e0a15-32e1-445f-a768-a8adb00d4905', 'keypoint': {'x': 341.0898, 'y': 438.7752}, 'name': 'green'}, {'id': '51ccd976-c149-458d-960f-bd00da3a5131', 'keypoint': {'x': 318.0584, 'y': 536.46}, 'name': 'green'}, {'id': 'd3e5315b-8e96-4659-97cb-07d656a65db8', 'keypoint': {'x': 503.8978, 'y': 456.2473}, 'name': 'green'}]}
[/INST]
{'x': 402.2421, 'y': 551.5495}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the green block to your left that has two green blocks in front of it and two green block behind it.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'd5bcc2b7-ff91-4686-8f50-3a73ffd3beeb', 'keypoint': {'x': 315.7358, 'y': 436.6274}, 'name': 'orange'}, {'id': '70515b86-98d5-4fa6-9118-a0bc484f6de1', 'keypoint': {'x': 313.4009, 'y': 525.3538}, 'name': 'orange'}, {'id': 'b40b0958-f706-45d2-a042-0440eb67121b', 'keypoint': {'x': 380.3349, 'y': 515.2358}, 'name': 'orange'}, {'id': '9dcd051f-bad9-4f84-b653-9a30a72cb545', 'keypoint': {'x': 389.6745, 'y': 552.5943}, 'name': 'orange'}, {'id': 'b0de6329-f4d9-486c-91ed-478f28262cff', 'keypoint': {'x': 363.9906, 'y': 439.7406}, 'name': 'yellow'}, {'id': '10eeedc5-3253-42e7-b401-33c4616e4623', 'keypoint': {'x': 396.6792, 'y': 457.6415}, 'name': 'yellow'}, {'id': 'f7642eee-44ca-4f1d-abf7-a79d43dfbd0d', 'keypoint': {'x': 279.934, 'y': 535.4717}, 'name': 'yellow'}, {'id': 'd87c603d-89da-4c0b-8f61-a496275c1297', 'keypoint': {'x': 311.066, 'y': 487.217}, 'name': 'blue'}, {'id': '2065d0e4-12eb-4ab3-bde4-75688bb3c142', 'keypoint': {'x': 457.3868, 'y': 439.7406}, 'name': 'green'}, {'id': '1550df58-31c4-49f4-8e6d-7db4a57bc294', 'keypoint': {'x': 435.5943, 'y': 445.967}, 'name': 'green'}, {'id': '1990bf50-38ac-43ab-8581-26778cea80ed', 'keypoint': {'x': 458.9434, 'y': 484.8821}, 'name': 'green'}, {'id': '7895d85c-2e57-4008-b9d7-74625608a42c', 'keypoint': {'x': 485.4057, 'y': 515.2358}, 'name': 'green'}, {'id': '991a571a-03aa-4a00-b80d-3d6a0845625b', 'keypoint': {'x': 440.2642, 'y': 514.4575}, 'name': 'green'}, {'id': 'e0e1086c-3a47-42f6-98fd-868d0756cd4d', 'keypoint': {'x': 378.7783, 'y': 495.7783}, 'name': 'green'}, {'id': '3d132917-dd96-4ae9-9302-da261696357f', 'keypoint': {'x': 333.6368, 'y': 505.1179}, 'name': 'green'}, {'id': 'bc7494f1-b049-49c6-bf24-85e942969065', 'keypoint': {'x': 383.4481, 'y': 56.0377}, 'name': 'you'}]}
[/INST]
{'x': 458.9434, 'y': 484.8821}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the blue block that is directly to the right of the furthest yellow block on your left.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'a7c6f9ab-55b7-422b-8e7f-70902f2e94d6', 'keypoint': {'x': 320.4057, 'y': 488.7736}, 'name': 'orange'}, {'id': 'acc187bd-2783-4707-be6a-314a58518228', 'keypoint': {'x': 423.9198, 'y': 451.4151}, 'name': 'orange'}, {'id': '7b9652c2-a499-48cc-ab07-68e686e885ac', 'keypoint': {'x': 358.5425, 'y': 462.3113}, 'name': 'yellow'}, {'id': '6e4e6633-28b0-4e37-a08b-d90cdb596a0d', 'keypoint': {'x': 334.4151, 'y': 548.7028}, 'name': 'yellow'}, {'id': 'dc28f152-a36a-458c-9f8a-21f4093b66bd', 'keypoint': {'x': 418.4717, 'y': 547.9245}, 'name': 'yellow'}, {'id': '34029874-d3f4-4998-a6fa-85ee314b35b5', 'keypoint': {'x': 479.1792, 'y': 484.8821}, 'name': 'yellow'}, {'id': '3d4114df-2e6e-4d36-8e12-47bbffbd6150', 'keypoint': {'x': 314.1792, 'y': 452.1934}, 'name': 'blue'}, {'id': '021cc2a3-2e83-4079-9777-64d3bc57e829', 'keypoint': {'x': 293.1651, 'y': 557.2642}, 'name': 'blue'}, {'id': '49911b5f-df0a-4694-9ff5-d8165f0f587a', 'keypoint': {'x': 390.4528, 'y': 513.6792}, 'name': 'blue'}, {'id': '95b20075-c6a2-46e0-b203-f8004a3ee4a3', 'keypoint': {'x': 440.2642, 'y': 484.8821}, 'name': 'blue'}, {'id': '031e7d06-9b93-4602-a1ef-c48bce3518c4', 'keypoint': {'x': 462.0566, 'y': 469.316}, 'name': 'blue'}, {'id': '3f2fdd03-f2f9-4892-999b-713f9e988be6', 'keypoint': {'x': 493.1887, 'y': 465.4245}, 'name': 'blue'}, {'id': '7ed0f47f-3bdf-429a-ab98-1d1a9d3058fa', 'keypoint': {'x': 482.2925, 'y': 507.4528}, 'name': 'green'}, {'id': '1fb5251e-aa36-4074-87f5-edef1e5a25c7', 'keypoint': {'x': 491.6321, 'y': 542.4764}, 'name': 'green'}, {'id': 'e7cf6134-82a1-4efe-99a5-a098dcdb3a3e', 'keypoint': {'x': 334.4151, 'y': 516.0142}, 'name': 'green'}, {'id': '0aeb291a-183c-47ce-9552-8b4d4860c05f', 'keypoint': {'x': 376.4434, 'y': 44.3632}, 'name': 'you'}]}
[/INST]
{'x': 440.2642, 'y': 484.8821}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the blue block behind the middle one that's closest to you.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'a7c6f9ab-55b7-422b-8e7f-70902f2e94d6', 'keypoint': {'x': 320.4057, 'y': 488.7736}, 'name': 'orange'}, {'id': 'acc187bd-2783-4707-be6a-314a58518228', 'keypoint': {'x': 423.9198, 'y': 451.4151}, 'name': 'orange'}, {'id': '7b9652c2-a499-48cc-ab07-68e686e885ac', 'keypoint': {'x': 358.5425, 'y': 462.3113}, 'name': 'yellow'}, {'id': '6e4e6633-28b0-4e37-a08b-d90cdb596a0d', 'keypoint': {'x': 334.4151, 'y': 548.7028}, 'name': 'yellow'}, {'id': 'dc28f152-a36a-458c-9f8a-21f4093b66bd', 'keypoint': {'x': 418.4717, 'y': 547.9245}, 'name': 'yellow'}, {'id': '34029874-d3f4-4998-a6fa-85ee314b35b5', 'keypoint': {'x': 479.1792, 'y': 484.8821}, 'name': 'yellow'}, {'id': '3d4114df-2e6e-4d36-8e12-47bbffbd6150', 'keypoint': {'x': 314.1792, 'y': 452.1934}, 'name': 'blue'}, {'id': '021cc2a3-2e83-4079-9777-64d3bc57e829', 'keypoint': {'x': 293.1651, 'y': 557.2642}, 'name': 'blue'}, {'id': '49911b5f-df0a-4694-9ff5-d8165f0f587a', 'keypoint': {'x': 390.4528, 'y': 513.6792}, 'name': 'blue'}, {'id': '95b20075-c6a2-46e0-b203-f8004a3ee4a3', 'keypoint': {'x': 440.2642, 'y': 484.8821}, 'name': 'blue'}, {'id': '031e7d06-9b93-4602-a1ef-c48bce3518c4', 'keypoint': {'x': 462.0566, 'y': 469.316}, 'name': 'blue'}, {'id': '3f2fdd03-f2f9-4892-999b-713f9e988be6', 'keypoint': {'x': 493.1887, 'y': 465.4245}, 'name': 'blue'}, {'id': '7ed0f47f-3bdf-429a-ab98-1d1a9d3058fa', 'keypoint': {'x': 482.2925, 'y': 507.4528}, 'name': 'green'}, {'id': '1fb5251e-aa36-4074-87f5-edef1e5a25c7', 'keypoint': {'x': 491.6321, 'y': 542.4764}, 'name': 'green'}, {'id': 'e7cf6134-82a1-4efe-99a5-a098dcdb3a3e', 'keypoint': {'x': 334.4151, 'y': 516.0142}, 'name': 'green'}, {'id': '0aeb291a-183c-47ce-9552-8b4d4860c05f', 'keypoint': {'x': 376.4434, 'y': 44.3632}, 'name': 'you'}]}
[/INST]
{'x': 440.2642, 'y': 484.8821}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the yellow block that is farthest from you.
 {'width': 756, 'height': 660, 'annotations': [{'id': '82770085-a20a-439c-b1a3-f74a8d8ccdd0', 'keypoint': {'x': 322.0293, 'y': 491.9856}, 'name': 'yellow'}, {'id': '975d2226-07a6-46ec-8aa1-01f44597bb5c', 'keypoint': {'x': 376.8281, 'y': 495.1624}, 'name': 'yellow'}, {'id': '9fbd0dd7-292a-46c1-854e-eb8c4bb880cf', 'keypoint': {'x': 402.2421, 'y': 551.5495}, 'name': 'yellow'}, {'id': 'c2d61b6b-0585-4aef-acaf-17bad430192b', 'keypoint': {'x': 464.9827, 'y': 498.3391}, 'name': 'yellow'}, {'id': '13d3414e-a81b-4dad-8963-6fbda2f0d436', 'keypoint': {'x': 376.0339, 'y': 27.387}, 'name': 'you'}, {'id': '9b80a1cd-fed1-49e9-8b69-3cd0088f7fca', 'keypoint': {'x': 302.9689, 'y': 454.6589}, 'name': 'blue'}, {'id': '17a9c4b0-bf5e-45d6-aa6f-94959ad755e2', 'keypoint': {'x': 421.3025, 'y': 473.7194}, 'name': 'blue'}, {'id': '4a1722fe-5d45-4bf9-a201-c0b8f7acc36b', 'keypoint': {'x': 456.2467, 'y': 437.981}, 'name': 'blue'}, {'id': '680351fd-2723-4f71-9103-17bb8a809738', 'keypoint': {'x': 375.2398, 'y': 452.2763}, 'name': 'orange'}, {'id': '188969f7-812f-47f7-a97e-5a0652f78c64', 'keypoint': {'x': 354.5909, 'y': 508.6635}, 'name': 'orange'}, {'id': 'b57e9350-23ed-40aa-96cc-2811f14cd796', 'keypoint': {'x': 396.6828, 'y': 520.5763}, 'name': 'orange'}, {'id': 'f041b6b6-0396-42fe-99f5-46de7b8e6b23', 'keypoint': {'x': 431.6269, 'y': 507.8693}, 'name': 'orange'}, {'id': 'ae76b953-f8b0-4121-9569-ce66605c9783', 'keypoint': {'x': 433.2153, 'y': 530.9007}, 'name': 'orange'}, {'id': '256e0a15-32e1-445f-a768-a8adb00d4905', 'keypoint': {'x': 341.0898, 'y': 438.7752}, 'name': 'green'}, {'id': '51ccd976-c149-458d-960f-bd00da3a5131', 'keypoint': {'x': 318.0584, 'y': 536.46}, 'name': 'green'}, {'id': 'd3e5315b-8e96-4659-97cb-07d656a65db8', 'keypoint': {'x': 503.8978, 'y': 456.2473}, 'name': 'green'}]}
[/INST]
{'x': 402.2421, 'y': 551.5495}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up ht blue block to the bottom left of the rightmost yellow block.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'fe98e63d-c226-4a25-9e75-229c9a2d828a', 'keypoint': {'x': 377.2217, 'y': 28.7972}, 'name': 'you'}, {'id': 'c1bfc6fe-9e0d-4152-aab0-9efe03f99ee1', 'keypoint': {'x': 291.6085, 'y': 487.217}, 'name': 'yellow'}, {'id': 'eefd5787-08ef-4674-ae9e-c41f4a2ed951', 'keypoint': {'x': 314.1792, 'y': 539.3632}, 'name': 'yellow'}, {'id': '8987e25d-8eb0-42ab-aa8e-390a7b046e44', 'keypoint': {'x': 363.2123, 'y': 544.8113}, 'name': 'yellow'}, {'id': 'af54d764-7469-4b39-95f2-d25ac6241715', 'keypoint': {'x': 434.0377, 'y': 542.4764}, 'name': 'yellow'}, {'id': 'c426887c-43fa-4c7c-9ee6-447f48c5b353', 'keypoint': {'x': 463.6132, 'y': 487.9953}, 'name': 'yellow'}, {'id': 'f1402f90-fe95-42c0-93e5-75019494beba', 'keypoint': {'x': 497.0802, 'y': 442.0755}, 'name': 'yellow'}, {'id': 'ad7b2c3b-efac-4705-8dc4-076514da9f0d', 'keypoint': {'x': 319.6274, 'y': 463.8679}, 'name': 'blue'}, {'id': '57fd11a6-5352-4e90-8c5e-78e080a9f63b', 'keypoint': {'x': 458.1651, 'y': 516.0142}, 'name': 'blue'}, {'id': 'bffb87fb-647e-44e1-b4c0-9bbf464cc4da', 'keypoint': {'x': 353.8726, 'y': 445.1887}, 'name': 'green'}, {'id': '2be751d9-34f0-411b-89e7-1b20b054e1ee', 'keypoint': {'x': 388.8962, 'y': 498.1132}, 'name': 'green'}, {'id': '98a41430-e570-492e-969a-8ea45857d765', 'keypoint': {'x': 395.9009, 'y': 533.1368}, 'name': 'green'}, {'id': '0327cb90-c17c-4528-8964-e978fbff5999', 'keypoint': {'x': 427.8113, 'y': 484.8821}, 'name': 'green'}, {'id': '97f91dd8-6cd8-4194-8783-e9e3b5786e68', 'keypoint': {'x': 456.6085, 'y': 452.9717}, 'name': 'green'}, {'id': '4ebadfd4-3497-4f80-a4e9-a870c25786bd', 'keypoint': {'x': 504.8632, 'y': 502.0047}, 'name': 'green'}, {'id': 'd2ff782c-210c-4090-91a5-de43a78f1232', 'keypoint': {'x': 331.3019, 'y': 512.9009}, 'name': 'orange'}]}
[/INST]
{'x': 319.6274, 'y': 463.8679}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:please pick up the blue block on the right
 {'width': 756, 'height': 660, 'annotations': [{'id': '8f941e4d-8e6b-4e37-a2cd-f19b3dbb1dda', 'keypoint': {'x': 293.1651, 'y': 452.9717}, 'name': 'orange'}, {'id': '77f8b560-0c54-4923-9c65-8e8973d5cc1a', 'keypoint': {'x': 293.9434, 'y': 485.6604}, 'name': 'orange'}, {'id': 'c2fdf254-12fa-43ee-9d04-fd097d6a2fad', 'keypoint': {'x': 332.8585, 'y': 517.5708}, 'name': 'orange'}, {'id': 'ee022624-310e-4867-8360-cd61e6a5e8ec', 'keypoint': {'x': 325.0755, 'y': 557.2642}, 'name': 'orange'}, {'id': 'ca78545f-1817-447e-a767-1ec186e40025', 'keypoint': {'x': 377.2217, 'y': 478.6557}, 'name': 'orange'}, {'id': 'b7ea891b-3095-4df8-a718-b193f34c851c', 'keypoint': {'x': 386.5613, 'y': 492.6651}, 'name': 'orange'}, {'id': '54b85951-9b7d-4a22-9b94-5856c8f4cc7e', 'keypoint': {'x': 441.8208, 'y': 499.6698}, 'name': 'orange'}, {'id': 'ef2b7cbb-fa7e-4894-8e50-310d43a6f3eb', 'keypoint': {'x': 385.0047, 'y': 448.3019}, 'name': 'yellow'}, {'id': 'ab7d8b08-4eaf-41cf-8066-9aba376bb546', 'keypoint': {'x': 392.0094, 'y': 523.0189}, 'name': 'yellow'}, {'id': 'aec059f3-2def-4458-9ea4-3fa5916cab5b', 'keypoint': {'x': 466.7264, 'y': 442.8538}, 'name': 'yellow'}, {'id': '39fe094d-1e93-4828-ae7e-d672aeb88451', 'keypoint': {'x': 356.2075, 'y': 533.9151}, 'name': 'blue'}, {'id': '57cde523-a685-45a5-b623-1b11057840da', 'keypoint': {'x': 487.7406, 'y': 488.7736}, 'name': 'blue'}, {'id': 'da91e8aa-4087-4689-95ed-80e4ba9a93ec', 'keypoint': {'x': 327.4104, 'y': 472.4292}, 'name': 'green'}, {'id': '9406ceb5-4de6-4915-8da6-132b32721d7c', 'keypoint': {'x': 508.7547, 'y': 511.3443}, 'name': 'green'}, {'id': 'fd93334e-be71-4598-9c11-abb001661b87', 'keypoint': {'x': 497.0802, 'y': 560.3774}, 'name': 'green'}, {'id': '3067ae97-d698-4a21-82f6-e94a33ed51a0', 'keypoint': {'x': 380.3349, 'y': 59.1509}, 'name': 'you'}]}
[/INST]
{'x': 487.7406, 'y': 488.7736}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the blue block that is behind the green block closest to you.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'bb2508d6-bab0-4991-a4b4-1d60addd89be', 'keypoint': {'x': 374.8868, 'y': 32.6887}, 'name': 'you'}, {'id': '80bad823-3c11-42a6-9b7a-8f92fd053db8', 'keypoint': {'x': 286.1604, 'y': 504.3396}, 'name': 'yellow'}, {'id': '74407b61-1d02-4456-a3be-f44e6d0621e4', 'keypoint': {'x': 427.8113, 'y': 511.3443}, 'name': 'yellow'}, {'id': '0c043462-b620-40d3-b17e-3f17c7108248', 'keypoint': {'x': 391.2311, 'y': 435.8491}, 'name': 'green'}, {'id': 'e97e7600-f671-4774-970c-bcd8e011471e', 'keypoint': {'x': 311.066, 'y': 530.8019}, 'name': 'green'}, {'id': '3c45bdee-a533-4cab-8e50-cd3e05ea379a', 'keypoint': {'x': 375.6651, 'y': 518.3491}, 'name': 'green'}, {'id': 'fbe906a6-d83a-45bc-bd1e-7c81ccde8b9b', 'keypoint': {'x': 487.7406, 'y': 473.2075}, 'name': 'green'}, {'id': '662455ed-994d-45d8-b864-0f07ccfe9a1e', 'keypoint': {'x': 309.5094, 'y': 477.8774}, 'name': 'blue'}, {'id': '92e619ee-cc43-4ba6-80a0-fd23cabc2b68', 'keypoint': {'x': 378.7783, 'y': 457.6415}, 'name': 'blue'}, {'id': '4d3a4397-37ee-4ea1-bff7-ea4e40e4ba91', 'keypoint': {'x': 395.1226, 'y': 483.3255}, 'name': 'blue'}, {'id': '83c6c716-2497-4e0e-b233-3cda2dca8049', 'keypoint': {'x': 438.7075, 'y': 448.3019}, 'name': 'blue'}, {'id': '24b23d82-8281-40fd-9942-dc852b76fea4', 'keypoint': {'x': 444.1557, 'y': 484.1038}, 'name': 'blue'}, {'id': '541179c1-e37f-4069-99a8-fe58be3053c6', 'keypoint': {'x': 466.7264, 'y': 502.783}, 'name': 'blue'}, {'id': 'd950b6f4-0fff-425b-bf69-419e3a15863f', 'keypoint': {'x': 343.7547, 'y': 543.2547}, 'name': 'orange'}, {'id': 'a5aa3394-3b02-4613-8a7d-f14c9cd0fa58', 'keypoint': {'x': 431.7028, 'y': 539.3632}, 'name': 'orange'}, {'id': 'd7eb7db6-f69d-4df7-861e-0cc9dbebac2a', 'keypoint': {'x': 484.6274, 'y': 532.3585}, 'name': 'orange'}]}
[/INST]
{'x': 378.7783, 'y': 457.6415}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:From your perspective i would choose the second blue block to the left.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'a7c6f9ab-55b7-422b-8e7f-70902f2e94d6', 'keypoint': {'x': 320.4057, 'y': 488.7736}, 'name': 'orange'}, {'id': 'acc187bd-2783-4707-be6a-314a58518228', 'keypoint': {'x': 423.9198, 'y': 451.4151}, 'name': 'orange'}, {'id': '7b9652c2-a499-48cc-ab07-68e686e885ac', 'keypoint': {'x': 358.5425, 'y': 462.3113}, 'name': 'yellow'}, {'id': '6e4e6633-28b0-4e37-a08b-d90cdb596a0d', 'keypoint': {'x': 334.4151, 'y': 548.7028}, 'name': 'yellow'}, {'id': 'dc28f152-a36a-458c-9f8a-21f4093b66bd', 'keypoint': {'x': 418.4717, 'y': 547.9245}, 'name': 'yellow'}, {'id': '34029874-d3f4-4998-a6fa-85ee314b35b5', 'keypoint': {'x': 479.1792, 'y': 484.8821}, 'name': 'yellow'}, {'id': '3d4114df-2e6e-4d36-8e12-47bbffbd6150', 'keypoint': {'x': 314.1792, 'y': 452.1934}, 'name': 'blue'}, {'id': '021cc2a3-2e83-4079-9777-64d3bc57e829', 'keypoint': {'x': 293.1651, 'y': 557.2642}, 'name': 'blue'}, {'id': '49911b5f-df0a-4694-9ff5-d8165f0f587a', 'keypoint': {'x': 390.4528, 'y': 513.6792}, 'name': 'blue'}, {'id': '95b20075-c6a2-46e0-b203-f8004a3ee4a3', 'keypoint': {'x': 440.2642, 'y': 484.8821}, 'name': 'blue'}, {'id': '031e7d06-9b93-4602-a1ef-c48bce3518c4', 'keypoint': {'x': 462.0566, 'y': 469.316}, 'name': 'blue'}, {'id': '3f2fdd03-f2f9-4892-999b-713f9e988be6', 'keypoint': {'x': 493.1887, 'y': 465.4245}, 'name': 'blue'}, {'id': '7ed0f47f-3bdf-429a-ab98-1d1a9d3058fa', 'keypoint': {'x': 482.2925, 'y': 507.4528}, 'name': 'green'}, {'id': '1fb5251e-aa36-4074-87f5-edef1e5a25c7', 'keypoint': {'x': 491.6321, 'y': 542.4764}, 'name': 'green'}, {'id': 'e7cf6134-82a1-4efe-99a5-a098dcdb3a3e', 'keypoint': {'x': 334.4151, 'y': 516.0142}, 'name': 'green'}, {'id': '0aeb291a-183c-47ce-9552-8b4d4860c05f', 'keypoint': {'x': 376.4434, 'y': 44.3632}, 'name': 'you'}]}
[/INST]
{'x': 462.0566, 'y': 469.316}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:In front of your left side are two blue blocks beside an orange one. Pick up the the blue block in between the orange one and the blue one.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'a7c6f9ab-55b7-422b-8e7f-70902f2e94d6', 'keypoint': {'x': 320.4057, 'y': 488.7736}, 'name': 'orange'}, {'id': 'acc187bd-2783-4707-be6a-314a58518228', 'keypoint': {'x': 423.9198, 'y': 451.4151}, 'name': 'orange'}, {'id': '7b9652c2-a499-48cc-ab07-68e686e885ac', 'keypoint': {'x': 358.5425, 'y': 462.3113}, 'name': 'yellow'}, {'id': '6e4e6633-28b0-4e37-a08b-d90cdb596a0d', 'keypoint': {'x': 334.4151, 'y': 548.7028}, 'name': 'yellow'}, {'id': 'dc28f152-a36a-458c-9f8a-21f4093b66bd', 'keypoint': {'x': 418.4717, 'y': 547.9245}, 'name': 'yellow'}, {'id': '34029874-d3f4-4998-a6fa-85ee314b35b5', 'keypoint': {'x': 479.1792, 'y': 484.8821}, 'name': 'yellow'}, {'id': '3d4114df-2e6e-4d36-8e12-47bbffbd6150', 'keypoint': {'x': 314.1792, 'y': 452.1934}, 'name': 'blue'}, {'id': '021cc2a3-2e83-4079-9777-64d3bc57e829', 'keypoint': {'x': 293.1651, 'y': 557.2642}, 'name': 'blue'}, {'id': '49911b5f-df0a-4694-9ff5-d8165f0f587a', 'keypoint': {'x': 390.4528, 'y': 513.6792}, 'name': 'blue'}, {'id': '95b20075-c6a2-46e0-b203-f8004a3ee4a3', 'keypoint': {'x': 440.2642, 'y': 484.8821}, 'name': 'blue'}, {'id': '031e7d06-9b93-4602-a1ef-c48bce3518c4', 'keypoint': {'x': 462.0566, 'y': 469.316}, 'name': 'blue'}, {'id': '3f2fdd03-f2f9-4892-999b-713f9e988be6', 'keypoint': {'x': 493.1887, 'y': 465.4245}, 'name': 'blue'}, {'id': '7ed0f47f-3bdf-429a-ab98-1d1a9d3058fa', 'keypoint': {'x': 482.2925, 'y': 507.4528}, 'name': 'green'}, {'id': '1fb5251e-aa36-4074-87f5-edef1e5a25c7', 'keypoint': {'x': 491.6321, 'y': 542.4764}, 'name': 'green'}, {'id': 'e7cf6134-82a1-4efe-99a5-a098dcdb3a3e', 'keypoint': {'x': 334.4151, 'y': 516.0142}, 'name': 'green'}, {'id': '0aeb291a-183c-47ce-9552-8b4d4860c05f', 'keypoint': {'x': 376.4434, 'y': 44.3632}, 'name': 'you'}]}
[/INST]
{'x': 462.0566, 'y': 469.316}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the orange block that is furthest away from me.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'b1c926ae-92c1-43eb-96c5-76c41d0fab06', 'keypoint': {'x': 342.1981, 'y': 437.4057}, 'name': 'orange'}, {'id': '76267614-95a1-4ded-b4e9-354b821f62bb', 'keypoint': {'x': 314.1792, 'y': 499.6698}, 'name': 'orange'}, {'id': '3124e476-176a-4feb-8844-b3973d48973a', 'keypoint': {'x': 453.4953, 'y': 463.8679}, 'name': 'orange'}, {'id': '728bd5ee-ece2-45c9-b0d7-b887692bb453', 'keypoint': {'x': 494.7453, 'y': 534.6934}, 'name': 'orange'}, {'id': 'da0db0c2-fdee-4550-8e2a-11f1aace764f', 'keypoint': {'x': 376.4434, 'y': 419.5047}, 'name': 'yellow'}, {'id': 'bd27285d-35e5-47b5-9b1e-78bd5fb9e595', 'keypoint': {'x': 360.0991, 'y': 516.7925}, 'name': 'yellow'}, {'id': '5a97f3d8-463a-4c65-9906-b8cf5f772e2b', 'keypoint': {'x': 408.3538, 'y': 526.9104}, 'name': 'yellow'}, {'id': 'f2444027-7c76-40d2-9424-36c46342a49f', 'keypoint': {'x': 318.8491, 'y': 449.0802}, 'name': 'blue'}, {'id': '0b2e3fca-8a71-43d2-a83e-f02af09891b9', 'keypoint': {'x': 315.7358, 'y': 480.2123}, 'name': 'blue'}, {'id': '897e21a9-0e62-42b6-8c01-826ad479026c', 'keypoint': {'x': 378.0, 'y': 490.3302}, 'name': 'blue'}, {'id': 'ca6c0f72-0600-47ba-b4b1-05dc3566a83e', 'keypoint': {'x': 402.1274, 'y': 463.0896}, 'name': 'blue'}, {'id': '41397d83-cab1-4c4d-ba20-b13a1c93f043', 'keypoint': {'x': 419.25, 'y': 432.7358}, 'name': 'blue'}, {'id': '051bcea0-383b-4791-9652-d84f6ef3fe4c', 'keypoint': {'x': 348.4245, 'y': 488.7736}, 'name': 'green'}, {'id': '58f619eb-2384-4a80-97f9-10520e5226a9', 'keypoint': {'x': 311.8443, 'y': 539.3632}, 'name': 'green'}, {'id': 'b9c15fa4-b07d-4f16-9f7d-f007698c83c2', 'keypoint': {'x': 417.6934, 'y': 502.0047}, 'name': 'green'}, {'id': '5c0026dd-e833-4ad7-8339-361bad3686e0', 'keypoint': {'x': 378.0, 'y': 59.1509}, 'name': 'you'}]}
[/INST]
{'x': 342.1981, 'y': 437.4057}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:pick the second yellow block, from the left
 {'width': 756, 'height': 660, 'annotations': [{'id': 'fe98e63d-c226-4a25-9e75-229c9a2d828a', 'keypoint': {'x': 377.2217, 'y': 28.7972}, 'name': 'you'}, {'id': 'c1bfc6fe-9e0d-4152-aab0-9efe03f99ee1', 'keypoint': {'x': 291.6085, 'y': 487.217}, 'name': 'yellow'}, {'id': 'eefd5787-08ef-4674-ae9e-c41f4a2ed951', 'keypoint': {'x': 314.1792, 'y': 539.3632}, 'name': 'yellow'}, {'id': '8987e25d-8eb0-42ab-aa8e-390a7b046e44', 'keypoint': {'x': 363.2123, 'y': 544.8113}, 'name': 'yellow'}, {'id': 'af54d764-7469-4b39-95f2-d25ac6241715', 'keypoint': {'x': 434.0377, 'y': 542.4764}, 'name': 'yellow'}, {'id': 'c426887c-43fa-4c7c-9ee6-447f48c5b353', 'keypoint': {'x': 463.6132, 'y': 487.9953}, 'name': 'yellow'}, {'id': 'f1402f90-fe95-42c0-93e5-75019494beba', 'keypoint': {'x': 497.0802, 'y': 442.0755}, 'name': 'yellow'}, {'id': 'ad7b2c3b-efac-4705-8dc4-076514da9f0d', 'keypoint': {'x': 319.6274, 'y': 463.8679}, 'name': 'blue'}, {'id': '57fd11a6-5352-4e90-8c5e-78e080a9f63b', 'keypoint': {'x': 458.1651, 'y': 516.0142}, 'name': 'blue'}, {'id': 'bffb87fb-647e-44e1-b4c0-9bbf464cc4da', 'keypoint': {'x': 353.8726, 'y': 445.1887}, 'name': 'green'}, {'id': '2be751d9-34f0-411b-89e7-1b20b054e1ee', 'keypoint': {'x': 388.8962, 'y': 498.1132}, 'name': 'green'}, {'id': '98a41430-e570-492e-969a-8ea45857d765', 'keypoint': {'x': 395.9009, 'y': 533.1368}, 'name': 'green'}, {'id': '0327cb90-c17c-4528-8964-e978fbff5999', 'keypoint': {'x': 427.8113, 'y': 484.8821}, 'name': 'green'}, {'id': '97f91dd8-6cd8-4194-8783-e9e3b5786e68', 'keypoint': {'x': 456.6085, 'y': 452.9717}, 'name': 'green'}, {'id': '4ebadfd4-3497-4f80-a4e9-a870c25786bd', 'keypoint': {'x': 504.8632, 'y': 502.0047}, 'name': 'green'}, {'id': 'd2ff782c-210c-4090-91a5-de43a78f1232', 'keypoint': {'x': 331.3019, 'y': 512.9009}, 'name': 'orange'}]}
[/INST]
{'x': 463.6132, 'y': 487.9953}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Find the only two green blocks. Pick up the yellow block in front of and just to the right of those green blocks.
 {'width': 756, 'height': 660, 'annotations': [{'id': '53de559f-b2c6-46fa-821d-b5a4e98c67c3', 'keypoint': {'x': 332.0802, 'y': 517.5708}, 'name': 'orange'}, {'id': 'fc5cb394-ebdb-4064-bb12-1d3da5b0cb6f', 'keypoint': {'x': 316.5142, 'y': 540.1415}, 'name': 'orange'}, {'id': 'b8f5d815-90be-4cd5-951f-44207215b594', 'keypoint': {'x': 373.3302, 'y': 537.8066}, 'name': 'orange'}, {'id': '78d88ded-7431-4260-a864-725e2dfef6ab', 'keypoint': {'x': 309.5094, 'y': 498.1132}, 'name': 'yellow'}, {'id': 'e5215008-d111-4bda-9009-f2c0795ffb0d', 'keypoint': {'x': 388.1179, 'y': 471.6509}, 'name': 'yellow'}, {'id': 'b722e1f1-0a3c-47ef-b35f-e9cd85898169', 'keypoint': {'x': 388.1179, 'y': 501.2264}, 'name': 'yellow'}, {'id': '27ad7266-481b-40f9-aaa0-e71dabd71d5e', 'keypoint': {'x': 413.8019, 'y': 491.1085}, 'name': 'yellow'}, {'id': '71348b8e-08dc-4a76-be0e-68cbee1a56a4', 'keypoint': {'x': 416.1368, 'y': 449.8585}, 'name': 'yellow'}, {'id': 'e46d6178-4284-433d-a81f-bc6cbbf7cdbb', 'keypoint': {'x': 490.8538, 'y': 505.1179}, 'name': 'yellow'}, {'id': '272a554a-aad9-4bba-9062-941465beeaa5', 'keypoint': {'x': 465.1698, 'y': 554.1509}, 'name': 'yellow'}, {'id': 'c6ba259b-1c8e-43d4-8bdb-404594f1c175', 'keypoint': {'x': 310.2877, 'y': 438.9623}, 'name': 'blue'}, {'id': '693a6454-810f-447d-8417-a2c57867901d', 'keypoint': {'x': 354.6509, 'y': 477.8774}, 'name': 'blue'}, {'id': '3ec7fa7f-c792-4c99-a423-b40bbb665d89', 'keypoint': {'x': 427.8113, 'y': 428.8443}, 'name': 'blue'}, {'id': '3e041f2e-9e78-4001-9fa0-3bebee80ac07', 'keypoint': {'x': 440.2642, 'y': 498.8915}, 'name': 'green'}, {'id': 'e2e59b33-f4e7-47cd-b137-77a874c834c7', 'keypoint': {'x': 450.3821, 'y': 519.1274}, 'name': 'green'}, {'id': '707959b0-76f6-463a-bc2f-62cd702815f4', 'keypoint': {'x': 377.2217, 'y': 55.2594}, 'name': 'you'}]}
[/INST]
{'x': 413.8019, 'y': 491.1085}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the green block to the right or the blue block in the middle.
 {'width': 756, 'height': 660, 'annotations': [{'id': '8beb9a9e-a451-450a-b594-7a9ae18e74f9', 'keypoint': {'x': 458.9434, 'y': 531.5802}, 'name': 'orange'}, {'id': 'ee956837-2183-4b52-8aa2-dfa990f1d376', 'keypoint': {'x': 386.5613, 'y': 529.2453}, 'name': 'yellow'}, {'id': '427a5df9-dc11-494e-8632-81ffcd5e66c8', 'keypoint': {'x': 471.3962, 'y': 453.75}, 'name': 'yellow'}, {'id': 'bcc302a4-c191-4bc2-82e0-1ceedd4a2928', 'keypoint': {'x': 329.7453, 'y': 453.75}, 'name': 'blue'}, {'id': '8584b004-32b4-47bd-9100-a2eab16c5301', 'keypoint': {'x': 294.7217, 'y': 501.2264}, 'name': 'blue'}, {'id': 'ba4430e4-6293-47e3-b512-0e78897c96fb', 'keypoint': {'x': 317.2925, 'y': 520.684}, 'name': 'blue'}, {'id': '9c57d65f-59e8-4d45-a882-721a56841969', 'keypoint': {'x': 433.2594, 'y': 498.1132}, 'name': 'blue'}, {'id': '0e241c7c-e225-4266-9044-a65451aceb68', 'keypoint': {'x': 361.6557, 'y': 428.8443}, 'name': 'green'}, {'id': '050252d9-484a-4a87-a46c-1f69f5d7ea14', 'keypoint': {'x': 416.9151, 'y': 435.8491}, 'name': 'green'}, {'id': '5b9ab273-0e91-4589-9a86-d2969adc4e30', 'keypoint': {'x': 435.5943, 'y': 445.967}, 'name': 'green'}, {'id': '4b452968-0027-4b94-92e6-e8dcd32d1e25', 'keypoint': {'x': 468.283, 'y': 489.5519}, 'name': 'green'}, {'id': '3ab98d12-5e5c-4bdf-931d-55f2ee2d23b2', 'keypoint': {'x': 386.5613, 'y': 498.8915}, 'name': 'green'}, {'id': '416f60f7-59aa-46b3-858c-7dfde5e23875', 'keypoint': {'x': 353.0943, 'y': 497.3349}, 'name': 'green'}, {'id': 'ce8faeb7-52eb-4dc8-8db9-4295f1b077e3', 'keypoint': {'x': 329.7453, 'y': 491.8868}, 'name': 'green'}, {'id': '6ea4f70c-aa80-4f14-a1ac-d26081a22a0d', 'keypoint': {'x': 379.5566, 'y': 52.1462}, 'name': 'you'}]}
[/INST]
{'x': 386.5613, 'y': 498.8915}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the green block that is directly in the middle of the table, right behind the two orange blocks that are furthest away from you.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'd5bcc2b7-ff91-4686-8f50-3a73ffd3beeb', 'keypoint': {'x': 315.7358, 'y': 436.6274}, 'name': 'orange'}, {'id': '70515b86-98d5-4fa6-9118-a0bc484f6de1', 'keypoint': {'x': 313.4009, 'y': 525.3538}, 'name': 'orange'}, {'id': 'b40b0958-f706-45d2-a042-0440eb67121b', 'keypoint': {'x': 380.3349, 'y': 515.2358}, 'name': 'orange'}, {'id': '9dcd051f-bad9-4f84-b653-9a30a72cb545', 'keypoint': {'x': 389.6745, 'y': 552.5943}, 'name': 'orange'}, {'id': 'b0de6329-f4d9-486c-91ed-478f28262cff', 'keypoint': {'x': 363.9906, 'y': 439.7406}, 'name': 'yellow'}, {'id': '10eeedc5-3253-42e7-b401-33c4616e4623', 'keypoint': {'x': 396.6792, 'y': 457.6415}, 'name': 'yellow'}, {'id': 'f7642eee-44ca-4f1d-abf7-a79d43dfbd0d', 'keypoint': {'x': 279.934, 'y': 535.4717}, 'name': 'yellow'}, {'id': 'd87c603d-89da-4c0b-8f61-a496275c1297', 'keypoint': {'x': 311.066, 'y': 487.217}, 'name': 'blue'}, {'id': '2065d0e4-12eb-4ab3-bde4-75688bb3c142', 'keypoint': {'x': 457.3868, 'y': 439.7406}, 'name': 'green'}, {'id': '1550df58-31c4-49f4-8e6d-7db4a57bc294', 'keypoint': {'x': 435.5943, 'y': 445.967}, 'name': 'green'}, {'id': '1990bf50-38ac-43ab-8581-26778cea80ed', 'keypoint': {'x': 458.9434, 'y': 484.8821}, 'name': 'green'}, {'id': '7895d85c-2e57-4008-b9d7-74625608a42c', 'keypoint': {'x': 485.4057, 'y': 515.2358}, 'name': 'green'}, {'id': '991a571a-03aa-4a00-b80d-3d6a0845625b', 'keypoint': {'x': 440.2642, 'y': 514.4575}, 'name': 'green'}, {'id': 'e0e1086c-3a47-42f6-98fd-868d0756cd4d', 'keypoint': {'x': 378.7783, 'y': 495.7783}, 'name': 'green'}, {'id': '3d132917-dd96-4ae9-9302-da261696357f', 'keypoint': {'x': 333.6368, 'y': 505.1179}, 'name': 'green'}, {'id': 'bc7494f1-b049-49c6-bf24-85e942969065', 'keypoint': {'x': 383.4481, 'y': 56.0377}, 'name': 'you'}]}
[/INST]
{'x': 378.7783, 'y': 495.7783}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the blue block in the exact center of the table closest to you.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'bb2508d6-bab0-4991-a4b4-1d60addd89be', 'keypoint': {'x': 374.8868, 'y': 32.6887}, 'name': 'you'}, {'id': '80bad823-3c11-42a6-9b7a-8f92fd053db8', 'keypoint': {'x': 286.1604, 'y': 504.3396}, 'name': 'yellow'}, {'id': '74407b61-1d02-4456-a3be-f44e6d0621e4', 'keypoint': {'x': 427.8113, 'y': 511.3443}, 'name': 'yellow'}, {'id': '0c043462-b620-40d3-b17e-3f17c7108248', 'keypoint': {'x': 391.2311, 'y': 435.8491}, 'name': 'green'}, {'id': 'e97e7600-f671-4774-970c-bcd8e011471e', 'keypoint': {'x': 311.066, 'y': 530.8019}, 'name': 'green'}, {'id': '3c45bdee-a533-4cab-8e50-cd3e05ea379a', 'keypoint': {'x': 375.6651, 'y': 518.3491}, 'name': 'green'}, {'id': 'fbe906a6-d83a-45bc-bd1e-7c81ccde8b9b', 'keypoint': {'x': 487.7406, 'y': 473.2075}, 'name': 'green'}, {'id': '662455ed-994d-45d8-b864-0f07ccfe9a1e', 'keypoint': {'x': 309.5094, 'y': 477.8774}, 'name': 'blue'}, {'id': '92e619ee-cc43-4ba6-80a0-fd23cabc2b68', 'keypoint': {'x': 378.7783, 'y': 457.6415}, 'name': 'blue'}, {'id': '4d3a4397-37ee-4ea1-bff7-ea4e40e4ba91', 'keypoint': {'x': 395.1226, 'y': 483.3255}, 'name': 'blue'}, {'id': '83c6c716-2497-4e0e-b233-3cda2dca8049', 'keypoint': {'x': 438.7075, 'y': 448.3019}, 'name': 'blue'}, {'id': '24b23d82-8281-40fd-9942-dc852b76fea4', 'keypoint': {'x': 444.1557, 'y': 484.1038}, 'name': 'blue'}, {'id': '541179c1-e37f-4069-99a8-fe58be3053c6', 'keypoint': {'x': 466.7264, 'y': 502.783}, 'name': 'blue'}, {'id': 'd950b6f4-0fff-425b-bf69-419e3a15863f', 'keypoint': {'x': 343.7547, 'y': 543.2547}, 'name': 'orange'}, {'id': 'a5aa3394-3b02-4613-8a7d-f14c9cd0fa58', 'keypoint': {'x': 431.7028, 'y': 539.3632}, 'name': 'orange'}, {'id': 'd7eb7db6-f69d-4df7-861e-0cc9dbebac2a', 'keypoint': {'x': 484.6274, 'y': 532.3585}, 'name': 'orange'}]}
[/INST]
{'x': 378.7783, 'y': 457.6415}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the orange block that is the furthest away, and which is spaced away from the rest of the blocks.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'b1c926ae-92c1-43eb-96c5-76c41d0fab06', 'keypoint': {'x': 342.1981, 'y': 437.4057}, 'name': 'orange'}, {'id': '76267614-95a1-4ded-b4e9-354b821f62bb', 'keypoint': {'x': 314.1792, 'y': 499.6698}, 'name': 'orange'}, {'id': '3124e476-176a-4feb-8844-b3973d48973a', 'keypoint': {'x': 453.4953, 'y': 463.8679}, 'name': 'orange'}, {'id': '728bd5ee-ece2-45c9-b0d7-b887692bb453', 'keypoint': {'x': 494.7453, 'y': 534.6934}, 'name': 'orange'}, {'id': 'da0db0c2-fdee-4550-8e2a-11f1aace764f', 'keypoint': {'x': 376.4434, 'y': 419.5047}, 'name': 'yellow'}, {'id': 'bd27285d-35e5-47b5-9b1e-78bd5fb9e595', 'keypoint': {'x': 360.0991, 'y': 516.7925}, 'name': 'yellow'}, {'id': '5a97f3d8-463a-4c65-9906-b8cf5f772e2b', 'keypoint': {'x': 408.3538, 'y': 526.9104}, 'name': 'yellow'}, {'id': 'f2444027-7c76-40d2-9424-36c46342a49f', 'keypoint': {'x': 318.8491, 'y': 449.0802}, 'name': 'blue'}, {'id': '0b2e3fca-8a71-43d2-a83e-f02af09891b9', 'keypoint': {'x': 315.7358, 'y': 480.2123}, 'name': 'blue'}, {'id': '897e21a9-0e62-42b6-8c01-826ad479026c', 'keypoint': {'x': 378.0, 'y': 490.3302}, 'name': 'blue'}, {'id': 'ca6c0f72-0600-47ba-b4b1-05dc3566a83e', 'keypoint': {'x': 402.1274, 'y': 463.0896}, 'name': 'blue'}, {'id': '41397d83-cab1-4c4d-ba20-b13a1c93f043', 'keypoint': {'x': 419.25, 'y': 432.7358}, 'name': 'blue'}, {'id': '051bcea0-383b-4791-9652-d84f6ef3fe4c', 'keypoint': {'x': 348.4245, 'y': 488.7736}, 'name': 'green'}, {'id': '58f619eb-2384-4a80-97f9-10520e5226a9', 'keypoint': {'x': 311.8443, 'y': 539.3632}, 'name': 'green'}, {'id': 'b9c15fa4-b07d-4f16-9f7d-f007698c83c2', 'keypoint': {'x': 417.6934, 'y': 502.0047}, 'name': 'green'}, {'id': '5c0026dd-e833-4ad7-8339-361bad3686e0', 'keypoint': {'x': 378.0, 'y': 59.1509}, 'name': 'you'}]}
[/INST]
{'x': 494.7453, 'y': 534.6934}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:yellow block surrounded by more yellow blocks
 {'width': 756, 'height': 660, 'annotations': [{'id': 'b328ac50-08b5-486e-a937-2cddbc04245c', 'keypoint': {'x': 304.8396, 'y': 434.2925}, 'name': 'orange'}, {'id': '7344b92c-5683-4670-9cef-33f4cf8e73fb', 'keypoint': {'x': 472.9528, 'y': 456.0849}, 'name': 'orange'}, {'id': '6958bbb7-3dce-4cfe-9049-02ebc5a32e43', 'keypoint': {'x': 306.3962, 'y': 460.7547}, 'name': 'yellow'}, {'id': '62285f29-522c-498a-9fdb-53e2e8421ea5', 'keypoint': {'x': 330.5236, 'y': 470.8726}, 'name': 'yellow'}, {'id': 'e4402e53-f01e-4254-a874-f6ffd3892ac4', 'keypoint': {'x': 381.8915, 'y': 475.5425}, 'name': 'yellow'}, {'id': '3f3ed2fa-5312-418f-9f29-f4750a72aa9e', 'keypoint': {'x': 402.9057, 'y': 435.0708}, 'name': 'yellow'}, {'id': 'fe5ba8f4-1f70-4712-ae25-cd20ac9670d6', 'keypoint': {'x': 455.8302, 'y': 481.7689}, 'name': 'yellow'}, {'id': 'bd1c4ddb-8345-4f24-a982-f6cd6a64fb0e', 'keypoint': {'x': 501.75, 'y': 505.8962}, 'name': 'yellow'}, {'id': '8f332be0-ad10-4130-9840-f3a1dc330890', 'keypoint': {'x': 352.316, 'y': 513.6792}, 'name': 'yellow'}, {'id': '5b19adbf-7efb-4a90-9f40-4c0bf2f3b9a8', 'keypoint': {'x': 324.2972, 'y': 499.6698}, 'name': 'yellow'}, {'id': '7f124547-f8b7-4280-b7e1-401876ce2bdb', 'keypoint': {'x': 308.7311, 'y': 552.5943}, 'name': 'yellow'}, {'id': '8a1fd773-bc7c-4be5-a292-c1c53d5c77df', 'keypoint': {'x': 298.6132, 'y': 491.8868}, 'name': 'blue'}, {'id': '7be8714d-676d-466e-a70a-e9d078b757f4', 'keypoint': {'x': 381.1132, 'y': 497.3349}, 'name': 'blue'}, {'id': 'b1ee1721-f1d2-4cc8-828b-2e6c5072d54f', 'keypoint': {'x': 451.9387, 'y': 432.7358}, 'name': 'blue'}, {'id': '0416eebd-320e-40d8-bbc7-a94a6333aa95', 'keypoint': {'x': 497.0802, 'y': 485.6604}, 'name': 'green'}, {'id': '2240b1ab-d515-4872-a587-09ca836dc397', 'keypoint': {'x': 378.7783, 'y': 28.7972}, 'name': 'you'}]}
[/INST]
{'x': 324.2972, 'y': 499.6698}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:There is one orange block separated from the other ones. Please pick this one.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'b1c926ae-92c1-43eb-96c5-76c41d0fab06', 'keypoint': {'x': 342.1981, 'y': 437.4057}, 'name': 'orange'}, {'id': '76267614-95a1-4ded-b4e9-354b821f62bb', 'keypoint': {'x': 314.1792, 'y': 499.6698}, 'name': 'orange'}, {'id': '3124e476-176a-4feb-8844-b3973d48973a', 'keypoint': {'x': 453.4953, 'y': 463.8679}, 'name': 'orange'}, {'id': '728bd5ee-ece2-45c9-b0d7-b887692bb453', 'keypoint': {'x': 494.7453, 'y': 534.6934}, 'name': 'orange'}, {'id': 'da0db0c2-fdee-4550-8e2a-11f1aace764f', 'keypoint': {'x': 376.4434, 'y': 419.5047}, 'name': 'yellow'}, {'id': 'bd27285d-35e5-47b5-9b1e-78bd5fb9e595', 'keypoint': {'x': 360.0991, 'y': 516.7925}, 'name': 'yellow'}, {'id': '5a97f3d8-463a-4c65-9906-b8cf5f772e2b', 'keypoint': {'x': 408.3538, 'y': 526.9104}, 'name': 'yellow'}, {'id': 'f2444027-7c76-40d2-9424-36c46342a49f', 'keypoint': {'x': 318.8491, 'y': 449.0802}, 'name': 'blue'}, {'id': '0b2e3fca-8a71-43d2-a83e-f02af09891b9', 'keypoint': {'x': 315.7358, 'y': 480.2123}, 'name': 'blue'}, {'id': '897e21a9-0e62-42b6-8c01-826ad479026c', 'keypoint': {'x': 378.0, 'y': 490.3302}, 'name': 'blue'}, {'id': 'ca6c0f72-0600-47ba-b4b1-05dc3566a83e', 'keypoint': {'x': 402.1274, 'y': 463.0896}, 'name': 'blue'}, {'id': '41397d83-cab1-4c4d-ba20-b13a1c93f043', 'keypoint': {'x': 419.25, 'y': 432.7358}, 'name': 'blue'}, {'id': '051bcea0-383b-4791-9652-d84f6ef3fe4c', 'keypoint': {'x': 348.4245, 'y': 488.7736}, 'name': 'green'}, {'id': '58f619eb-2384-4a80-97f9-10520e5226a9', 'keypoint': {'x': 311.8443, 'y': 539.3632}, 'name': 'green'}, {'id': 'b9c15fa4-b07d-4f16-9f7d-f007698c83c2', 'keypoint': {'x': 417.6934, 'y': 502.0047}, 'name': 'green'}, {'id': '5c0026dd-e833-4ad7-8339-361bad3686e0', 'keypoint': {'x': 378.0, 'y': 59.1509}, 'name': 'you'}]}
[/INST]
{'x': 494.7453, 'y': 534.6934}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the middle of the three green blocks that are located on the middle of the table.
 {'width': 756, 'height': 660, 'annotations': [{'id': '8beb9a9e-a451-450a-b594-7a9ae18e74f9', 'keypoint': {'x': 458.9434, 'y': 531.5802}, 'name': 'orange'}, {'id': 'ee956837-2183-4b52-8aa2-dfa990f1d376', 'keypoint': {'x': 386.5613, 'y': 529.2453}, 'name': 'yellow'}, {'id': '427a5df9-dc11-494e-8632-81ffcd5e66c8', 'keypoint': {'x': 471.3962, 'y': 453.75}, 'name': 'yellow'}, {'id': 'bcc302a4-c191-4bc2-82e0-1ceedd4a2928', 'keypoint': {'x': 329.7453, 'y': 453.75}, 'name': 'blue'}, {'id': '8584b004-32b4-47bd-9100-a2eab16c5301', 'keypoint': {'x': 294.7217, 'y': 501.2264}, 'name': 'blue'}, {'id': 'ba4430e4-6293-47e3-b512-0e78897c96fb', 'keypoint': {'x': 317.2925, 'y': 520.684}, 'name': 'blue'}, {'id': '9c57d65f-59e8-4d45-a882-721a56841969', 'keypoint': {'x': 433.2594, 'y': 498.1132}, 'name': 'blue'}, {'id': '0e241c7c-e225-4266-9044-a65451aceb68', 'keypoint': {'x': 361.6557, 'y': 428.8443}, 'name': 'green'}, {'id': '050252d9-484a-4a87-a46c-1f69f5d7ea14', 'keypoint': {'x': 416.9151, 'y': 435.8491}, 'name': 'green'}, {'id': '5b9ab273-0e91-4589-9a86-d2969adc4e30', 'keypoint': {'x': 435.5943, 'y': 445.967}, 'name': 'green'}, {'id': '4b452968-0027-4b94-92e6-e8dcd32d1e25', 'keypoint': {'x': 468.283, 'y': 489.5519}, 'name': 'green'}, {'id': '3ab98d12-5e5c-4bdf-931d-55f2ee2d23b2', 'keypoint': {'x': 386.5613, 'y': 498.8915}, 'name': 'green'}, {'id': '416f60f7-59aa-46b3-858c-7dfde5e23875', 'keypoint': {'x': 353.0943, 'y': 497.3349}, 'name': 'green'}, {'id': 'ce8faeb7-52eb-4dc8-8db9-4295f1b077e3', 'keypoint': {'x': 329.7453, 'y': 491.8868}, 'name': 'green'}, {'id': '6ea4f70c-aa80-4f14-a1ac-d26081a22a0d', 'keypoint': {'x': 379.5566, 'y': 52.1462}, 'name': 'you'}]}
[/INST]
{'x': 353.0943, 'y': 497.3349}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the middle blue block closest to you.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'bb2508d6-bab0-4991-a4b4-1d60addd89be', 'keypoint': {'x': 374.8868, 'y': 32.6887}, 'name': 'you'}, {'id': '80bad823-3c11-42a6-9b7a-8f92fd053db8', 'keypoint': {'x': 286.1604, 'y': 504.3396}, 'name': 'yellow'}, {'id': '74407b61-1d02-4456-a3be-f44e6d0621e4', 'keypoint': {'x': 427.8113, 'y': 511.3443}, 'name': 'yellow'}, {'id': '0c043462-b620-40d3-b17e-3f17c7108248', 'keypoint': {'x': 391.2311, 'y': 435.8491}, 'name': 'green'}, {'id': 'e97e7600-f671-4774-970c-bcd8e011471e', 'keypoint': {'x': 311.066, 'y': 530.8019}, 'name': 'green'}, {'id': '3c45bdee-a533-4cab-8e50-cd3e05ea379a', 'keypoint': {'x': 375.6651, 'y': 518.3491}, 'name': 'green'}, {'id': 'fbe906a6-d83a-45bc-bd1e-7c81ccde8b9b', 'keypoint': {'x': 487.7406, 'y': 473.2075}, 'name': 'green'}, {'id': '662455ed-994d-45d8-b864-0f07ccfe9a1e', 'keypoint': {'x': 309.5094, 'y': 477.8774}, 'name': 'blue'}, {'id': '92e619ee-cc43-4ba6-80a0-fd23cabc2b68', 'keypoint': {'x': 378.7783, 'y': 457.6415}, 'name': 'blue'}, {'id': '4d3a4397-37ee-4ea1-bff7-ea4e40e4ba91', 'keypoint': {'x': 395.1226, 'y': 483.3255}, 'name': 'blue'}, {'id': '83c6c716-2497-4e0e-b233-3cda2dca8049', 'keypoint': {'x': 438.7075, 'y': 448.3019}, 'name': 'blue'}, {'id': '24b23d82-8281-40fd-9942-dc852b76fea4', 'keypoint': {'x': 444.1557, 'y': 484.1038}, 'name': 'blue'}, {'id': '541179c1-e37f-4069-99a8-fe58be3053c6', 'keypoint': {'x': 466.7264, 'y': 502.783}, 'name': 'blue'}, {'id': 'd950b6f4-0fff-425b-bf69-419e3a15863f', 'keypoint': {'x': 343.7547, 'y': 543.2547}, 'name': 'orange'}, {'id': 'a5aa3394-3b02-4613-8a7d-f14c9cd0fa58', 'keypoint': {'x': 431.7028, 'y': 539.3632}, 'name': 'orange'}, {'id': 'd7eb7db6-f69d-4df7-861e-0cc9dbebac2a', 'keypoint': {'x': 484.6274, 'y': 532.3585}, 'name': 'orange'}]}
[/INST]
{'x': 378.7783, 'y': 457.6415}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the brown block that is set apart from all the other blocks on the table.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'b1c926ae-92c1-43eb-96c5-76c41d0fab06', 'keypoint': {'x': 342.1981, 'y': 437.4057}, 'name': 'orange'}, {'id': '76267614-95a1-4ded-b4e9-354b821f62bb', 'keypoint': {'x': 314.1792, 'y': 499.6698}, 'name': 'orange'}, {'id': '3124e476-176a-4feb-8844-b3973d48973a', 'keypoint': {'x': 453.4953, 'y': 463.8679}, 'name': 'orange'}, {'id': '728bd5ee-ece2-45c9-b0d7-b887692bb453', 'keypoint': {'x': 494.7453, 'y': 534.6934}, 'name': 'orange'}, {'id': 'da0db0c2-fdee-4550-8e2a-11f1aace764f', 'keypoint': {'x': 376.4434, 'y': 419.5047}, 'name': 'yellow'}, {'id': 'bd27285d-35e5-47b5-9b1e-78bd5fb9e595', 'keypoint': {'x': 360.0991, 'y': 516.7925}, 'name': 'yellow'}, {'id': '5a97f3d8-463a-4c65-9906-b8cf5f772e2b', 'keypoint': {'x': 408.3538, 'y': 526.9104}, 'name': 'yellow'}, {'id': 'f2444027-7c76-40d2-9424-36c46342a49f', 'keypoint': {'x': 318.8491, 'y': 449.0802}, 'name': 'blue'}, {'id': '0b2e3fca-8a71-43d2-a83e-f02af09891b9', 'keypoint': {'x': 315.7358, 'y': 480.2123}, 'name': 'blue'}, {'id': '897e21a9-0e62-42b6-8c01-826ad479026c', 'keypoint': {'x': 378.0, 'y': 490.3302}, 'name': 'blue'}, {'id': 'ca6c0f72-0600-47ba-b4b1-05dc3566a83e', 'keypoint': {'x': 402.1274, 'y': 463.0896}, 'name': 'blue'}, {'id': '41397d83-cab1-4c4d-ba20-b13a1c93f043', 'keypoint': {'x': 419.25, 'y': 432.7358}, 'name': 'blue'}, {'id': '051bcea0-383b-4791-9652-d84f6ef3fe4c', 'keypoint': {'x': 348.4245, 'y': 488.7736}, 'name': 'green'}, {'id': '58f619eb-2384-4a80-97f9-10520e5226a9', 'keypoint': {'x': 311.8443, 'y': 539.3632}, 'name': 'green'}, {'id': 'b9c15fa4-b07d-4f16-9f7d-f007698c83c2', 'keypoint': {'x': 417.6934, 'y': 502.0047}, 'name': 'green'}, {'id': '5c0026dd-e833-4ad7-8339-361bad3686e0', 'keypoint': {'x': 378.0, 'y': 59.1509}, 'name': 'you'}]}
[/INST]
{'x': 494.7453, 'y': 534.6934}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the yellow block that is immediately to  the right of a green block.
 {'width': 756, 'height': 660, 'annotations': [{'id': '53de559f-b2c6-46fa-821d-b5a4e98c67c3', 'keypoint': {'x': 332.0802, 'y': 517.5708}, 'name': 'orange'}, {'id': 'fc5cb394-ebdb-4064-bb12-1d3da5b0cb6f', 'keypoint': {'x': 316.5142, 'y': 540.1415}, 'name': 'orange'}, {'id': 'b8f5d815-90be-4cd5-951f-44207215b594', 'keypoint': {'x': 373.3302, 'y': 537.8066}, 'name': 'orange'}, {'id': '78d88ded-7431-4260-a864-725e2dfef6ab', 'keypoint': {'x': 309.5094, 'y': 498.1132}, 'name': 'yellow'}, {'id': 'e5215008-d111-4bda-9009-f2c0795ffb0d', 'keypoint': {'x': 388.1179, 'y': 471.6509}, 'name': 'yellow'}, {'id': 'b722e1f1-0a3c-47ef-b35f-e9cd85898169', 'keypoint': {'x': 388.1179, 'y': 501.2264}, 'name': 'yellow'}, {'id': '27ad7266-481b-40f9-aaa0-e71dabd71d5e', 'keypoint': {'x': 413.8019, 'y': 491.1085}, 'name': 'yellow'}, {'id': '71348b8e-08dc-4a76-be0e-68cbee1a56a4', 'keypoint': {'x': 416.1368, 'y': 449.8585}, 'name': 'yellow'}, {'id': 'e46d6178-4284-433d-a81f-bc6cbbf7cdbb', 'keypoint': {'x': 490.8538, 'y': 505.1179}, 'name': 'yellow'}, {'id': '272a554a-aad9-4bba-9062-941465beeaa5', 'keypoint': {'x': 465.1698, 'y': 554.1509}, 'name': 'yellow'}, {'id': 'c6ba259b-1c8e-43d4-8bdb-404594f1c175', 'keypoint': {'x': 310.2877, 'y': 438.9623}, 'name': 'blue'}, {'id': '693a6454-810f-447d-8417-a2c57867901d', 'keypoint': {'x': 354.6509, 'y': 477.8774}, 'name': 'blue'}, {'id': '3ec7fa7f-c792-4c99-a423-b40bbb665d89', 'keypoint': {'x': 427.8113, 'y': 428.8443}, 'name': 'blue'}, {'id': '3e041f2e-9e78-4001-9fa0-3bebee80ac07', 'keypoint': {'x': 440.2642, 'y': 498.8915}, 'name': 'green'}, {'id': 'e2e59b33-f4e7-47cd-b137-77a874c834c7', 'keypoint': {'x': 450.3821, 'y': 519.1274}, 'name': 'green'}, {'id': '707959b0-76f6-463a-bc2f-62cd702815f4', 'keypoint': {'x': 377.2217, 'y': 55.2594}, 'name': 'you'}]}
[/INST]
{'x': 413.8019, 'y': 491.1085}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pickup the green block that is second farthest to the right.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'd5bcc2b7-ff91-4686-8f50-3a73ffd3beeb', 'keypoint': {'x': 315.7358, 'y': 436.6274}, 'name': 'orange'}, {'id': '70515b86-98d5-4fa6-9118-a0bc484f6de1', 'keypoint': {'x': 313.4009, 'y': 525.3538}, 'name': 'orange'}, {'id': 'b40b0958-f706-45d2-a042-0440eb67121b', 'keypoint': {'x': 380.3349, 'y': 515.2358}, 'name': 'orange'}, {'id': '9dcd051f-bad9-4f84-b653-9a30a72cb545', 'keypoint': {'x': 389.6745, 'y': 552.5943}, 'name': 'orange'}, {'id': 'b0de6329-f4d9-486c-91ed-478f28262cff', 'keypoint': {'x': 363.9906, 'y': 439.7406}, 'name': 'yellow'}, {'id': '10eeedc5-3253-42e7-b401-33c4616e4623', 'keypoint': {'x': 396.6792, 'y': 457.6415}, 'name': 'yellow'}, {'id': 'f7642eee-44ca-4f1d-abf7-a79d43dfbd0d', 'keypoint': {'x': 279.934, 'y': 535.4717}, 'name': 'yellow'}, {'id': 'd87c603d-89da-4c0b-8f61-a496275c1297', 'keypoint': {'x': 311.066, 'y': 487.217}, 'name': 'blue'}, {'id': '2065d0e4-12eb-4ab3-bde4-75688bb3c142', 'keypoint': {'x': 457.3868, 'y': 439.7406}, 'name': 'green'}, {'id': '1550df58-31c4-49f4-8e6d-7db4a57bc294', 'keypoint': {'x': 435.5943, 'y': 445.967}, 'name': 'green'}, {'id': '1990bf50-38ac-43ab-8581-26778cea80ed', 'keypoint': {'x': 458.9434, 'y': 484.8821}, 'name': 'green'}, {'id': '7895d85c-2e57-4008-b9d7-74625608a42c', 'keypoint': {'x': 485.4057, 'y': 515.2358}, 'name': 'green'}, {'id': '991a571a-03aa-4a00-b80d-3d6a0845625b', 'keypoint': {'x': 440.2642, 'y': 514.4575}, 'name': 'green'}, {'id': 'e0e1086c-3a47-42f6-98fd-868d0756cd4d', 'keypoint': {'x': 378.7783, 'y': 495.7783}, 'name': 'green'}, {'id': '3d132917-dd96-4ae9-9302-da261696357f', 'keypoint': {'x': 333.6368, 'y': 505.1179}, 'name': 'green'}, {'id': 'bc7494f1-b049-49c6-bf24-85e942969065', 'keypoint': {'x': 383.4481, 'y': 56.0377}, 'name': 'you'}]}
[/INST]
{'x': 378.7783, 'y': 495.7783}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the yellow block on the left that is in front of the blue block
 {'width': 756, 'height': 660, 'annotations': [{'id': 'fe98e63d-c226-4a25-9e75-229c9a2d828a', 'keypoint': {'x': 377.2217, 'y': 28.7972}, 'name': 'you'}, {'id': 'c1bfc6fe-9e0d-4152-aab0-9efe03f99ee1', 'keypoint': {'x': 291.6085, 'y': 487.217}, 'name': 'yellow'}, {'id': 'eefd5787-08ef-4674-ae9e-c41f4a2ed951', 'keypoint': {'x': 314.1792, 'y': 539.3632}, 'name': 'yellow'}, {'id': '8987e25d-8eb0-42ab-aa8e-390a7b046e44', 'keypoint': {'x': 363.2123, 'y': 544.8113}, 'name': 'yellow'}, {'id': 'af54d764-7469-4b39-95f2-d25ac6241715', 'keypoint': {'x': 434.0377, 'y': 542.4764}, 'name': 'yellow'}, {'id': 'c426887c-43fa-4c7c-9ee6-447f48c5b353', 'keypoint': {'x': 463.6132, 'y': 487.9953}, 'name': 'yellow'}, {'id': 'f1402f90-fe95-42c0-93e5-75019494beba', 'keypoint': {'x': 497.0802, 'y': 442.0755}, 'name': 'yellow'}, {'id': 'ad7b2c3b-efac-4705-8dc4-076514da9f0d', 'keypoint': {'x': 319.6274, 'y': 463.8679}, 'name': 'blue'}, {'id': '57fd11a6-5352-4e90-8c5e-78e080a9f63b', 'keypoint': {'x': 458.1651, 'y': 516.0142}, 'name': 'blue'}, {'id': 'bffb87fb-647e-44e1-b4c0-9bbf464cc4da', 'keypoint': {'x': 353.8726, 'y': 445.1887}, 'name': 'green'}, {'id': '2be751d9-34f0-411b-89e7-1b20b054e1ee', 'keypoint': {'x': 388.8962, 'y': 498.1132}, 'name': 'green'}, {'id': '98a41430-e570-492e-969a-8ea45857d765', 'keypoint': {'x': 395.9009, 'y': 533.1368}, 'name': 'green'}, {'id': '0327cb90-c17c-4528-8964-e978fbff5999', 'keypoint': {'x': 427.8113, 'y': 484.8821}, 'name': 'green'}, {'id': '97f91dd8-6cd8-4194-8783-e9e3b5786e68', 'keypoint': {'x': 456.6085, 'y': 452.9717}, 'name': 'green'}, {'id': '4ebadfd4-3497-4f80-a4e9-a870c25786bd', 'keypoint': {'x': 504.8632, 'y': 502.0047}, 'name': 'green'}, {'id': 'd2ff782c-210c-4090-91a5-de43a78f1232', 'keypoint': {'x': 331.3019, 'y': 512.9009}, 'name': 'orange'}]}
[/INST]
{'x': 463.6132, 'y': 487.9953}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:To your right, grab the block closest to you that is crooked.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'b1c926ae-92c1-43eb-96c5-76c41d0fab06', 'keypoint': {'x': 342.1981, 'y': 437.4057}, 'name': 'orange'}, {'id': '76267614-95a1-4ded-b4e9-354b821f62bb', 'keypoint': {'x': 314.1792, 'y': 499.6698}, 'name': 'orange'}, {'id': '3124e476-176a-4feb-8844-b3973d48973a', 'keypoint': {'x': 453.4953, 'y': 463.8679}, 'name': 'orange'}, {'id': '728bd5ee-ece2-45c9-b0d7-b887692bb453', 'keypoint': {'x': 494.7453, 'y': 534.6934}, 'name': 'orange'}, {'id': 'da0db0c2-fdee-4550-8e2a-11f1aace764f', 'keypoint': {'x': 376.4434, 'y': 419.5047}, 'name': 'yellow'}, {'id': 'bd27285d-35e5-47b5-9b1e-78bd5fb9e595', 'keypoint': {'x': 360.0991, 'y': 516.7925}, 'name': 'yellow'}, {'id': '5a97f3d8-463a-4c65-9906-b8cf5f772e2b', 'keypoint': {'x': 408.3538, 'y': 526.9104}, 'name': 'yellow'}, {'id': 'f2444027-7c76-40d2-9424-36c46342a49f', 'keypoint': {'x': 318.8491, 'y': 449.0802}, 'name': 'blue'}, {'id': '0b2e3fca-8a71-43d2-a83e-f02af09891b9', 'keypoint': {'x': 315.7358, 'y': 480.2123}, 'name': 'blue'}, {'id': '897e21a9-0e62-42b6-8c01-826ad479026c', 'keypoint': {'x': 378.0, 'y': 490.3302}, 'name': 'blue'}, {'id': 'ca6c0f72-0600-47ba-b4b1-05dc3566a83e', 'keypoint': {'x': 402.1274, 'y': 463.0896}, 'name': 'blue'}, {'id': '41397d83-cab1-4c4d-ba20-b13a1c93f043', 'keypoint': {'x': 419.25, 'y': 432.7358}, 'name': 'blue'}, {'id': '051bcea0-383b-4791-9652-d84f6ef3fe4c', 'keypoint': {'x': 348.4245, 'y': 488.7736}, 'name': 'green'}, {'id': '58f619eb-2384-4a80-97f9-10520e5226a9', 'keypoint': {'x': 311.8443, 'y': 539.3632}, 'name': 'green'}, {'id': 'b9c15fa4-b07d-4f16-9f7d-f007698c83c2', 'keypoint': {'x': 417.6934, 'y': 502.0047}, 'name': 'green'}, {'id': '5c0026dd-e833-4ad7-8339-361bad3686e0', 'keypoint': {'x': 378.0, 'y': 59.1509}, 'name': 'you'}]}
[/INST]
{'x': 342.1981, 'y': 437.4057}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the orange block that is closest to you.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'b1c926ae-92c1-43eb-96c5-76c41d0fab06', 'keypoint': {'x': 342.1981, 'y': 437.4057}, 'name': 'orange'}, {'id': '76267614-95a1-4ded-b4e9-354b821f62bb', 'keypoint': {'x': 314.1792, 'y': 499.6698}, 'name': 'orange'}, {'id': '3124e476-176a-4feb-8844-b3973d48973a', 'keypoint': {'x': 453.4953, 'y': 463.8679}, 'name': 'orange'}, {'id': '728bd5ee-ece2-45c9-b0d7-b887692bb453', 'keypoint': {'x': 494.7453, 'y': 534.6934}, 'name': 'orange'}, {'id': 'da0db0c2-fdee-4550-8e2a-11f1aace764f', 'keypoint': {'x': 376.4434, 'y': 419.5047}, 'name': 'yellow'}, {'id': 'bd27285d-35e5-47b5-9b1e-78bd5fb9e595', 'keypoint': {'x': 360.0991, 'y': 516.7925}, 'name': 'yellow'}, {'id': '5a97f3d8-463a-4c65-9906-b8cf5f772e2b', 'keypoint': {'x': 408.3538, 'y': 526.9104}, 'name': 'yellow'}, {'id': 'f2444027-7c76-40d2-9424-36c46342a49f', 'keypoint': {'x': 318.8491, 'y': 449.0802}, 'name': 'blue'}, {'id': '0b2e3fca-8a71-43d2-a83e-f02af09891b9', 'keypoint': {'x': 315.7358, 'y': 480.2123}, 'name': 'blue'}, {'id': '897e21a9-0e62-42b6-8c01-826ad479026c', 'keypoint': {'x': 378.0, 'y': 490.3302}, 'name': 'blue'}, {'id': 'ca6c0f72-0600-47ba-b4b1-05dc3566a83e', 'keypoint': {'x': 402.1274, 'y': 463.0896}, 'name': 'blue'}, {'id': '41397d83-cab1-4c4d-ba20-b13a1c93f043', 'keypoint': {'x': 419.25, 'y': 432.7358}, 'name': 'blue'}, {'id': '051bcea0-383b-4791-9652-d84f6ef3fe4c', 'keypoint': {'x': 348.4245, 'y': 488.7736}, 'name': 'green'}, {'id': '58f619eb-2384-4a80-97f9-10520e5226a9', 'keypoint': {'x': 311.8443, 'y': 539.3632}, 'name': 'green'}, {'id': 'b9c15fa4-b07d-4f16-9f7d-f007698c83c2', 'keypoint': {'x': 417.6934, 'y': 502.0047}, 'name': 'green'}, {'id': '5c0026dd-e833-4ad7-8339-361bad3686e0', 'keypoint': {'x': 378.0, 'y': 59.1509}, 'name': 'you'}]}
[/INST]
{'x': 342.1981, 'y': 437.4057}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:please pick up the green block in the middle that has two orange ones underneath
 {'width': 756, 'height': 660, 'annotations': [{'id': 'd5bcc2b7-ff91-4686-8f50-3a73ffd3beeb', 'keypoint': {'x': 315.7358, 'y': 436.6274}, 'name': 'orange'}, {'id': '70515b86-98d5-4fa6-9118-a0bc484f6de1', 'keypoint': {'x': 313.4009, 'y': 525.3538}, 'name': 'orange'}, {'id': 'b40b0958-f706-45d2-a042-0440eb67121b', 'keypoint': {'x': 380.3349, 'y': 515.2358}, 'name': 'orange'}, {'id': '9dcd051f-bad9-4f84-b653-9a30a72cb545', 'keypoint': {'x': 389.6745, 'y': 552.5943}, 'name': 'orange'}, {'id': 'b0de6329-f4d9-486c-91ed-478f28262cff', 'keypoint': {'x': 363.9906, 'y': 439.7406}, 'name': 'yellow'}, {'id': '10eeedc5-3253-42e7-b401-33c4616e4623', 'keypoint': {'x': 396.6792, 'y': 457.6415}, 'name': 'yellow'}, {'id': 'f7642eee-44ca-4f1d-abf7-a79d43dfbd0d', 'keypoint': {'x': 279.934, 'y': 535.4717}, 'name': 'yellow'}, {'id': 'd87c603d-89da-4c0b-8f61-a496275c1297', 'keypoint': {'x': 311.066, 'y': 487.217}, 'name': 'blue'}, {'id': '2065d0e4-12eb-4ab3-bde4-75688bb3c142', 'keypoint': {'x': 457.3868, 'y': 439.7406}, 'name': 'green'}, {'id': '1550df58-31c4-49f4-8e6d-7db4a57bc294', 'keypoint': {'x': 435.5943, 'y': 445.967}, 'name': 'green'}, {'id': '1990bf50-38ac-43ab-8581-26778cea80ed', 'keypoint': {'x': 458.9434, 'y': 484.8821}, 'name': 'green'}, {'id': '7895d85c-2e57-4008-b9d7-74625608a42c', 'keypoint': {'x': 485.4057, 'y': 515.2358}, 'name': 'green'}, {'id': '991a571a-03aa-4a00-b80d-3d6a0845625b', 'keypoint': {'x': 440.2642, 'y': 514.4575}, 'name': 'green'}, {'id': 'e0e1086c-3a47-42f6-98fd-868d0756cd4d', 'keypoint': {'x': 378.7783, 'y': 495.7783}, 'name': 'green'}, {'id': '3d132917-dd96-4ae9-9302-da261696357f', 'keypoint': {'x': 333.6368, 'y': 505.1179}, 'name': 'green'}, {'id': 'bc7494f1-b049-49c6-bf24-85e942969065', 'keypoint': {'x': 383.4481, 'y': 56.0377}, 'name': 'you'}]}
[/INST]
{'x': 378.7783, 'y': 495.7783}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:grab the yellow block next to the blue block on your right.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'b328ac50-08b5-486e-a937-2cddbc04245c', 'keypoint': {'x': 304.8396, 'y': 434.2925}, 'name': 'orange'}, {'id': '7344b92c-5683-4670-9cef-33f4cf8e73fb', 'keypoint': {'x': 472.9528, 'y': 456.0849}, 'name': 'orange'}, {'id': '6958bbb7-3dce-4cfe-9049-02ebc5a32e43', 'keypoint': {'x': 306.3962, 'y': 460.7547}, 'name': 'yellow'}, {'id': '62285f29-522c-498a-9fdb-53e2e8421ea5', 'keypoint': {'x': 330.5236, 'y': 470.8726}, 'name': 'yellow'}, {'id': 'e4402e53-f01e-4254-a874-f6ffd3892ac4', 'keypoint': {'x': 381.8915, 'y': 475.5425}, 'name': 'yellow'}, {'id': '3f3ed2fa-5312-418f-9f29-f4750a72aa9e', 'keypoint': {'x': 402.9057, 'y': 435.0708}, 'name': 'yellow'}, {'id': 'fe5ba8f4-1f70-4712-ae25-cd20ac9670d6', 'keypoint': {'x': 455.8302, 'y': 481.7689}, 'name': 'yellow'}, {'id': 'bd1c4ddb-8345-4f24-a982-f6cd6a64fb0e', 'keypoint': {'x': 501.75, 'y': 505.8962}, 'name': 'yellow'}, {'id': '8f332be0-ad10-4130-9840-f3a1dc330890', 'keypoint': {'x': 352.316, 'y': 513.6792}, 'name': 'yellow'}, {'id': '5b19adbf-7efb-4a90-9f40-4c0bf2f3b9a8', 'keypoint': {'x': 324.2972, 'y': 499.6698}, 'name': 'yellow'}, {'id': '7f124547-f8b7-4280-b7e1-401876ce2bdb', 'keypoint': {'x': 308.7311, 'y': 552.5943}, 'name': 'yellow'}, {'id': '8a1fd773-bc7c-4be5-a292-c1c53d5c77df', 'keypoint': {'x': 298.6132, 'y': 491.8868}, 'name': 'blue'}, {'id': '7be8714d-676d-466e-a70a-e9d078b757f4', 'keypoint': {'x': 381.1132, 'y': 497.3349}, 'name': 'blue'}, {'id': 'b1ee1721-f1d2-4cc8-828b-2e6c5072d54f', 'keypoint': {'x': 451.9387, 'y': 432.7358}, 'name': 'blue'}, {'id': '0416eebd-320e-40d8-bbc7-a94a6333aa95', 'keypoint': {'x': 497.0802, 'y': 485.6604}, 'name': 'green'}, {'id': '2240b1ab-d515-4872-a587-09ca836dc397', 'keypoint': {'x': 378.7783, 'y': 28.7972}, 'name': 'you'}]}
[/INST]
{'x': 324.2972, 'y': 499.6698}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the green block furthest from you.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'f8ce6e33-d9e3-4687-bd11-4537937b3def', 'keypoint': {'x': 313.4009, 'y': 480.9906}, 'name': 'orange'}, {'id': '8758b8e0-78a7-4dc2-b595-9eec97d96b3c', 'keypoint': {'x': 403.684, 'y': 466.9811}, 'name': 'orange'}, {'id': '9024f068-b46e-400e-bf69-507f61b74f80', 'keypoint': {'x': 459.7217, 'y': 452.9717}, 'name': 'orange'}, {'id': 'd7ab991d-e3d6-4ffb-8d9b-9cffbb59b9c6', 'keypoint': {'x': 502.5283, 'y': 457.6415}, 'name': 'orange'}, {'id': 'b9fb860f-d2d0-4b4a-a275-416d46a99ab8', 'keypoint': {'x': 335.9717, 'y': 450.6368}, 'name': 'yellow'}, {'id': '7404a736-3ae3-4f5c-b40d-5d5af4635c6e', 'keypoint': {'x': 316.5142, 'y': 531.5802}, 'name': 'yellow'}, {'id': 'd2d6b9ff-fd85-48a1-b368-3f0e1d09cc6f', 'keypoint': {'x': 396.6792, 'y': 491.8868}, 'name': 'yellow'}, {'id': 'c4551bcd-78f8-4205-b828-5b040d49a2c3', 'keypoint': {'x': 444.1557, 'y': 433.5142}, 'name': 'yellow'}, {'id': '3e0cda0a-1e8a-4472-8984-7886c02ff560', 'keypoint': {'x': 347.6462, 'y': 514.4575}, 'name': 'blue'}, {'id': '266b6be1-582b-4c4d-8544-906fad90cbd2', 'keypoint': {'x': 359.3208, 'y': 478.6557}, 'name': 'blue'}, {'id': 'c58796fa-46b4-4231-8589-0612cd52983f', 'keypoint': {'x': 395.9009, 'y': 444.4104}, 'name': 'blue'}, {'id': '3de58fe6-02ae-4e5a-b0eb-ac1e3ac60960', 'keypoint': {'x': 444.934, 'y': 537.8066}, 'name': 'blue'}, {'id': 'a1a96c53-7f31-4121-8983-bf6a69f42935', 'keypoint': {'x': 502.5283, 'y': 541.6981}, 'name': 'blue'}, {'id': 'c2e223f1-9da7-4029-ab29-81edb6be3c94', 'keypoint': {'x': 355.4292, 'y': 424.9528}, 'name': 'green'}, {'id': 'd1f0facf-050a-4a92-bfb1-1367c5fdd80d', 'keypoint': {'x': 466.7264, 'y': 500.4481}, 'name': 'green'}, {'id': '727b2df8-1454-47db-9eff-8cfe072974ff', 'keypoint': {'x': 383.4481, 'y': 44.3632}, 'name': 'you'}]}
[/INST]
{'x': 466.7264, 'y': 500.4481}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the green block that is centered between the two sets of two green blocks that are on your left.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'd5bcc2b7-ff91-4686-8f50-3a73ffd3beeb', 'keypoint': {'x': 315.7358, 'y': 436.6274}, 'name': 'orange'}, {'id': '70515b86-98d5-4fa6-9118-a0bc484f6de1', 'keypoint': {'x': 313.4009, 'y': 525.3538}, 'name': 'orange'}, {'id': 'b40b0958-f706-45d2-a042-0440eb67121b', 'keypoint': {'x': 380.3349, 'y': 515.2358}, 'name': 'orange'}, {'id': '9dcd051f-bad9-4f84-b653-9a30a72cb545', 'keypoint': {'x': 389.6745, 'y': 552.5943}, 'name': 'orange'}, {'id': 'b0de6329-f4d9-486c-91ed-478f28262cff', 'keypoint': {'x': 363.9906, 'y': 439.7406}, 'name': 'yellow'}, {'id': '10eeedc5-3253-42e7-b401-33c4616e4623', 'keypoint': {'x': 396.6792, 'y': 457.6415}, 'name': 'yellow'}, {'id': 'f7642eee-44ca-4f1d-abf7-a79d43dfbd0d', 'keypoint': {'x': 279.934, 'y': 535.4717}, 'name': 'yellow'}, {'id': 'd87c603d-89da-4c0b-8f61-a496275c1297', 'keypoint': {'x': 311.066, 'y': 487.217}, 'name': 'blue'}, {'id': '2065d0e4-12eb-4ab3-bde4-75688bb3c142', 'keypoint': {'x': 457.3868, 'y': 439.7406}, 'name': 'green'}, {'id': '1550df58-31c4-49f4-8e6d-7db4a57bc294', 'keypoint': {'x': 435.5943, 'y': 445.967}, 'name': 'green'}, {'id': '1990bf50-38ac-43ab-8581-26778cea80ed', 'keypoint': {'x': 458.9434, 'y': 484.8821}, 'name': 'green'}, {'id': '7895d85c-2e57-4008-b9d7-74625608a42c', 'keypoint': {'x': 485.4057, 'y': 515.2358}, 'name': 'green'}, {'id': '991a571a-03aa-4a00-b80d-3d6a0845625b', 'keypoint': {'x': 440.2642, 'y': 514.4575}, 'name': 'green'}, {'id': 'e0e1086c-3a47-42f6-98fd-868d0756cd4d', 'keypoint': {'x': 378.7783, 'y': 495.7783}, 'name': 'green'}, {'id': '3d132917-dd96-4ae9-9302-da261696357f', 'keypoint': {'x': 333.6368, 'y': 505.1179}, 'name': 'green'}, {'id': 'bc7494f1-b049-49c6-bf24-85e942969065', 'keypoint': {'x': 383.4481, 'y': 56.0377}, 'name': 'you'}]}
[/INST]
{'x': 458.9434, 'y': 484.8821}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the rightmost blue block.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'bb2508d6-bab0-4991-a4b4-1d60addd89be', 'keypoint': {'x': 374.8868, 'y': 32.6887}, 'name': 'you'}, {'id': '80bad823-3c11-42a6-9b7a-8f92fd053db8', 'keypoint': {'x': 286.1604, 'y': 504.3396}, 'name': 'yellow'}, {'id': '74407b61-1d02-4456-a3be-f44e6d0621e4', 'keypoint': {'x': 427.8113, 'y': 511.3443}, 'name': 'yellow'}, {'id': '0c043462-b620-40d3-b17e-3f17c7108248', 'keypoint': {'x': 391.2311, 'y': 435.8491}, 'name': 'green'}, {'id': 'e97e7600-f671-4774-970c-bcd8e011471e', 'keypoint': {'x': 311.066, 'y': 530.8019}, 'name': 'green'}, {'id': '3c45bdee-a533-4cab-8e50-cd3e05ea379a', 'keypoint': {'x': 375.6651, 'y': 518.3491}, 'name': 'green'}, {'id': 'fbe906a6-d83a-45bc-bd1e-7c81ccde8b9b', 'keypoint': {'x': 487.7406, 'y': 473.2075}, 'name': 'green'}, {'id': '662455ed-994d-45d8-b864-0f07ccfe9a1e', 'keypoint': {'x': 309.5094, 'y': 477.8774}, 'name': 'blue'}, {'id': '92e619ee-cc43-4ba6-80a0-fd23cabc2b68', 'keypoint': {'x': 378.7783, 'y': 457.6415}, 'name': 'blue'}, {'id': '4d3a4397-37ee-4ea1-bff7-ea4e40e4ba91', 'keypoint': {'x': 395.1226, 'y': 483.3255}, 'name': 'blue'}, {'id': '83c6c716-2497-4e0e-b233-3cda2dca8049', 'keypoint': {'x': 438.7075, 'y': 448.3019}, 'name': 'blue'}, {'id': '24b23d82-8281-40fd-9942-dc852b76fea4', 'keypoint': {'x': 444.1557, 'y': 484.1038}, 'name': 'blue'}, {'id': '541179c1-e37f-4069-99a8-fe58be3053c6', 'keypoint': {'x': 466.7264, 'y': 502.783}, 'name': 'blue'}, {'id': 'd950b6f4-0fff-425b-bf69-419e3a15863f', 'keypoint': {'x': 343.7547, 'y': 543.2547}, 'name': 'orange'}, {'id': 'a5aa3394-3b02-4613-8a7d-f14c9cd0fa58', 'keypoint': {'x': 431.7028, 'y': 539.3632}, 'name': 'orange'}, {'id': 'd7eb7db6-f69d-4df7-861e-0cc9dbebac2a', 'keypoint': {'x': 484.6274, 'y': 532.3585}, 'name': 'orange'}]}
[/INST]
{'x': 309.5094, 'y': 477.8774}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Look for a green block. Pick up the green block furthest from you.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'cad61a18-85b8-4f6e-93b3-c3ce9e28cda2', 'keypoint': {'x': 316.5142, 'y': 554.9292}, 'name': 'orange'}, {'id': '17682384-25d2-46d1-ae87-f04077458497', 'keypoint': {'x': 330.5236, 'y': 516.7925}, 'name': 'orange'}, {'id': '5d36ae41-004b-494e-b32d-33279b1dbfee', 'keypoint': {'x': 447.2689, 'y': 457.6415}, 'name': 'orange'}, {'id': 'fee58518-97d8-4c98-9b1b-ebeaa7ef3967', 'keypoint': {'x': 488.5189, 'y': 530.8019}, 'name': 'orange'}, {'id': '4fa5b6b4-f98b-4588-95e6-8255a4bef84b', 'keypoint': {'x': 380.3349, 'y': 446.7453}, 'name': 'yellow'}, {'id': '22f0cd3c-99be-40d9-9b3f-a7bc782da1f8', 'keypoint': {'x': 297.8349, 'y': 529.2453}, 'name': 'yellow'}, {'id': 'd5f859c2-a998-4ee4-bff1-d6025abd4883', 'keypoint': {'x': 375.6651, 'y': 536.25}, 'name': 'yellow'}, {'id': '96691580-8803-4b4b-b31f-4920d402e03e', 'keypoint': {'x': 473.7311, 'y': 435.0708}, 'name': 'yellow'}, {'id': '86653055-abe8-4a82-8b79-fb7f461438c0', 'keypoint': {'x': 474.5094, 'y': 487.217}, 'name': 'yellow'}, {'id': '34e55929-db77-4549-8f50-5adc4da9bd5d', 'keypoint': {'x': 370.9953, 'y': 492.6651}, 'name': 'blue'}, {'id': '500fbe66-22dd-4552-a43c-63bf2efeb1c9', 'keypoint': {'x': 417.6934, 'y': 477.0991}, 'name': 'blue'}, {'id': '85cb36b6-c443-488a-85b6-363a2284870a', 'keypoint': {'x': 345.3113, 'y': 424.9528}, 'name': 'green'}, {'id': '1c037117-1e04-4cd7-a544-c3e10b3639bb', 'keypoint': {'x': 295.5, 'y': 466.9811}, 'name': 'green'}, {'id': 'a8f28480-33ff-4a0a-883f-f1f226ced914', 'keypoint': {'x': 299.3915, 'y': 501.2264}, 'name': 'green'}, {'id': 'b9578593-4dcd-4808-9119-c6439b45d587', 'keypoint': {'x': 427.033, 'y': 534.6934}, 'name': 'green'}, {'id': '70024c2f-19b2-412a-a76d-d9e741635b7e', 'keypoint': {'x': 380.3349, 'y': 33.467}, 'name': 'you'}]}
[/INST]
{'x': 427.033, 'y': 534.6934}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the green block at the top left portion of the blocks.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'f8ce6e33-d9e3-4687-bd11-4537937b3def', 'keypoint': {'x': 313.4009, 'y': 480.9906}, 'name': 'orange'}, {'id': '8758b8e0-78a7-4dc2-b595-9eec97d96b3c', 'keypoint': {'x': 403.684, 'y': 466.9811}, 'name': 'orange'}, {'id': '9024f068-b46e-400e-bf69-507f61b74f80', 'keypoint': {'x': 459.7217, 'y': 452.9717}, 'name': 'orange'}, {'id': 'd7ab991d-e3d6-4ffb-8d9b-9cffbb59b9c6', 'keypoint': {'x': 502.5283, 'y': 457.6415}, 'name': 'orange'}, {'id': 'b9fb860f-d2d0-4b4a-a275-416d46a99ab8', 'keypoint': {'x': 335.9717, 'y': 450.6368}, 'name': 'yellow'}, {'id': '7404a736-3ae3-4f5c-b40d-5d5af4635c6e', 'keypoint': {'x': 316.5142, 'y': 531.5802}, 'name': 'yellow'}, {'id': 'd2d6b9ff-fd85-48a1-b368-3f0e1d09cc6f', 'keypoint': {'x': 396.6792, 'y': 491.8868}, 'name': 'yellow'}, {'id': 'c4551bcd-78f8-4205-b828-5b040d49a2c3', 'keypoint': {'x': 444.1557, 'y': 433.5142}, 'name': 'yellow'}, {'id': '3e0cda0a-1e8a-4472-8984-7886c02ff560', 'keypoint': {'x': 347.6462, 'y': 514.4575}, 'name': 'blue'}, {'id': '266b6be1-582b-4c4d-8544-906fad90cbd2', 'keypoint': {'x': 359.3208, 'y': 478.6557}, 'name': 'blue'}, {'id': 'c58796fa-46b4-4231-8589-0612cd52983f', 'keypoint': {'x': 395.9009, 'y': 444.4104}, 'name': 'blue'}, {'id': '3de58fe6-02ae-4e5a-b0eb-ac1e3ac60960', 'keypoint': {'x': 444.934, 'y': 537.8066}, 'name': 'blue'}, {'id': 'a1a96c53-7f31-4121-8983-bf6a69f42935', 'keypoint': {'x': 502.5283, 'y': 541.6981}, 'name': 'blue'}, {'id': 'c2e223f1-9da7-4029-ab29-81edb6be3c94', 'keypoint': {'x': 355.4292, 'y': 424.9528}, 'name': 'green'}, {'id': 'd1f0facf-050a-4a92-bfb1-1367c5fdd80d', 'keypoint': {'x': 466.7264, 'y': 500.4481}, 'name': 'green'}, {'id': '727b2df8-1454-47db-9eff-8cfe072974ff', 'keypoint': {'x': 383.4481, 'y': 44.3632}, 'name': 'you'}]}
[/INST]
{'x': 466.7264, 'y': 500.4481}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:pick up the yellow block that is located near the front but between the two blue blocks on your right. It's the yellow block on the right.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'b328ac50-08b5-486e-a937-2cddbc04245c', 'keypoint': {'x': 304.8396, 'y': 434.2925}, 'name': 'orange'}, {'id': '7344b92c-5683-4670-9cef-33f4cf8e73fb', 'keypoint': {'x': 472.9528, 'y': 456.0849}, 'name': 'orange'}, {'id': '6958bbb7-3dce-4cfe-9049-02ebc5a32e43', 'keypoint': {'x': 306.3962, 'y': 460.7547}, 'name': 'yellow'}, {'id': '62285f29-522c-498a-9fdb-53e2e8421ea5', 'keypoint': {'x': 330.5236, 'y': 470.8726}, 'name': 'yellow'}, {'id': 'e4402e53-f01e-4254-a874-f6ffd3892ac4', 'keypoint': {'x': 381.8915, 'y': 475.5425}, 'name': 'yellow'}, {'id': '3f3ed2fa-5312-418f-9f29-f4750a72aa9e', 'keypoint': {'x': 402.9057, 'y': 435.0708}, 'name': 'yellow'}, {'id': 'fe5ba8f4-1f70-4712-ae25-cd20ac9670d6', 'keypoint': {'x': 455.8302, 'y': 481.7689}, 'name': 'yellow'}, {'id': 'bd1c4ddb-8345-4f24-a982-f6cd6a64fb0e', 'keypoint': {'x': 501.75, 'y': 505.8962}, 'name': 'yellow'}, {'id': '8f332be0-ad10-4130-9840-f3a1dc330890', 'keypoint': {'x': 352.316, 'y': 513.6792}, 'name': 'yellow'}, {'id': '5b19adbf-7efb-4a90-9f40-4c0bf2f3b9a8', 'keypoint': {'x': 324.2972, 'y': 499.6698}, 'name': 'yellow'}, {'id': '7f124547-f8b7-4280-b7e1-401876ce2bdb', 'keypoint': {'x': 308.7311, 'y': 552.5943}, 'name': 'yellow'}, {'id': '8a1fd773-bc7c-4be5-a292-c1c53d5c77df', 'keypoint': {'x': 298.6132, 'y': 491.8868}, 'name': 'blue'}, {'id': '7be8714d-676d-466e-a70a-e9d078b757f4', 'keypoint': {'x': 381.1132, 'y': 497.3349}, 'name': 'blue'}, {'id': 'b1ee1721-f1d2-4cc8-828b-2e6c5072d54f', 'keypoint': {'x': 451.9387, 'y': 432.7358}, 'name': 'blue'}, {'id': '0416eebd-320e-40d8-bbc7-a94a6333aa95', 'keypoint': {'x': 497.0802, 'y': 485.6604}, 'name': 'green'}, {'id': '2240b1ab-d515-4872-a587-09ca836dc397', 'keypoint': {'x': 378.7783, 'y': 28.7972}, 'name': 'you'}]}
[/INST]
{'x': 324.2972, 'y': 499.6698}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Of the two green boxes that are on your left, choose the one that is closer to you.
 {'width': 756, 'height': 660, 'annotations': [{'id': '8f941e4d-8e6b-4e37-a2cd-f19b3dbb1dda', 'keypoint': {'x': 293.1651, 'y': 452.9717}, 'name': 'orange'}, {'id': '77f8b560-0c54-4923-9c65-8e8973d5cc1a', 'keypoint': {'x': 293.9434, 'y': 485.6604}, 'name': 'orange'}, {'id': 'c2fdf254-12fa-43ee-9d04-fd097d6a2fad', 'keypoint': {'x': 332.8585, 'y': 517.5708}, 'name': 'orange'}, {'id': 'ee022624-310e-4867-8360-cd61e6a5e8ec', 'keypoint': {'x': 325.0755, 'y': 557.2642}, 'name': 'orange'}, {'id': 'ca78545f-1817-447e-a767-1ec186e40025', 'keypoint': {'x': 377.2217, 'y': 478.6557}, 'name': 'orange'}, {'id': 'b7ea891b-3095-4df8-a718-b193f34c851c', 'keypoint': {'x': 386.5613, 'y': 492.6651}, 'name': 'orange'}, {'id': '54b85951-9b7d-4a22-9b94-5856c8f4cc7e', 'keypoint': {'x': 441.8208, 'y': 499.6698}, 'name': 'orange'}, {'id': 'ef2b7cbb-fa7e-4894-8e50-310d43a6f3eb', 'keypoint': {'x': 385.0047, 'y': 448.3019}, 'name': 'yellow'}, {'id': 'ab7d8b08-4eaf-41cf-8066-9aba376bb546', 'keypoint': {'x': 392.0094, 'y': 523.0189}, 'name': 'yellow'}, {'id': 'aec059f3-2def-4458-9ea4-3fa5916cab5b', 'keypoint': {'x': 466.7264, 'y': 442.8538}, 'name': 'yellow'}, {'id': '39fe094d-1e93-4828-ae7e-d672aeb88451', 'keypoint': {'x': 356.2075, 'y': 533.9151}, 'name': 'blue'}, {'id': '57cde523-a685-45a5-b623-1b11057840da', 'keypoint': {'x': 487.7406, 'y': 488.7736}, 'name': 'blue'}, {'id': 'da91e8aa-4087-4689-95ed-80e4ba9a93ec', 'keypoint': {'x': 327.4104, 'y': 472.4292}, 'name': 'green'}, {'id': '9406ceb5-4de6-4915-8da6-132b32721d7c', 'keypoint': {'x': 508.7547, 'y': 511.3443}, 'name': 'green'}, {'id': 'fd93334e-be71-4598-9c11-abb001661b87', 'keypoint': {'x': 497.0802, 'y': 560.3774}, 'name': 'green'}, {'id': '3067ae97-d698-4a21-82f6-e94a33ed51a0', 'keypoint': {'x': 380.3349, 'y': 59.1509}, 'name': 'you'}]}
[/INST]
{'x': 508.7547, 'y': 511.3443}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Grab the green block that is closest to you.
 {'width': 756, 'height': 660, 'annotations': [{'id': '4665780d-e410-4494-8512-3557afddb1a4', 'keypoint': {'x': 311.8443, 'y': 504.3396}, 'name': None}, {'id': '565e6ab8-2ba1-4136-96ed-e63f3cdba23d', 'keypoint': {'x': 368.6604, 'y': 505.1179}, 'name': 'orange'}, {'id': '487a86d1-7908-404f-99e9-16ab87ea0d04', 'keypoint': {'x': 359.3208, 'y': 560.3774}, 'name': 'orange'}, {'id': 'ab7b691c-c47b-4c26-8083-096140fac645', 'keypoint': {'x': 413.0236, 'y': 447.5236}, 'name': 'orange'}, {'id': 'fdb6ba1e-6ab3-4d82-862c-1bdaf73940dc', 'keypoint': {'x': 464.3915, 'y': 459.9764}, 'name': 'orange'}, {'id': '7768e90f-d1af-4f66-abc0-01fa10bc5e61', 'keypoint': {'x': 447.2689, 'y': 530.8019}, 'name': 'orange'}, {'id': '698efbba-5197-4d9f-96eb-fdc8c950c48f', 'keypoint': {'x': 290.8302, 'y': 554.1509}, 'name': 'yellow'}, {'id': '1e9d76c9-a045-4417-94ae-129e3daeb548', 'keypoint': {'x': 407.5755, 'y': 519.9057}, 'name': 'blue'}, {'id': 'dc0ba9e0-8eb1-478a-88c5-3eb00d02992d', 'keypoint': {'x': 454.2736, 'y': 505.8962}, 'name': 'blue'}, {'id': '13165129-2db5-428d-ba05-982c1ae9a179', 'keypoint': {'x': 503.3066, 'y': 487.217}, 'name': 'green'}, {'id': 'a2feb4b6-353e-4876-b8e6-d5351bc5f056', 'keypoint': {'x': 459.7217, 'y': 490.3302}, 'name': 'green'}, {'id': 'bd636fd9-8ba5-408a-a286-413978cb55e7', 'keypoint': {'x': 362.434, 'y': 476.3208}, 'name': 'green'}, {'id': '4d316993-cf9a-4115-894b-940d569fafc4', 'keypoint': {'x': 311.066, 'y': 439.7406}, 'name': 'green'}, {'id': 'b6f5df26-1415-40e4-bbf7-01af31da49b9', 'keypoint': {'x': 380.3349, 'y': 544.8113}, 'name': 'green'}, {'id': 'f34f5fb5-74bd-4265-8123-8eed80fffdd5', 'keypoint': {'x': 322.7406, 'y': 536.25}, 'name': 'green'}, {'id': '847a9a8d-0e7a-49f2-b274-f833cdac716e', 'keypoint': {'x': 375.6651, 'y': 49.8113}, 'name': 'you'}]}
[/INST]
{'x': 311.066, 'y': 439.7406}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the green block closest to you.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'f8ce6e33-d9e3-4687-bd11-4537937b3def', 'keypoint': {'x': 313.4009, 'y': 480.9906}, 'name': 'orange'}, {'id': '8758b8e0-78a7-4dc2-b595-9eec97d96b3c', 'keypoint': {'x': 403.684, 'y': 466.9811}, 'name': 'orange'}, {'id': '9024f068-b46e-400e-bf69-507f61b74f80', 'keypoint': {'x': 459.7217, 'y': 452.9717}, 'name': 'orange'}, {'id': 'd7ab991d-e3d6-4ffb-8d9b-9cffbb59b9c6', 'keypoint': {'x': 502.5283, 'y': 457.6415}, 'name': 'orange'}, {'id': 'b9fb860f-d2d0-4b4a-a275-416d46a99ab8', 'keypoint': {'x': 335.9717, 'y': 450.6368}, 'name': 'yellow'}, {'id': '7404a736-3ae3-4f5c-b40d-5d5af4635c6e', 'keypoint': {'x': 316.5142, 'y': 531.5802}, 'name': 'yellow'}, {'id': 'd2d6b9ff-fd85-48a1-b368-3f0e1d09cc6f', 'keypoint': {'x': 396.6792, 'y': 491.8868}, 'name': 'yellow'}, {'id': 'c4551bcd-78f8-4205-b828-5b040d49a2c3', 'keypoint': {'x': 444.1557, 'y': 433.5142}, 'name': 'yellow'}, {'id': '3e0cda0a-1e8a-4472-8984-7886c02ff560', 'keypoint': {'x': 347.6462, 'y': 514.4575}, 'name': 'blue'}, {'id': '266b6be1-582b-4c4d-8544-906fad90cbd2', 'keypoint': {'x': 359.3208, 'y': 478.6557}, 'name': 'blue'}, {'id': 'c58796fa-46b4-4231-8589-0612cd52983f', 'keypoint': {'x': 395.9009, 'y': 444.4104}, 'name': 'blue'}, {'id': '3de58fe6-02ae-4e5a-b0eb-ac1e3ac60960', 'keypoint': {'x': 444.934, 'y': 537.8066}, 'name': 'blue'}, {'id': 'a1a96c53-7f31-4121-8983-bf6a69f42935', 'keypoint': {'x': 502.5283, 'y': 541.6981}, 'name': 'blue'}, {'id': 'c2e223f1-9da7-4029-ab29-81edb6be3c94', 'keypoint': {'x': 355.4292, 'y': 424.9528}, 'name': 'green'}, {'id': 'd1f0facf-050a-4a92-bfb1-1367c5fdd80d', 'keypoint': {'x': 466.7264, 'y': 500.4481}, 'name': 'green'}, {'id': '727b2df8-1454-47db-9eff-8cfe072974ff', 'keypoint': {'x': 383.4481, 'y': 44.3632}, 'name': 'you'}]}
[/INST]
{'x': 355.4292, 'y': 424.9528}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the furthest away green block.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'f8ce6e33-d9e3-4687-bd11-4537937b3def', 'keypoint': {'x': 313.4009, 'y': 480.9906}, 'name': 'orange'}, {'id': '8758b8e0-78a7-4dc2-b595-9eec97d96b3c', 'keypoint': {'x': 403.684, 'y': 466.9811}, 'name': 'orange'}, {'id': '9024f068-b46e-400e-bf69-507f61b74f80', 'keypoint': {'x': 459.7217, 'y': 452.9717}, 'name': 'orange'}, {'id': 'd7ab991d-e3d6-4ffb-8d9b-9cffbb59b9c6', 'keypoint': {'x': 502.5283, 'y': 457.6415}, 'name': 'orange'}, {'id': 'b9fb860f-d2d0-4b4a-a275-416d46a99ab8', 'keypoint': {'x': 335.9717, 'y': 450.6368}, 'name': 'yellow'}, {'id': '7404a736-3ae3-4f5c-b40d-5d5af4635c6e', 'keypoint': {'x': 316.5142, 'y': 531.5802}, 'name': 'yellow'}, {'id': 'd2d6b9ff-fd85-48a1-b368-3f0e1d09cc6f', 'keypoint': {'x': 396.6792, 'y': 491.8868}, 'name': 'yellow'}, {'id': 'c4551bcd-78f8-4205-b828-5b040d49a2c3', 'keypoint': {'x': 444.1557, 'y': 433.5142}, 'name': 'yellow'}, {'id': '3e0cda0a-1e8a-4472-8984-7886c02ff560', 'keypoint': {'x': 347.6462, 'y': 514.4575}, 'name': 'blue'}, {'id': '266b6be1-582b-4c4d-8544-906fad90cbd2', 'keypoint': {'x': 359.3208, 'y': 478.6557}, 'name': 'blue'}, {'id': 'c58796fa-46b4-4231-8589-0612cd52983f', 'keypoint': {'x': 395.9009, 'y': 444.4104}, 'name': 'blue'}, {'id': '3de58fe6-02ae-4e5a-b0eb-ac1e3ac60960', 'keypoint': {'x': 444.934, 'y': 537.8066}, 'name': 'blue'}, {'id': 'a1a96c53-7f31-4121-8983-bf6a69f42935', 'keypoint': {'x': 502.5283, 'y': 541.6981}, 'name': 'blue'}, {'id': 'c2e223f1-9da7-4029-ab29-81edb6be3c94', 'keypoint': {'x': 355.4292, 'y': 424.9528}, 'name': 'green'}, {'id': 'd1f0facf-050a-4a92-bfb1-1367c5fdd80d', 'keypoint': {'x': 466.7264, 'y': 500.4481}, 'name': 'green'}, {'id': '727b2df8-1454-47db-9eff-8cfe072974ff', 'keypoint': {'x': 383.4481, 'y': 44.3632}, 'name': 'you'}]}
[/INST]
{'x': 466.7264, 'y': 500.4481}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:I would tell the robot to pick up the yellow block that is directly in front of the blue block on the right hand side.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'fe98e63d-c226-4a25-9e75-229c9a2d828a', 'keypoint': {'x': 377.2217, 'y': 28.7972}, 'name': 'you'}, {'id': 'c1bfc6fe-9e0d-4152-aab0-9efe03f99ee1', 'keypoint': {'x': 291.6085, 'y': 487.217}, 'name': 'yellow'}, {'id': 'eefd5787-08ef-4674-ae9e-c41f4a2ed951', 'keypoint': {'x': 314.1792, 'y': 539.3632}, 'name': 'yellow'}, {'id': '8987e25d-8eb0-42ab-aa8e-390a7b046e44', 'keypoint': {'x': 363.2123, 'y': 544.8113}, 'name': 'yellow'}, {'id': 'af54d764-7469-4b39-95f2-d25ac6241715', 'keypoint': {'x': 434.0377, 'y': 542.4764}, 'name': 'yellow'}, {'id': 'c426887c-43fa-4c7c-9ee6-447f48c5b353', 'keypoint': {'x': 463.6132, 'y': 487.9953}, 'name': 'yellow'}, {'id': 'f1402f90-fe95-42c0-93e5-75019494beba', 'keypoint': {'x': 497.0802, 'y': 442.0755}, 'name': 'yellow'}, {'id': 'ad7b2c3b-efac-4705-8dc4-076514da9f0d', 'keypoint': {'x': 319.6274, 'y': 463.8679}, 'name': 'blue'}, {'id': '57fd11a6-5352-4e90-8c5e-78e080a9f63b', 'keypoint': {'x': 458.1651, 'y': 516.0142}, 'name': 'blue'}, {'id': 'bffb87fb-647e-44e1-b4c0-9bbf464cc4da', 'keypoint': {'x': 353.8726, 'y': 445.1887}, 'name': 'green'}, {'id': '2be751d9-34f0-411b-89e7-1b20b054e1ee', 'keypoint': {'x': 388.8962, 'y': 498.1132}, 'name': 'green'}, {'id': '98a41430-e570-492e-969a-8ea45857d765', 'keypoint': {'x': 395.9009, 'y': 533.1368}, 'name': 'green'}, {'id': '0327cb90-c17c-4528-8964-e978fbff5999', 'keypoint': {'x': 427.8113, 'y': 484.8821}, 'name': 'green'}, {'id': '97f91dd8-6cd8-4194-8783-e9e3b5786e68', 'keypoint': {'x': 456.6085, 'y': 452.9717}, 'name': 'green'}, {'id': '4ebadfd4-3497-4f80-a4e9-a870c25786bd', 'keypoint': {'x': 504.8632, 'y': 502.0047}, 'name': 'green'}, {'id': 'd2ff782c-210c-4090-91a5-de43a78f1232', 'keypoint': {'x': 331.3019, 'y': 512.9009}, 'name': 'orange'}]}
[/INST]
{'x': 463.6132, 'y': 487.9953}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the center most green block.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'd5bcc2b7-ff91-4686-8f50-3a73ffd3beeb', 'keypoint': {'x': 315.7358, 'y': 436.6274}, 'name': 'orange'}, {'id': '70515b86-98d5-4fa6-9118-a0bc484f6de1', 'keypoint': {'x': 313.4009, 'y': 525.3538}, 'name': 'orange'}, {'id': 'b40b0958-f706-45d2-a042-0440eb67121b', 'keypoint': {'x': 380.3349, 'y': 515.2358}, 'name': 'orange'}, {'id': '9dcd051f-bad9-4f84-b653-9a30a72cb545', 'keypoint': {'x': 389.6745, 'y': 552.5943}, 'name': 'orange'}, {'id': 'b0de6329-f4d9-486c-91ed-478f28262cff', 'keypoint': {'x': 363.9906, 'y': 439.7406}, 'name': 'yellow'}, {'id': '10eeedc5-3253-42e7-b401-33c4616e4623', 'keypoint': {'x': 396.6792, 'y': 457.6415}, 'name': 'yellow'}, {'id': 'f7642eee-44ca-4f1d-abf7-a79d43dfbd0d', 'keypoint': {'x': 279.934, 'y': 535.4717}, 'name': 'yellow'}, {'id': 'd87c603d-89da-4c0b-8f61-a496275c1297', 'keypoint': {'x': 311.066, 'y': 487.217}, 'name': 'blue'}, {'id': '2065d0e4-12eb-4ab3-bde4-75688bb3c142', 'keypoint': {'x': 457.3868, 'y': 439.7406}, 'name': 'green'}, {'id': '1550df58-31c4-49f4-8e6d-7db4a57bc294', 'keypoint': {'x': 435.5943, 'y': 445.967}, 'name': 'green'}, {'id': '1990bf50-38ac-43ab-8581-26778cea80ed', 'keypoint': {'x': 458.9434, 'y': 484.8821}, 'name': 'green'}, {'id': '7895d85c-2e57-4008-b9d7-74625608a42c', 'keypoint': {'x': 485.4057, 'y': 515.2358}, 'name': 'green'}, {'id': '991a571a-03aa-4a00-b80d-3d6a0845625b', 'keypoint': {'x': 440.2642, 'y': 514.4575}, 'name': 'green'}, {'id': 'e0e1086c-3a47-42f6-98fd-868d0756cd4d', 'keypoint': {'x': 378.7783, 'y': 495.7783}, 'name': 'green'}, {'id': '3d132917-dd96-4ae9-9302-da261696357f', 'keypoint': {'x': 333.6368, 'y': 505.1179}, 'name': 'green'}, {'id': 'bc7494f1-b049-49c6-bf24-85e942969065', 'keypoint': {'x': 383.4481, 'y': 56.0377}, 'name': 'you'}]}
[/INST]
{'x': 378.7783, 'y': 495.7783}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the blue block that is closest to you.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'fe98e63d-c226-4a25-9e75-229c9a2d828a', 'keypoint': {'x': 377.2217, 'y': 28.7972}, 'name': 'you'}, {'id': 'c1bfc6fe-9e0d-4152-aab0-9efe03f99ee1', 'keypoint': {'x': 291.6085, 'y': 487.217}, 'name': 'yellow'}, {'id': 'eefd5787-08ef-4674-ae9e-c41f4a2ed951', 'keypoint': {'x': 314.1792, 'y': 539.3632}, 'name': 'yellow'}, {'id': '8987e25d-8eb0-42ab-aa8e-390a7b046e44', 'keypoint': {'x': 363.2123, 'y': 544.8113}, 'name': 'yellow'}, {'id': 'af54d764-7469-4b39-95f2-d25ac6241715', 'keypoint': {'x': 434.0377, 'y': 542.4764}, 'name': 'yellow'}, {'id': 'c426887c-43fa-4c7c-9ee6-447f48c5b353', 'keypoint': {'x': 463.6132, 'y': 487.9953}, 'name': 'yellow'}, {'id': 'f1402f90-fe95-42c0-93e5-75019494beba', 'keypoint': {'x': 497.0802, 'y': 442.0755}, 'name': 'yellow'}, {'id': 'ad7b2c3b-efac-4705-8dc4-076514da9f0d', 'keypoint': {'x': 319.6274, 'y': 463.8679}, 'name': 'blue'}, {'id': '57fd11a6-5352-4e90-8c5e-78e080a9f63b', 'keypoint': {'x': 458.1651, 'y': 516.0142}, 'name': 'blue'}, {'id': 'bffb87fb-647e-44e1-b4c0-9bbf464cc4da', 'keypoint': {'x': 353.8726, 'y': 445.1887}, 'name': 'green'}, {'id': '2be751d9-34f0-411b-89e7-1b20b054e1ee', 'keypoint': {'x': 388.8962, 'y': 498.1132}, 'name': 'green'}, {'id': '98a41430-e570-492e-969a-8ea45857d765', 'keypoint': {'x': 395.9009, 'y': 533.1368}, 'name': 'green'}, {'id': '0327cb90-c17c-4528-8964-e978fbff5999', 'keypoint': {'x': 427.8113, 'y': 484.8821}, 'name': 'green'}, {'id': '97f91dd8-6cd8-4194-8783-e9e3b5786e68', 'keypoint': {'x': 456.6085, 'y': 452.9717}, 'name': 'green'}, {'id': '4ebadfd4-3497-4f80-a4e9-a870c25786bd', 'keypoint': {'x': 504.8632, 'y': 502.0047}, 'name': 'green'}, {'id': 'd2ff782c-210c-4090-91a5-de43a78f1232', 'keypoint': {'x': 331.3019, 'y': 512.9009}, 'name': 'orange'}]}
[/INST]
{'x': 319.6274, 'y': 463.8679}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:Pick up the green block that is right next to a blue block.
 {'width': 756, 'height': 660, 'annotations': [{'id': '8f941e4d-8e6b-4e37-a2cd-f19b3dbb1dda', 'keypoint': {'x': 293.1651, 'y': 452.9717}, 'name': 'orange'}, {'id': '77f8b560-0c54-4923-9c65-8e8973d5cc1a', 'keypoint': {'x': 293.9434, 'y': 485.6604}, 'name': 'orange'}, {'id': 'c2fdf254-12fa-43ee-9d04-fd097d6a2fad', 'keypoint': {'x': 332.8585, 'y': 517.5708}, 'name': 'orange'}, {'id': 'ee022624-310e-4867-8360-cd61e6a5e8ec', 'keypoint': {'x': 325.0755, 'y': 557.2642}, 'name': 'orange'}, {'id': 'ca78545f-1817-447e-a767-1ec186e40025', 'keypoint': {'x': 377.2217, 'y': 478.6557}, 'name': 'orange'}, {'id': 'b7ea891b-3095-4df8-a718-b193f34c851c', 'keypoint': {'x': 386.5613, 'y': 492.6651}, 'name': 'orange'}, {'id': '54b85951-9b7d-4a22-9b94-5856c8f4cc7e', 'keypoint': {'x': 441.8208, 'y': 499.6698}, 'name': 'orange'}, {'id': 'ef2b7cbb-fa7e-4894-8e50-310d43a6f3eb', 'keypoint': {'x': 385.0047, 'y': 448.3019}, 'name': 'yellow'}, {'id': 'ab7d8b08-4eaf-41cf-8066-9aba376bb546', 'keypoint': {'x': 392.0094, 'y': 523.0189}, 'name': 'yellow'}, {'id': 'aec059f3-2def-4458-9ea4-3fa5916cab5b', 'keypoint': {'x': 466.7264, 'y': 442.8538}, 'name': 'yellow'}, {'id': '39fe094d-1e93-4828-ae7e-d672aeb88451', 'keypoint': {'x': 356.2075, 'y': 533.9151}, 'name': 'blue'}, {'id': '57cde523-a685-45a5-b623-1b11057840da', 'keypoint': {'x': 487.7406, 'y': 488.7736}, 'name': 'blue'}, {'id': 'da91e8aa-4087-4689-95ed-80e4ba9a93ec', 'keypoint': {'x': 327.4104, 'y': 472.4292}, 'name': 'green'}, {'id': '9406ceb5-4de6-4915-8da6-132b32721d7c', 'keypoint': {'x': 508.7547, 'y': 511.3443}, 'name': 'green'}, {'id': 'fd93334e-be71-4598-9c11-abb001661b87', 'keypoint': {'x': 497.0802, 'y': 560.3774}, 'name': 'green'}, {'id': '3067ae97-d698-4a21-82f6-e94a33ed51a0', 'keypoint': {'x': 380.3349, 'y': 59.1509}, 'name': 'you'}]}
[/INST]
{'x': 508.7547, 'y': 511.3443}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:pick up the blue cube that is the second furthest away from you
 {'width': 756, 'height': 660, 'annotations': [{'id': 'c7519096-f7ea-4ab1-a66d-97c708976599', 'keypoint': {'x': 380.3349, 'y': 34.2453}, 'name': 'you'}, {'id': 'a95968ee-51d4-482d-ae79-97cb54663aed', 'keypoint': {'x': 293.9434, 'y': 479.434}, 'name': 'blue'}, {'id': '2fc8b299-6074-43eb-92bc-7ed758a958fc', 'keypoint': {'x': 329.7453, 'y': 557.2642}, 'name': 'blue'}, {'id': '11dbc045-de4f-4d3a-bc50-a1c2ba279bd1', 'keypoint': {'x': 366.3255, 'y': 546.3679}, 'name': 'blue'}, {'id': 'a3036737-fb7e-4f86-88c8-b0c5262cc051', 'keypoint': {'x': 379.5566, 'y': 438.9623}, 'name': 'blue'}, {'id': '5370f3f3-ef52-45e8-bdb4-3d96d54f4bd9', 'keypoint': {'x': 455.8302, 'y': 449.0802}, 'name': 'blue'}, {'id': '95294165-9354-41e7-b1be-60d4033bd1a3', 'keypoint': {'x': 318.0708, 'y': 454.5283}, 'name': 'yellow'}, {'id': '71f49ac9-1905-457f-9219-701cac7c42e4', 'keypoint': {'x': 318.0708, 'y': 530.8019}, 'name': 'yellow'}, {'id': '6f7a8a19-794f-4ffc-80ed-8a42dedeab0f', 'keypoint': {'x': 463.6132, 'y': 524.5755}, 'name': 'yellow'}, {'id': '407068c6-6a1e-4a14-b9d0-c0a4ed13b9c2', 'keypoint': {'x': 395.1226, 'y': 466.2028}, 'name': 'green'}, {'id': '847fb0f9-02cc-42df-8984-2b193e2abb8d', 'keypoint': {'x': 403.684, 'y': 523.0189}, 'name': 'green'}, {'id': 'bd78738b-2c39-4b85-9575-960d46ecd66c', 'keypoint': {'x': 325.0755, 'y': 500.4481}, 'name': 'orange'}, {'id': 'dafb3b80-4c1e-48a7-b398-e3aa56c5dcb0', 'keypoint': {'x': 374.1085, 'y': 515.2358}, 'name': 'orange'}, {'id': '36cf3f26-cac5-4368-91eb-ae8937ec2f91', 'keypoint': {'x': 412.2453, 'y': 427.2877}, 'name': 'orange'}, {'id': 'f93ff614-2250-4f4d-8ec0-ac87ec62305c', 'keypoint': {'x': 436.3726, 'y': 437.4057}, 'name': 'orange'}, {'id': '09066da9-88d0-4617-8cf6-7d604ee0190d', 'keypoint': {'x': 485.4057, 'y': 510.566}, 'name': 'orange'}]}
[/INST]
{'x': 366.3255, 'y': 546.3679}</s>


<s>[INST]Output only the keypoint location of the block corresponding to following instruction, nothing else. Instructions are from the perspective you looking forward at the table of blocks with their keypoints and colors given. Instruction:grab the green block that is right in front of you.
 {'width': 756, 'height': 660, 'annotations': [{'id': 'cad61a18-85b8-4f6e-93b3-c3ce9e28cda2', 'keypoint': {'x': 316.5142, 'y': 554.9292}, 'name': 'orange'}, {'id': '17682384-25d2-46d1-ae87-f04077458497', 'keypoint': {'x': 330.5236, 'y': 516.7925}, 'name': 'orange'}, {'id': '5d36ae41-004b-494e-b32d-33279b1dbfee', 'keypoint': {'x': 447.2689, 'y': 457.6415}, 'name': 'orange'}, {'id': 'fee58518-97d8-4c98-9b1b-ebeaa7ef3967', 'keypoint': {'x': 488.5189, 'y': 530.8019}, 'name': 'orange'}, {'id': '4fa5b6b4-f98b-4588-95e6-8255a4bef84b', 'keypoint': {'x': 380.3349, 'y': 446.7453}, 'name': 'yellow'}, {'id': '22f0cd3c-99be-40d9-9b3f-a7bc782da1f8', 'keypoint': {'x': 297.8349, 'y': 529.2453}, 'name': 'yellow'}, {'id': 'd5f859c2-a998-4ee4-bff1-d6025abd4883', 'keypoint': {'x': 375.6651, 'y': 536.25}, 'name': 'yellow'}, {'id': '96691580-8803-4b4b-b31f-4920d402e03e', 'keypoint': {'x': 473.7311, 'y': 435.0708}, 'name': 'yellow'}, {'id': '86653055-abe8-4a82-8b79-fb7f461438c0', 'keypoint': {'x': 474.5094, 'y': 487.217}, 'name': 'yellow'}, {'id': '34e55929-db77-4549-8f50-5adc4da9bd5d', 'keypoint': {'x': 370.9953, 'y': 492.6651}, 'name': 'blue'}, {'id': '500fbe66-22dd-4552-a43c-63bf2efeb1c9', 'keypoint': {'x': 417.6934, 'y': 477.0991}, 'name': 'blue'}, {'id': '85cb36b6-c443-488a-85b6-363a2284870a', 'keypoint': {'x': 345.3113, 'y': 424.9528}, 'name': 'green'}, {'id': '1c037117-1e04-4cd7-a544-c3e10b3639bb', 'keypoint': {'x': 295.5, 'y': 466.9811}, 'name': 'green'}, {'id': 'a8f28480-33ff-4a0a-883f-f1f226ced914', 'keypoint': {'x': 299.3915, 'y': 501.2264}, 'name': 'green'}, {'id': 'b9578593-4dcd-4808-9119-c6439b45d587', 'keypoint': {'x': 427.033, 'y': 534.6934}, 'name': 'green'}, {'id': '70024c2f-19b2-412a-a76d-d9e741635b7e', 'keypoint': {'x': 380.3349, 'y': 33.467}, 'name': 'you'}]}
[/INST]
{'x': 345.3113, 'y': 424.9528}</s>