[0.0][S01] It was a professor[1.458] [1.458][S01]that said, oh, I bet if we can get robots to tie shoelaces,[5.13] [5.13][S01]I will retire.[6.0] [6.0][S01]And the researchers in the team were like, right on.[9.96] [9.96][S01]I'm going to add that as a task.[11.58] [11.58][S02] Do you think that's what[12.14] [12.14][S02]we're about to see, then, in the same way as we've[13.67] [13.67][S02]seen the explosion of large language models?[15.75] [15.75][S02]Do you think the next thing is the explosion of robotics?[18.54] [18.54][S01] We used to have discussions about[20.623] [20.623][S01]whether it would happen in our lifetime or even in our careers,[24.3] [24.3][S01]and now we have debates about whether it[26.39] [26.39][S01]would be 5 or 10 years.[28.44] [28.44][MUSIC PLAYING][31.533] [31.533][S02] Welcome back to "Google DeepMind--[33.45] [33.45][S02]The Podcast."[34.14] [34.14][S02]I'm Professor Hannah Fry.[35.66] [35.66][S02]Sometimes, maybe even often, the terms AI and robot[40.01] [40.01][S02]are used interchangeably in casual conversation, people[43.58] [43.58][S02]talking about chatting to a robot on an app.[46.5] [46.5][S02]But robots have a physical body.[49.74] [49.74][S02]And here at Google DeepMind, they[51.62] [51.62][S02]care about robots with AI embedded in the real world.[56.37] [56.37][S02]And while AI has made huge strides,[59.28] [59.28][S02]embodied intelligence has lagged behind.[62.84] [62.84][S02]But perhaps all of that is about to change.[65.269] [65.269][S02]Carolina Parada leads robotics research[67.9] [67.9][S02]here at Google DeepMind, the international team[70.78] [70.78][S02]responsible for some extraordinary advances[73.51] [73.51][S02]in robotics, most recently Gemini Robotics, which[76.99] [76.99][S02]brings Gemini's multimodal understanding[79.33] [79.33][S02]to the physical world.[80.93] [80.93][S02]Welcome to the podcast, Carolina.[82.46] [82.46][S01] Thanks for having me.[83.6] [83.6][S02] Now, I know you've been[84.43] [84.43][S02]working with these robots for quite a long time.[87.05] [87.05][S02]How have you seen them evolve?[88.89] [88.89][S01] Yeah, it's been super exciting.[90.89] [90.89][S01]I have been excited about robotics[92.32] [92.32][S01]since I was 10 years old.[94.31] [94.31][S01]Super, super excited because of what I've seen in cartoons.[98.66] [98.66][S01]You see robots like Rosie the Robot helping do all the chores.[102.53] [102.53][S01]And as a kid you're like, of course,[104.03] [104.03][S01]that's what I want to build when I grow up.[105.822] [105.822][S01]And really, I've been at the Google DeepMind Robotics team[109.03] [109.03][S01]for about seven years, and, really,[111.02] [111.02][S01]things have changed dramatically in the last three years[113.83] [113.83][S01]in particular.[114.71] [114.71][S01]We've always believed from the very beginning[117.01] [117.01][S01]that AI was going to be completely[119.46] [119.46][S01]transformative to robotics.[120.585] [120.585][S01]I mean, there's a lot of robots out there[122.293] [122.293][S01]that are really helpful today.[123.64] [123.64][S01]There's robots in manufacturing lines.[125.65] [125.65][S01]There's robots that are navigating the moon.[128.02] [128.02][S01]There is robots that are in our oceans.[130.0] [130.0][S01]But these robots have been programmed[132.24] [132.24][S01]to do specifically those tasks.[134.08] [134.08][S01]They make a lot of assumptions about those environments[136.65] [136.65][S01]or the objects they might encounter,[139.01] [139.01][S01]or they might be remotely operated by humans.[141.355] [141.355][S01]But we have believed from the beginning[142.98] [142.98][S01]that AI is the way to transform robotics[145.83] [145.83][S01]so that we can build robots that are truly intelligent so[149.7] [149.7][S01]that they can interact with you, that they can reason[152.25] [152.25][S01]about their environment, and they[153.72] [153.72][S01]can take action in a way that feels very general.[156.187] [156.187][S01]So that has been our mission from the start.[158.02] [158.02][S01]And so I think three years ago, you[159.96] [159.96][S01]had Robotics in your podcast, and back[162.21] [162.21][S01]then, we were doing reinforcement[163.71] [163.71][S01]learning for robotics.[164.89] [164.89][S01]And so essentially, we were teaching robots[166.86] [166.86][S01]to stack blocks by giving them a simple reward, like a plus 1[171.55] [171.55][S01]if your tower got taller.[173.37] [173.37][S01]And we made some progress there.[175.69] [175.69][S01]But since we've been at the forefront of AI,[178.35] [178.35][S01]we've been bringing more and more of AI[181.16] [181.16][S01]into the entire world of robotics.[183.36] [183.36][S01]So about 2022, we introduced, for example, LLMs to robots.[187.885] [187.885][S01]And that was the first time that you could actually[190.01] [190.01][S01]talk to a robot and say something like, I'm thirsty,[193.25] [193.25][S01]and it would know what you meant.[194.94] [194.94][S01]And then later on, we brought VLMs[197.66] [197.66][S01]so the robot could understand natural language,[199.89] [199.89][S01]but it could also understand the visual input that it was getting[203.15] [203.15][S01]and then make decisions based on that.[205.28] [205.28][S01]And then in 2023, we introduced robotics transformers.[208.943] [208.943][S01]And this is the first time that the transformer architecture was[211.61] [211.61][S01]actually included in robotics.[213.45] [213.45][S01]And it basically showed us that robot performance scales[217.16] [217.16][S01]with data, and that essentially started[219.8] [219.8][S01]a new foundation or a new era of large-scale, data-driven robot[223.97] [223.97][S01]learning.[224.58] [224.58][S01]And then more recently, we introduced just now[226.52] [226.52][S01]Gemini Robotics, which is essentially[228.74] [228.74][S01]our most advanced model for actions.[231.21] [231.21][S01]And it essentially takes the multimodal world understanding[235.7] [235.7][S01]of Gemini and brings it to the physical world[238.34] [238.34][S01]by adding actions as a new modality in Gemini.[242.09] [242.09][S01]And that really enables models to be very general because it's[245.6] [245.6][S01]understanding the world through Gemini's understanding[248.48] [248.48][S01]and enables it to be interactive.[250.59] [250.59][S01]In fact, it can understand any language that Gemini supports[253.88] [253.88][S01]and enable it to be dexterous.[255.99] [255.99][S01]So it can still do very complex manipulation[258.95] [258.95][S01]while talking to you and also understanding[262.01] [262.01][S01]a completely new situation, which today is actually[264.89] [264.89][S01]very hard for robots to do.[266.78] [266.78][S02] In terms of your big goal, your big ambition,[269.82] [269.82][S02]how will we know when we get there?[271.65] [271.65][S01] I think it's definitely[273.317] [273.317][S01]going to be gradual, where robots are able to understand[276.89] [276.89][S01]a new situation and reason about something they need to do[280.34] [280.34][S01]that they haven't seen before.[282.0] [282.0][S01]And that's exactly what we're seeing right now.[284.82] [284.82][S01]But it's still going to be difficult[287.36] [287.36][S01]for them to learn more and more complex tasks.[290.18] [290.18][S01]In fact, that's what we see.[291.48] [291.48][S01]The robot can feel sort of a two-year-old toddler that can[295.7] [295.7][S01]understand its world around it.[297.39] [297.39][S01]It can start to play with objects.[299.37] [299.37][S01]It understands concepts.[301.05] [301.05][S01]But if you teach it to do something more complex,[303.68] [303.68][S01]like we have an example where we're teaching the robot[305.93] [305.93][S01]to do an origami fold, it actually[308.48] [308.48][S01]needs time to practice that.[310.29] [310.29][S01]And once it has more practice, in that case,[312.69] [312.69][S01]it can actually do it.[313.74] [313.74][S01]So that's roughly where we are today,[315.54] [315.54][S01]but that's far from where we need[317.18] [317.18][S01]to be if we want robots to be in everyday spaces[320.27] [320.27][S01]doing all kinds of tasks for us.[321.96] [321.96][S01]So there's still quite a bit to go.[323.73] [323.73][S02] I thought that what we could do[325.522] [325.522][S02]is take a little look at some of what these robots can[328.43] [328.43][S02]do because there's a video that you guys have recently released.[331.74] [331.74][S02]What we have here, then, is we have a humanoid robot who[335.63] [335.63][S02]is packing a lunch for its human, also playing noughts[340.57] [340.57][S02]and crosses.[341.07] [341.07][S02]Is it any good at the noughts and crosses?[342.58] [342.58][S01] [LAUGHS] I think[343.955] [343.955][S01]we still beat it because it's very simple understanding.[346.365] [346.365][S02] Tell you what it's doing, though,[348.24] [348.24][S02]is picking up the pieces and moving them around quite easily.[351.71] [351.71][S02]There's also a bit here where it can make its own anagram based[356.03] [356.03][S02]on tiles that appear.[357.693] [357.693][S02]What were you particularly impressed by?[359.36] [359.36][S01] I think that's what's[359.95] [359.95][S01]most exciting about these models,[361.78] [361.78][S01]is that, in many occasions, our own researchers were[364.51] [364.51][S01]excited and impressed by what it was doing.[366.65] [366.65][S01]And it was primarily because the way[369.55] [369.55][S01]we were testing it was by putting the robot in front[372.55] [372.55][S01]of situations that it's never seen before.[374.51] [374.51][S01]So even us didn't know whether the robot was going[376.9] [376.9][S01]to be able to get it right.[378.2] [378.2][S01]And in many occasions, it did.[380.41] [380.41][S01]So many of the examples that we show in this video,[383.51] [383.51][S01]as well as the other videos where[385.51] [385.51][S01]you have the two arms moving around,[388.63] [388.63][S01]is that it's actually understanding a complex concept.[391.64] [391.64][S01]So a really cool example that where we were all like, gasp,[395.71] [395.71][S01]was when we showed the video where the robot is actually[399.25] [399.25][S01]doing a slam dunk.[400.73] [400.73][S01]And what was cool about that case is that that day,[403.24] [403.24][S01]we were just having the creative team come and film the robots,[406.58] [406.58][S01]and we asked them to bring toys.[408.32] [408.32][S01]We didn't say anything else.[409.88] [409.88][S01]They're like, just bring toys to play with the robot.[412.15] [412.15][S02] All things the robot hadn't seen before.[413.73] [413.73][S01] All things-- yeah.[414.76] [414.76][S01]They had no idea what the robot was trained on.[416.718] [416.718][S01]So they actually brought this little basketball hoop[419.19] [419.19][S01]that was a little cute toy with a little ball,[421.99] [421.99][S01]and they put it in front of the robot.[423.88] [423.88][S01]Again, the robot had never seen anything related to basketball.[426.91] [426.91][S01]It certainly had never seen this toy.[428.74] [428.74][S01]And they asked it to do a slam dunk of the ball.[431.83] [431.83][S01]And we were all like, I have no idea if it will work.[434.53] [434.53][S01]And actually, it took not even a quarter of a second[438.18] [438.18][S01]and it actually decided to put the ball inside the basketball[441.87] [441.87][S01]hoop.[442.45] [442.45][S01]And we were all like, that's amazing.[444.49] [444.49][S01]And it was just essentially drawing[446.28] [446.28][S01]from Gemini's understanding of what basketball is[449.01] [449.01][S01]and what a slam dunk is, which is a concept[452.07] [452.07][S01]that we couldn't have thought of teaching it to do.[455.73] [455.73][S01]And it essentially did the right motion.[458.71] [458.71][S01]So that was a really cool example.[460.275] [460.275][S02] Talk to me about the packing lunch one.[462.4] [462.4][S02]It has a conceptual understanding[463.86] [463.86][S02]of what a banana is, for example.[465.61] [465.61][S02]Does it know how to grip a banana in the sense[468.017] [468.017][S02]that you can't grip a banana in quite the same way[470.1] [470.1][S02]as you could a clay pot or something even more fragile[472.96] [472.96][S02]than a banana?[473.543] [473.543][S01] Actually, one of the things that[475.585] [475.585][S01]is super impressive is that these robots[477.76] [477.76][S01]are extremely simple.[478.97] [478.97][S01]They actually don't have touch sensing.[481.1] [481.1][S01]They don't have depth sensing.[482.66] [482.66][S01]They don't have force sensing.[483.98] [483.98][S01]So they're literally doing eye-hand coordination[486.64] [486.64][S01]and using an understanding of how you grasp a banana.[489.62] [489.62][S01]So it actually is looking at the object and grasping it.[492.71] [492.71][S01]And once it sees that it has it in hand,[494.78] [494.78][S01]that's how it knows that it has detected it.[498.32] [498.32][S01]There's other robots out there that are much more complex.[500.78] [500.78][S01]But this forces the model to really reason about[503.98] [503.98][S01]what is seen and making a decision about how[506.29] [506.29][S01]to pick that up.[507.11] [507.11][S02] And that's the thing that's really original here.[509.36] [509.36][S01] Yeah, that is one of the many things,[511.03] [511.03][S01]is the fact that it's doing it not just because we taught it[513.94] [513.94][S01]1,000 times how to pick up a banana.[515.9] [515.9][S01]It's because it's pulling this out of it's understanding of how[519.52] [519.52][S01]to pick up objects from Gemini and then adapting it[523.96] [523.96][S01]to the world of actions.[524.987] [524.987][S02] I mean, there have been lots of videos doing[527.32] [527.32][S02]the rounds on the internet for a number of years of extremely[529.87] [529.87][S02]impressive-looking robots doing backflips and--[532.17] [532.17][S02]I don't know-- being kicked over and running up and down[535.23] [535.23][S02]mountains and things.[537.255] [537.255][S02]In comparison to those videos, picking up[540.3] [540.3][S02]and putting down a banana into a lunchbox[542.835] [542.835][S02]seems like quite a simple task.[545.71] [545.71][S02]But we're talking about a different type of robot here,[548.86] [548.86][S02]aren't we?[549.46] [549.46][S01] Yeah.[550.02] [550.02][S01]I mean, this is a completely different problem[551.937] [551.937][S01]you're trying to solve.[553.0] [553.0][S01]Many of those videos are basically rehearsed sequences[557.25] [557.25][S01]that the robot has learned and memorized,[559.18] [559.18][S01]and we're actually very impressed by them.[562.025] [562.025][S01]But it's a different problem that you're trying to solve.[564.4] [564.4][S01]What you're trying to solve here is for the robot[566.442] [566.442][S01]to reason about what it means to pack a lunch, given[569.687] [569.687][S01]the objects in front, what it needs to do in order[571.77] [571.77][S01]to put a piece of bread inside of a bag,[574.75] [574.75][S01]and then what it means to close it.[576.43] [576.43][S01]And it's never going to go as you[577.98] [577.98][S01]expect because these are very flexible things that[580.56] [580.56][S01]move around.[581.35] [581.35][S01]So it needs to react and respond to what's happening[583.74] [583.74][S01]and then actually complete the task.[585.672] [585.672][S02] It's that idea of generality.[587.38] [587.38][S01] That's right.[587.98] [587.98][S01]Yeah.[588.48] [588.48][S02] How do you decide--[590.3] [590.3][S02]or how do you compare one robot against another?[593.34] [593.34][S02]How do you decide whether this robot is doing[595.43] [595.43][S02]generally better than another?[597.373] [597.373][S01] That was actually[598.79] [598.79][S01]one of the things that was hard for us[601.07] [601.07][S01]to express when we were even recording[603.68] [603.68][S01]for the demos in this release.[606.06] [606.06][S01]A demo is, by definition, prescripted.[609.782] [609.782][S01]So we were like, this doesn't quite[611.24] [611.24][S01]capture what we want to share.[613.2] [613.2][S01]So what we did-- that's why we asked the team[615.23] [615.23][S01]to bring a bunch of toys and actually[617.09] [617.09][S01]start playing with the robots and see what emerges.[619.95] [619.95][S01]And the best way to capture it is[621.92] [621.92][S01]that we're able to change the behavior of the robot[625.22] [625.22][S01]by talking to it, and you can see that in the videos.[627.81] [627.81][S01]We're actually able to put raw objects[629.9] [629.9][S01]that it's never seen before.[631.53] [631.53][S01]And we move objects around to make sure[633.92] [633.92][S01]that people understand that this is actually[635.78] [635.78][S01]not a prescripted behavior.[638.34] [638.34][S01]In fact, in our benchmarks, we evaluate our models[641.84] [641.84][S01]in all kinds of ways in terms of generalization.[644.85] [644.85][S01]So we will change the visual background.[647.1] [647.1][S01]We will change the background.[648.36] [648.36][S01]The objects were new.[649.53] [649.53][S01]We will add objects to distract the robot.[652.97] [652.97][S01]We would also ask it to do completely new things,[655.71] [655.71][S01]or even you can talk to it in a different language.[657.84] [657.84][S01]So I could just give it the instruction in Spanish,[660.08] [660.08][S01]and it would just actually work.[662.243] [662.243][S02] I want to talk about interactivity, too.[664.41] [664.41][S02]Because in a few of your videos, there's[666.077] [666.077][S02]one where a human is sat at a desk,[668.66] [668.66][S02]and the robot is clearing up after him as he goes.[671.6] [671.6][S02]In another, you've got a human moving a cup around,[674.19] [674.19][S02]and the robot is chasing it, trying to put an object inside.[677.04] [677.04][S02]How much more difficult are those interactive scenarios[680.87] [680.87][S02]than just a static task?[682.183] [682.183][S01] They're significantly more advanced[684.35] [684.35][S01]behavior, and a lot of the interactivity[686.99] [686.99][S01]just fell out of the model.[688.83] [688.83][S01]We were not thinking, for example, how fast[692.09] [692.09][S01]can we move these objects before the robot would react.[695.88] [695.88][S01]We certainly knew that we wanted a model that[698.06] [698.06][S01]could react quickly.[699.54] [699.54][S01]But a lot of these examples that we posted in videos[702.14] [702.14][S01]just fell out of people playing with the model[704.24] [704.24][S01]and seeing how it would behave.[706.52] [706.52][S01]Same with organizing the desk.[708.798] [708.798][S01]That was actually someone playing with the robot,[710.84] [710.84][S01]deciding to see how much he could game it[714.52] [714.52][S01]until it actually was able to complete the full task.[718.22] [718.22][S01]So yeah, it actually is amazing to see[721.63] [721.63][S01]how a lot of these other capabilities that are already[724.18] [724.18][S01]there in Gemini are actually extremely valuable when[726.79] [726.79][S01]you bring them into a robot, which is now able to adapt based[731.26] [731.26][S01]on what you're saying.[732.283] [732.283][S01]So you could actually have a full conversation[734.2] [734.2][S01]and change the behavior of the robot as it's moving.[736.377] [736.377][S01]So you can say, I want you to do this.[737.96] [737.96][S01]Oh, no, actually, never mind.[739.07] [739.07][S01]I want you to do this other thing.[740.54] [740.54][S01]And it would actually just follow you.[743.09] [743.09][S01]It's actually kind of comical.[744.83] [744.83][S01]And then you could also change the objects around,[749.26] [749.26][S01]and it will just do it.[750.5] [750.5][S02] I think it's kind of a good job[751.87] [751.87][S02]sometimes that these robots don't have feelings[753.828] [753.828][S02]because it'd feel very forlorn, just being chased around[759.4] [759.4][S02]on a table by researchers.[760.867] [760.867][S01] It's super fun.[762.2] [762.2][S02] That's the large language model[764.41] [764.41][S02]sitting underneath it that's helping it do that, that's[766.81] [766.81][S02]giving it that conceptual understanding of the objects[768.7] [768.7][S02]that it's manipulating.[769.658] [769.658][S01] We're leveraging[771.033] [771.033][S01]Gemini's multimodal understanding[772.42] [772.42][S01]to take the visual input of the role[774.7] [774.7][S01]that the robot is seeing through its cameras[776.72] [776.72][S01]and the natural language that is hearing from the human[779.57] [779.57][S01]and then translate that into how to act.[783.1] [783.1][S01]And it actually also speaks back.[784.91] [784.91][S01]So you can ask it a question about whether it's done.[787.25] [787.25][S01]You can ask it a question about how far it[789.01] [789.01][S01]is in the process of folding an origami figure.[791.81] [791.81][S01]It actually understands that.[793.03] [793.03][S01]It can respond.[793.93] [793.93][S02] I remember when Gemini was first being launched,[796.43] [796.43][S02]and people went to great lengths to talk[799.87] [799.87][S02]about how it was multimodal.[802.54] [802.54][S02]Is this one of the main reasons?[804.71] [804.71][S02]Is this the payoff for putting in all of that extra groundwork[808.75] [808.75][S02]and making sure that it can understand videos and photos[812.44] [812.44][S02]and so on?[812.978] [812.978][S01] I mean, one of many.[814.52] [814.52][S01]I think us humans capture the world[817.75] [817.75][S01]through many different senses.[819.46] [819.46][S01]So I think it's super important, if you[821.2] [821.2][S01]want to build an intelligence as powerful as our brains,[824.68] [824.68][S01]to be able to take input in a multimodal way.[827.79] [827.79][S01]And definitely robotics is a perfect example[830.57] [830.57][S01]where you can see that it absolutely[832.19] [832.19][S01]requires to have understanding of natural language[834.77] [834.77][S01]and visual input and, presumably in the future,[837.51] [837.51][S01]also touch sensing in order to make decisions about how to act,[841.04] [841.04][S01]the same way humans do.[842.48] [842.48][S02] Why does it matter, though,[844.11] [844.11][S02]that robots should have a conceptual understanding of what[847.85] [847.85][S02]they're doing?[848.58] [848.58][S02]I mean, OK, maybe you wouldn't call them intelligent,[851.37] [851.37][S02]but there are robots like--[852.57] [852.57][S02]I don't know-- dishwashers or lawnmowers.[856.288] [856.288][S02]They don't have a conceptual understanding of what a plate is[858.83] [858.83][S02]or what grass is.[860.73] [860.73][S02]Is it actually necessary?[862.912] [862.912][S01] I'm sure that there are applications where[865.37] [865.37][S01]you can have a robot that can just repeat the actions,[868.26] [868.26][S01]and it will be just fine.[869.46] [869.46][S01]But we're interested in actually building robots that can really[874.31] [874.31][S01]reason and act in a very general way,[876.89] [876.89][S01]just because the world is really messy.[879.12] [879.12][S01]Things will never go exactly according to plan,[881.46] [881.46][S01]and there's a lot of tasks where things are constantly changing.[885.26] [885.26][S01]And it actually just opens up the opportunity of applications[888.67] [888.67][S01]for these robots.[889.61] [889.61][S01]They could literally be anywhere that a human[892.24] [892.24][S01]could be doing a task.[894.2] [894.2][S01]So that enables them to be helpful in home environments[897.38] [897.38][S01]but also in manufacturing environments.[899.835] [899.835][S02] There are some things[901.21] [901.21][S02]that are important in robotics that,[903.97] [903.97][S02]I think, now with the standard Gemini, actually are quite easy,[909.68] [909.68][S02]things like pointing or drawing bounding boxes.[913.01] [913.01][S02]Just explain to us what those are.[915.11] [915.11][S01] Well, basically this is one of the areas[917.56] [917.56][S01]that we actually had to improve Gemini in order[921.37] [921.37][S01]to help with robotics.[923.66] [923.66][S01]If you have, for example, an object in front of you,[927.14] [927.14][S01]what we mean by pointing is that I can literally identify[931.39] [931.39][S01]any point in that object.[933.38] [933.38][S01]So I can say, imagine that you have a T-shirt in front of you.[937.22] [937.22][S01]If I point to the collar, it should say, this is the collar.[940.13] [940.13][S01]Or if I say collar, it should identify where the collar is.[943.34] [943.34][S01]And you might imagine that this is not that important.[946.03] [946.03][S01]But actually, if you're trying to fold that T-shirt,[948.73] [948.73][S01]you need to know where the collar is,[950.62] [950.62][S01]where the bottom of the T-shirt is, and all[953.61] [953.61][S01]of the different components.[955.05] [955.05][S01]Bounding boxes, what it means is that you[957.57] [957.57][S01]can identify all the edges of that[959.67] [959.67][S01]object so that you know where the object ends[962.58] [962.58][S01]and the rest of the environment begins.[964.6] [964.6][S01]So these kinds of examples are, I think, trivial for us humans.[969.25] [969.25][S01]We don't even think about it.[970.6] [970.6][S01]But if robots are actually able to have access[973.17] [973.17][S01]to that kind of information, then they[974.85] [974.85][S01]can be smarter about the way that they take[978.63] [978.63][S01]action in the physical world.[979.93] [979.93][S01]This is what we call embodied reasoning, essentially.[981.85] [981.85][S02] How is it different from the kind of reasoning[984.267] [984.267][S02]that you get in the standard Gemini model?[986.35] [986.35][S01] We refer to embodied reasoning[988.308] [988.308][S01]as reasoning about the physical world in a lot more detail,[991.21] [991.21][S01]the way humans do.[992.29] [992.29][S01]If you are going to take action--[993.85] [993.85][S01]say that you're trying to pack a lunch for your kid--[996.622] [996.622][S01]in order to do that, you have to understand where all the objects[999.33] [999.33][S01]are in 3D space.[1000.81] [1000.81][S01]Then you need to understand how to grasp each object in order[1005.39] [1005.39][S01]to pack it into that box.[1007.05] [1007.05][S01]And then you need to figure out how to organize all those pieces[1011.49] [1011.49][S01]so that they fit.[1012.54] [1012.54][S01]All of this is what we mean by embodied reasoning.[1015.99] [1015.99][S02] So is this things like--[1017.6] [1017.6][S02]I don't know.[1018.142] [1018.142][S02]Let's say you got two-camera view.[1020.09] [1020.09][S02]You're there, and I'm here, for instance,[1022.35] [1022.35][S02]I can see your microphone, and so can you.[1024.24] [1024.24][S02]But we've got a completely different view of it.[1026.24] [1026.24][S02]Is it that kind of stuff?[1027.363] [1027.363][S01] Yeah.[1028.28] [1028.28][S01]It can understand, for example, how far the microphone[1030.53] [1030.53][S01]is from our face.[1031.349] [1031.349][S01]But also, if I move around, it can do object correspondence,[1034.98] [1034.98][S01]meaning it understands that microphone[1036.65] [1036.65][S01]is the same one that I'm seeing from the other point of view,[1039.192] [1039.192][S01]which you can imagine is super important[1040.88] [1040.88][S01]if a robot is moving and reasoning about its environment.[1044.55] [1044.55][S02] How hard is it to switch from a 2D image,[1048.39] [1048.39][S02]a single-camera view, to a 3D understanding of the space?[1052.8] [1052.8][S01] So actually, today, what robots are doing[1055.4] [1055.4][S01]is that they're taking camera views from different places.[1059.07] [1059.07][S01]So actually, the robot has cameras in its wrist,[1061.16] [1061.16][S01]and it has a camera on top, and it is actually[1064.31] [1064.31][S01]taking all of the inputs from the three images[1066.87] [1066.87][S01]and doing this on its own.[1068.7] [1068.7][S01]It's actually reasoning, oh, I'm closer to the object[1071.69] [1071.69][S01]because now this camera looks closer.[1074.19] [1074.19][S01]This camera, I can see my hand.[1076.07] [1076.07][S01]And it's doing all of that association on its own.[1078.33] [1078.33][S01]So we're not explicitly adding depth as an additional input.[1083.82] [1083.82][S01]We're just giving it multiple camera views,[1086.37] [1086.37][S01]and it's realizing how to use them[1088.52] [1088.52][S01]in order to understand depth.[1090.0] [1090.0][S02] And how much of that was you deliberately setting[1094.31] [1094.31][S02]that as a task for the robots?[1096.0] [1096.0][S02]Or how much of it emerged from the conceptual understanding[1099.47] [1099.47][S02]that you get from the Gemini models?[1101.385] [1101.385][S01] It simply emerged, actually.[1103.26] [1103.26][S01]So we were able to give it multiple cameras[1106.58] [1106.58][S01]and just see if it actually could reason between them.[1108.987] [1108.987][S02] That's got to be quite shocking.[1110.82] [1110.82][S02]Many, many people must have spent many, many years thinking[1113.81] [1113.81][S02]very hard about that problem of how[1116.51] [1116.51][S02]do you align different camera views to make it[1119.45] [1119.45][S02]so that you're tracking an object across different angles.[1123.34] [1123.34][S02]And then all of a sudden you get these large language models,[1127.62] [1127.62][S02]like Gemini, and it can just do it automatically.[1130.38] [1130.38][S01] Yeah, it's actually[1131.88] [1131.88][S01]wonderful to be able to leverage these models to bring simplicity[1137.04] [1137.04][S01]to the system.[1137.86] [1137.86][S01]You really don't need to have all of these different stages[1140.64] [1140.64][S01]that you extract depth.[1142.18] [1142.18][S01]Then only then you extract where the objects are,[1144.67] [1144.67][S01]and only then you plan how to move,[1146.44] [1146.44][S01]and then only then you're able to do the task.[1148.54] [1148.54][S02] And that's because the foundational model is--[1150.09] [1150.09][S02]I mean, it's effectively like a Swiss army knife.[1151.69] [1151.69][S02]It can do all of the things simultaneously.[1153.305] [1153.305][S01] Yes, exactly.[1154.14] [1154.14][S01]And it can reason between them.[1155.62] [1155.62][S02] OK, so you enhance the physical reasoning, almost.[1159.348] [1159.348][S01] Exactly.[1160.39] [1160.39][S01]You enhance physical reasoning and spatial understanding.[1163.81] [1163.81][S01]And then motion understanding would be the next thing,[1167.64] [1167.64][S01]is understanding, what would happen[1169.44] [1169.44][S01]if I put a glass at the edge of the table?[1172.15] [1172.15][S01]What's actually likely to happen?[1174.22] [1174.22][S01]All of these areas is the areas that we enhance.[1177.01] [1177.01][S01]But that's not enough.[1178.12] [1178.12][S01]You actually have to take it another step[1180.2] [1180.2][S01]and essentially start to teach Gemini the language of actions.[1185.21] [1185.21][S01]And actions for us means understanding[1188.09] [1188.09][S01]how you are actually moving each joint in a robot.[1190.62] [1190.62][S01]So if this is my robot arm, then I'm[1192.35] [1192.35][S01]teaching Gemini how to move the robot, how[1195.08] [1195.08][S01]to move my arm like this.[1196.71] [1196.71][S01]And these are all essentially numbers.[1199.22] [1199.22][S01]And it's learning to translate what[1201.17] [1201.17][S01]it means to pick up a glass versus move my arm[1203.81] [1203.81][S01]in order to pick up a glass.[1205.17] [1205.17][S01]And so you're essentially teaching it a new language.[1207.383] [1207.383][S02] You're connecting those different ideas.[1209.55] [1209.55][S01] Exactly.[1210.02] [1210.02][S02] Can we think of this as two systems working[1212.312] [1212.312][S02]in tandem then?[1213.57] [1213.57][S02]I'm thinking of the analogy here of system 1 and system 2,[1216.96] [1216.96][S02]the Danny Kahneman "Thinking Fast and Slow" thing.[1219.31] [1219.31][S01] Yes, exactly.[1220.56] [1220.56][S01]So essentially, the model that we built actually[1222.56] [1222.56][S01]has two models.[1223.5] [1223.5][S01]It has a system that is slow but very powerful at reasoning[1227.57] [1227.57][S01]and thinking and a system that is faster[1229.91] [1229.91][S01]but very, very good at reactivity.[1232.77] [1232.77][S01]So this is the concept of slow and fast thinking.[1236.4] [1236.4][S02] This is like how human brains work,[1238.42] [1238.42][S02]that you have the part of your brain[1240.58] [1240.58][S02]that's very good at calculation and analysis,[1243.62] [1243.62][S02]and then you also have your very instinctive, reactive side too.[1246.892] [1246.892][S01] Yes, that's right.[1248.35] [1248.35][S01]In fact, what we do today is that one of these models[1251.56] [1251.56][S01]is much bigger than the other, as you can imagine,[1253.97] [1253.97][S01]and it actually lives on a server.[1255.49] [1255.49][S01]And the fast model lives on device,[1257.98] [1257.98][S01]and it can respond very quickly.[1259.58] [1259.58][S02] Talk me through how this works then.[1261.58] [1261.58][S02]In terms of the system 1 and system 2 and that example[1264.91] [1264.91][S02]of a slam dunk, something it's never seen before,[1267.68] [1267.68][S02]how does it work?[1268.85] [1268.85][S01] What happens is when[1270.392] [1270.392][S01]you ask the robot to take the basketball and do a slam dunk,[1273.8] [1273.8][S01]the system 2 has to understand what that means.[1276.55] [1276.55][S01]What is basketball?[1277.61] [1277.61][S01]It has to understand where the objects that are in front of it[1280.42] [1280.42][S01]are, where the basketball is, understand there is a hoop,[1283.79] [1283.79][S01]and then that a slam dunk actually means picking up[1286.66] [1286.66][S01]that ball and putting it there.[1288.14] [1288.14][S01]So it understands all of that and predicts a rough trajectory[1292.03] [1292.03][S01]of what the robot should do in terms[1293.86] [1293.86][S01]of how it should move and then hands that over to the system[1298.09] [1298.09][S01]1, which is on device and is able to take that trajectory,[1302.23] [1302.23][S01]but it also takes the visual input[1304.54] [1304.54][S01]and is able to adjust that trajectory.[1306.73] [1306.73][S01]So if I were to, for example, get in the way,[1308.74] [1308.74][S01]put my hand in the middle, or move the object around,[1311.68] [1311.68][S01]it would still be able to respond because it already[1314.13] [1314.13][S01]understood the concept of what a slam dunk was[1317.1] [1317.1][S01]and respond very quickly.[1318.238] [1318.238][S02] Why do you need two systems at all, though?[1320.53] [1320.53][S02]Why can't you just use the slow, clever one?[1322.613] [1322.613][S01] We could actually[1324.03] [1324.03][S01]just use the slow, clever one, but then it would actually[1326.82] [1326.82][S01]be significantly visually slower,[1330.0] [1330.0][S01]and it won't adapt as quickly to changes in its environment.[1332.95] [1332.95][S01]And that's important, especially if you're doing something where[1336.39] [1336.39][S01]the objects will move around.[1338.02] [1338.02][S01]So if you have, for example--[1339.25] [1339.25][S01]imagine when you're folding a T-shirt in the air, which[1342.06] [1342.06][S01]us humans do pretty cleverly, you're[1344.083] [1344.083][S01]actually moving this T-shirt, and things[1345.75] [1345.75][S01]are moving for you in ways that you don't predict.[1348.707] [1348.707][S01]So you need to be able to respond quickly in order[1350.79] [1350.79][S01]to actually complete the task.[1352.48] [1352.48][S01]So you need definitely a fast system.[1354.9] [1354.9][S01]And the slow system simply enables[1357.35] [1357.35][S01]us to do much more complex reasoning.[1359.55] [1359.55][S01]So you could also just live with a small system[1362.55] [1362.55][S01]if you could do tasks that don't require advanced reasoning.[1365.265] [1365.265][S02] Was it a direct copy of how[1366.89] [1366.89][S02]things work in the human brain?[1368.57] [1368.57][S02]The Daniel Kahneman work comes back to 1970s or so, that[1372.5] [1372.5][S02]we've understood that that's how the human brain works.[1374.85] [1374.85][S02]Was it a direct--[1375.812] [1375.812][S01] [LAUGHS] No, not at all.[1377.52] [1377.52][S01]I think we started definitely with the slow system,[1379.83] [1379.83][S01]as you said.[1380.43] [1380.43][S01]Why don't we just solve this with one model?[1382.38] [1382.38][S01]And we found that, actually, if you[1383.9] [1383.9][S01]want to do highly dexterous behaviors[1387.11] [1387.11][S01]or any kind of complex manipulation,[1389.16] [1389.16][S01]you need to respond quickly.[1390.57] [1390.57][S01]And that was the best combination that we could find.[1393.643] [1393.643][S02] Wow.[1394.31] [1394.31][S02]It's almost like evolution is a really good optimization[1397.176] [1397.176][S02][INAUDIBLE], finds really good strategies[1400.1] [1400.1][S02]for quick but clever things.[1401.765] [1401.765][S01] Yes, definitely.[1403.14] [1403.14][S01]It was surprising to us that that combination just worked.[1405.63] [1405.63][S02] I do think that there's sometimes[1407.505] [1407.505][S02]where the human body knows stuff before your brain does,[1413.07] [1413.07][S02]as it were.[1414.273] [1414.273][S02]You can catch a falling glass without thinking.[1417.17] [1417.17][S02]Or you can commit things to muscle memory,[1419.098] [1419.098][S02]like playing a piano, where you can actually just[1421.14] [1421.14][S02]be thinking about completely different things.[1423.84] [1423.84][S02]Are you seeing similar things with the robots,[1426.51] [1426.51][S02]that they almost have a physical intelligence that's[1428.82] [1428.82][S02]separate from the slow, clever system?[1432.108] [1432.108][S01] So we definitely see[1433.65] [1433.65][S01]that if you take the model that can reason[1436.65] [1436.65][S01]and you give it a lot of examples of a particular task,[1439.96] [1439.96][S01]it will get really, really good at that task.[1442.03] [1442.03][S01]But at the moment, if you do too much of that,[1444.67] [1444.67][S01]it will start forgetting some of the generalization.[1446.895] [1446.895][S02] Oh.[1447.52] [1447.52][S01] So this is an active area of research,[1449.812] [1449.812][S01]is, how do we enable the robot to get really, really[1452.7] [1452.7][S01]good at a task, like a really extremely difficult one,[1456.18] [1456.18][S01]and then not lose any of the generalization?[1459.07] [1459.07][S01]So it actually right now is a balancing act.[1461.982] [1461.982][S02] In some ways, that does happen with humans too.[1464.44] [1464.44][S02]I know some people who are really, really,[1466.47] [1466.47][S02]really, really, really good at maths[1467.97] [1467.97][S02]and terrible at tying their own shoelaces.[1469.736] [1469.736][LAUGHTER][1471.408] [1471.408][S02]I think that does happen.[1472.45] [1472.45][S02]They forget.[1472.95] [1472.95][LAUGHTER][1474.46] [1474.46][S02]OK, then.[1475.404] [1475.404][S02]So if this is what's going on behind the scenes--[1478.13] [1478.13][S02]so if we've got system 1 and system 2, as you described it,[1483.01] [1483.01][S02]it's also definitely true that these robots[1485.95] [1485.95][S02]have these very impressive new abilities and capabilities,[1489.91] [1489.91][S02]which are very different from where we were before.[1492.98] [1492.98][S02]Last time I visited DeepMind's robotics lab,[1495.437] [1495.437][S02]I think it's fair to say that the robots' movements were[1497.77] [1497.77][S02]a bit clumsy.[1499.0] [1499.0][S02]I think that's probably the kindest way to say it.[1501.627] [1501.627][S02]Let me just play you a little clip.[1503.085] [1503.085][AUDIO PLAYBACK][1503.752] [1503.752][S02]- So there's only one way around that it can hold this red object[1507.76] [1507.76][S02]and successfully pick it up.[1509.48] [1509.48][S02]And it hasn't worked out which way.[1511.86] [1511.86][S02]And unfortunately, every time it tries to rotate and pick it[1514.36] [1514.36][S02]up-- oh, hang on.[1515.09] [1515.09][S02]I think it's got it.[1515.923] [1515.923][S02]It's got it.[1516.638] [1516.638][S02]It's a good job these things don't get disheartened.[1518.805] [1518.805][END PLAYBACK][1519.29] [1519.29][S02] The thing is that I've been in that lab maybe five[1521.873] [1521.873][S02]years earlier, and these poor robots were still[1524.23] [1524.23][S02]there five years later trying to do the same minimally dexterous[1528.37] [1528.37][S02]tasks.[1529.19] [1529.19][S02]What changed?[1530.51] [1530.51][S02]Because I understand how having Gemini, the slow, clever system,[1535.53] [1535.53][S02]could improve the conceptual understanding of things.[1537.76] [1537.76][S02]But that doesn't change the dexterity.[1539.65] [1539.65][S02]It doesn't change how easily it can manipulate these objects,[1543.09] [1543.09][S02]does it?[1543.61] [1543.61][S01] Yeah, last year, we[1545.11] [1545.11][S01]spent basically all of our effort on tackling dexterity.[1549.07] [1549.07][S01]And this is still an area of active research.[1550.96] [1550.96][S01]But there's a couple of things that changed.[1552.85] [1552.85][S01]One is that we realized that if we can enable humans to show[1557.97] [1557.97][S01]the robot how to do very complex behaviors through teleoperation[1562.29] [1562.29][S01]or puppeteering, what this means is[1564.09] [1564.09][S01]that you give the human an extra pair of arms, robot arms,[1567.04] [1567.04][S01]and they can actually pretend to be the robot[1569.31] [1569.31][S01]and show the robot how to do the task.[1572.02] [1572.02][S01]And if that becomes really intuitive,[1574.3] [1574.3][S01]then you can capture a lot of data[1575.76] [1575.76][S01]of the robot doing the task, being teleoperated by a human,[1579.4] [1579.4][S01]but it's robot data.[1580.387] [1580.387][S02] So let me understand then.[1581.97] [1581.97][S02]So the human is wearing maybe a head cam.[1584.005] [1584.005][S01] Yes.[1584.88] [1584.88][S02] And it's quite literally pretending[1585.93] [1585.93][S02]to be the robot then.[1586.805] [1586.805][S02]So it's operating the robot's hands with its hands,[1589.092] [1589.092][S02]wearing the head cam, watching what[1590.55] [1590.55][S02]the robot would be watching, but doing the task[1593.43] [1593.43][S02]as it wants the robot to do.[1595.055] [1595.055][S01] So there's different teleoperation[1597.18] [1597.18][S01]examples.[1598.09] [1598.09][S01]One is where you actually sit in front of the robot,[1600.61] [1600.61][S01]so you have direct visibility to what the robot is doing,[1603.13] [1603.13][S01]and you move the robot arms.[1604.84] [1604.84][S01]Literally, you're puppeteering the robot.[1606.73] [1606.73][S01]And there's other examples where you put a VR set and gloves,[1610.23] [1610.23][S01]and actually, you pretend to be the robot,[1612.03] [1612.03][S01]and you move the stuff.[1613.06] [1613.06][S01]And that required a second component,[1614.86] [1614.86][S01]which was diffusion models.[1616.193] [1616.193][S01]And these are the same models that[1617.61] [1617.61][S01]actually get used, for example, by Imagen[1620.22] [1620.22][S01]in order to generate videos.[1622.06] [1622.06][S01]And essentially, what it's doing is[1623.55] [1623.55][S01]extracting from a lot of data a lot[1625.95] [1625.95][S01]of examples of doing that task and predicting the actual action[1629.76] [1629.76][S01]trajectory that it needs to do in order to do that task.[1632.68] [1632.68][S01]So when you combine those two with a clever transformer[1636.63] [1636.63][S01]architecture and a good data set,[1639.55] [1639.55][S01]you can actually learn anything.[1641.14] [1641.14][S01]And that was actually really, again, surprising[1643.86] [1643.86][S01]to the researchers.[1644.83] [1644.83][S01]That's when, for example, we discovered[1646.47] [1646.47][S01]that we could tie shoelaces.[1648.19] [1648.19][S01]We could fold laundry.[1649.75] [1649.75][S01]We could do origami.[1651.38] [1651.38][S01]And so what we did in this work is[1654.16] [1654.16][S01]that we combined the powerful reasoning module from Gemini[1657.88] [1657.88][S01]with what we had learned around being[1660.58] [1660.58][S01]able to do dexterous tasks.[1662.63] [1662.63][S02] Do you remember when you[1664.99] [1664.99][S02]realized that these kind of properties were emerging?[1670.33] [1670.33][S02]It must have been a bit of a shock.[1671.96] [1671.96][S01] I think the first time[1673.585] [1673.585][S01]was when we saw the robots actually tying shoelaces.[1677.06] [1677.06][S01]We were like, that's not possible.[1678.59] [1678.59][S01]In fact, when the researchers set up this task,[1681.35] [1681.35][S01]they actually did it to challenge themselves.[1684.003] [1684.003][S01]I think there was a professor that[1685.42] [1685.42][S01]said, oh, I bet if we can get robots to tie shoelaces,[1689.57] [1689.57][S01]I will retire.[1690.38] [1690.38][S01]And the researchers in the team were like, right on.[1694.37] [1694.37][S01]I'm going to add that as a task.[1695.99] [1695.99][S01]And so they actually did.[1697.4] [1697.4][S01]And they were surprised when it was able to do it.[1700.1] [1700.1][S01]I don't know what happened, whether the professor actually[1702.88] [1702.88][S01]saw the video and decided to retire.[1705.14] [1705.14][S01]But the inspiration came from there.[1708.46] [1708.46][S01]And we just continued to add tasks, more and more tasks.[1711.64] [1711.64][S01]Same with the origami example.[1713.068] [1713.068][S01]We were like, we have no idea if this is going to work,[1715.36] [1715.36][S01]but let's try it.[1716.5] [1716.5][S01]And it actually was surprisingly good at it.[1719.62] [1719.62][S01]And it's actually really delicate.[1721.27] [1721.27][S01]It has to actually fold every part of the paper,[1724.63] [1724.63][S01]and it has to do it in the right sequence.[1726.4] [1726.4][S01]If anything goes wrong, it loses its way[1729.21] [1729.21][S01]and has to restart, same as if a human was doing it.[1732.23] [1732.23][S02] I remember the very first time[1733.98] [1733.98][S02]that I got to interview Demis.[1735.28] [1735.28][S02]He was talking about Moravec's paradox, this idea[1737.73] [1737.73][S02]that the tasks that are easy for humans[1740.47] [1740.47][S02]are hard for machines and vice versa.[1742.66] [1742.66][S02]With all of these advances that we have now[1744.96] [1744.96][S02]in robotics, do you think that Moravec's paradox will[1747.45] [1747.45][S02]hold going forwards?[1748.493] [1748.493][S01] I certainly think[1749.91] [1749.91][S01]that it is still more difficult for robots[1753.63] [1753.63][S01]to do something that is incredibly[1755.22] [1755.22][S01]intuitive for us humans to do.[1756.58] [1756.58][S01]So I think Moravec's paradox still holds.[1759.19] [1759.19][S01]We are now at the point where we are confident[1761.64] [1761.64][S01]that if you can teach a robot, if you[1763.56] [1763.56][S01]can operate a robot to do a very complex task, it can learn it.[1768.4] [1768.4][S02] And how quickly does it happen?[1770.27] [1770.27][S02]How many origami foxes does a robot[1772.61] [1772.61][S02]need to watch a human do before it can do one itself?[1775.17] [1775.17][S01] It varies by the complexity of the task,[1777.545] [1777.545][S01]pretty similar to the way it is for humans, right?[1779.63] [1779.63][S01]The more complex the task, the more[1781.7] [1781.7][S01]you need to practice it before you can master it.[1783.84] [1783.84][S01]So there's a lot of tasks that you can master with just about[1787.46] [1787.46][S01]100 examples, and tasks like the origami fox takes about 1,000[1791.96] [1791.96][S01]examples.[1792.69] [1792.69][S02] Wait, so people had to fold origami foxes while[1795.68] [1795.68][S02]pretending to be a robot 1,000 time?[1797.18] [1797.18][S01] Yes, that's right.[1798.638] [1798.638][S02] OK, that is incredibly amusing to me.[1800.875] [1800.875][S01] [LAUGHS] We're trying to reduce it[1803.0] [1803.0][S01]as much as possible, and we are able to get quite a bit of tasks[1806.69] [1806.69][S01]with just, like, a dozen of examples.[1808.232] [1808.232][S02] Are there some that you don't need any examples[1810.69] [1810.69][S02]for at all?[1811.32] [1811.32][S01] In a lot of the examples that we were testing,[1813.42] [1813.42][S01]like when you're playing with the robot[1815.045] [1815.045][S01]and asking it to do a lot of pick-and-place tasks[1817.64] [1817.64][S01]with completely new scenarios, you[1819.98] [1819.98][S01]don't have to teach it again.[1821.28] [1821.28][S01]And this is expanding and getting more and more complex.[1824.31] [1824.31][S01]So for example, the cases with the tiles,[1826.76] [1826.76][S01]where you're moving the tiles around,[1828.89] [1828.89][S01]those it just can reason about positioning of the tiles[1832.34] [1832.34][S01]and decide where to put them.[1834.147] [1834.147][S02] OK, what about the packed lunch?[1835.98] [1835.98][S01] That one is more complex[1837.02] [1837.02][S01]because, again, in that one, you're actually[1838.853] [1838.853][S01]doing a long sequence of tasks, about five minutes of task,[1842.61] [1842.61][S01]and you're actually picking up very deformable things,[1847.511] [1847.511][S01]like the Ziploc bag, and doing very delicate stuff.[1850.71] [1850.71][S01]So the more delicate the task, the more likely it is you[1853.73] [1853.73][S01]need to see examples in that task.[1855.403] [1855.403][S02] So if these robots are having to see examples,[1857.82] [1857.82][S02]does that end up impacting the generality of it?[1860.333] [1860.333][S01] Only to a degree.[1861.75] [1861.75][S01]So one thing that we make sure that we do[1863.54] [1863.54][S01]is that we collect data in thousands of examples[1867.11] [1867.11][S01]without doing a very large emphasis on any new task.[1872.97] [1872.97][S01]If you do want to do the origami task,[1876.21] [1876.21][S01]we simply specialize it for the origami task,[1878.43] [1878.43][S01]and that does affect the generalization of the models[1880.77] [1880.77][S01]today.[1881.69] [1881.69][S01]We're hoping to get to a state where you basically can teach it[1884.63] [1884.63][S01]any new task, to master any new task,[1887.48] [1887.48][S01]and the generality remains intact.[1889.76] [1889.76][S01]But today, it's a trade-off.[1891.17] [1891.17][S02] So in the dream world,[1892.81] [1892.81][S02]you would be able to say--[1894.77] [1894.77][S02]I don't know-- fold me an origami boat.[1897.68] [1897.68][S02]And it would be able to do that just from everything[1900.64] [1900.64][S02]that it understood before.[1902.05] [1902.05][S01] Yeah, in the dream world,[1903.8] [1903.8][S01]it could just watch a video of someone doing it,[1906.19] [1906.19][S01]and it would learn from that.[1907.55] [1907.55][S02] So reinforcement learning[1909.092] [1909.092][S02]was a big thing in robotics for quite a stretch of time.[1914.21] [1914.21][S02]Has that just disappeared now?[1915.86] [1915.86][S01] Not at all.[1916.46] [1916.46][S01]Not at all.[1916.96] [1916.96][S01]We do quite a bit of work still with reinforcement learning,[1919.79] [1919.79][S01]and we continue to explore ways to combine this big foundation[1924.01] [1924.01][S01]models with reinforcement learning.[1925.84] [1925.84][S01]First of all, all of the work that we do around whole body[1928.69] [1928.69][S01]control, like if we have a humanoid that is walking around[1931.78] [1931.78][S01]or a quadruped, they're all using reinforcement learning[1934.33] [1934.33][S01]to learn how to walk around.[1935.9] [1935.9][S02] Because it's very easy to say[1937.608] [1937.608][S02]you fail when you fall over.[1939.26] [1939.26][S01] It's a very mature technology.[1941.36] [1941.36][S01]And you actually can learn it all in simulation.[1943.04] [1943.04][S01]So it doesn't need to fall in order to learn.[1944.93] [1944.93][S01]So you can learn in simulation and then[1946.56] [1946.56][S01]transfer it to the real world.[1947.89] [1947.89][S01]One example that we had on this was a recent paper[1950.31] [1950.31][S01]called DemoStart.[1951.58] [1951.58][S01]In DemoStart, you basically show the robot[1953.88] [1953.88][S01]how to do five different examples.[1955.96] [1955.96][S01]This is manipulating a hand.[1957.847] [1957.847][S01]So you show it five different examples[1959.43] [1959.43][S01]of how to pick up an object and place it[1961.097] [1961.097][S01]in a particular way on an insertion.[1963.125] [1963.125][S02] By insertion, do you mean things like--[1965.25] [1965.25][S02]I don't know-- putting a key in a lock, for instance?[1967.48] [1967.48][S01] Yes, exactly, being[1968.98] [1968.98][S01]able to put one object inside another.[1970.99] [1970.99][S01]So a key in a lock is a great example.[1972.61] [1972.61][S01]You just give it five examples, and it explores on its own[1975.33] [1975.33][S01]and learns how to do it and drastically reduces the amount[1978.21] [1978.21][S01]of data that you need in the real world by 100x.[1981.61] [1981.61][S01]We think this is going to be critical because the truth is[1984.397] [1984.397][S01]you're not going to be able to demonstrate for the robot[1986.73] [1986.73][S01]how to do every single task.[1988.24] [1988.24][S02] Of course.[1989.44] [1989.44][S01] Some of the tasks are going to be complex,[1991.898] [1991.898][S01]and they won't be able to extract that directly[1994.35] [1994.35][S01]from its knowledge of the internet.[1996.455] [1996.455][S01]So it's going to have to explore.[1997.83] [1997.83][S02] Like if you're doing surgery, for example, maybe.[1999.76] [1999.76][S01] Yes.[2000.35] [2000.35][S01]So it's going to have to explore and learn from its behavior.[2003.03] [2003.03][S01]And that's one of the areas that we[2004.488] [2004.488][S01]want to spend a lot more time on,[2005.89] [2005.89][S01]is, how do you get robots that learn on the job?[2008.93] [2008.93][S02] OK.[2009.97] [2009.97][S02]Is doing things in simulation part of the solution then?[2013.84] [2013.84][S01] Yeah, we definitely[2015.34] [2015.34][S01]leverage simulation in multiple ways.[2017.12] [2017.12][S01]We leverage simulation even to learn better how to do 3D[2021.97] [2021.97][S01]understanding of the physical world.[2023.9] [2023.9][S01]We also leverage simulation to learn new behaviors,[2026.42] [2026.42][S01]like in the case of DemoStart.[2028.69] [2028.69][S01]But yeah, when we talk about reinforcement learning,[2031.1] [2031.1][S01]it is not always in simulation.[2032.6] [2032.6][S01]You can also do reinforcement learning[2034.78] [2034.78][S01]to learn how the robot is doing in the real world directly.[2037.67] [2037.67][S01]So we do it in both cases.[2039.23] [2039.23][S01]And simulation is a critical component.[2041.03] [2041.03][S02] Does it work, though?[2042.5] [2042.5][S02]Isn't the real world a bit messier than simulations?[2044.9] [2044.9][S01] Yes.[2045.19] [2045.19][S01]So there is things that are actually much harder[2047.53] [2047.53][S01]to do in simulation first, for example, anything that has to do[2050.92] [2050.92][S01]with deformable simulating.[2053.26] [2053.26][S01]Folding that T-shirt in the air is actually extremely hard.[2057.05] [2057.05][S01]Simulating fluids is really hard.[2059.264] [2059.264][S01]So there are some things that are[2060.639] [2060.639][S01]just easier to learn in the physical world and some things[2064.05] [2064.05][S01]that you can learn a much larger scale in the simulator world.[2067.13] [2067.13][S02] And does one translate to the other?[2069.13] [2069.13][S02]If you do the learning in simulation--[2071.54] [2071.54][S02]I seem to remember, actually-- maybe this is like--[2073.665] [2073.665][S02]I don't know-- eight years ago or something.[2075.969] [2075.969][S02]But there was one robot that was trying to get a ball in a cup,[2079.739] [2079.739][S02]and it could do it in simulation.[2081.73] [2081.73][S02]But then once it came to the reality,[2083.71] [2083.71][S02]all sorts of other factors came into play,[2085.78] [2085.78][S02]maybe the lighting on the camera angle, the exact dimensions[2089.82] [2089.82][S02]of its own limbs.[2090.793] [2090.793][S02]I mean, all of that kind of stuff starts[2092.46] [2092.46][S02]to mess with the numbers, doesn't it?[2094.57] [2094.57][S01] Yes, definitely.[2096.239] [2096.239][S01]That's what we call the sim-to-real gap,[2097.99] [2097.99][S01]and we still have the sim-to-real gap.[2099.94] [2099.94][S01]It certainly has been reduced significantly.[2103.6] [2103.6][S01]When it comes to modeling interactions[2105.75] [2105.75][S01]between a robot and the world, which[2108.12] [2108.12][S01]is really messy and complicated, it's still a problem.[2111.19] [2111.19][S01]We still have some sim-to-real gaps.[2112.75] [2112.75][S01]Essentially, what we end up doing[2114.39] [2114.39][S01]is identifying areas where it is easy to simulate,[2118.51] [2118.51][S01]and we can actually see sim-to-real transfer,[2120.73] [2120.73][S01]and we do quite a bit of that in simulation in areas[2123.8] [2123.8][S01]where it's actually simpler to learn in the physical world.[2126.48] [2126.48][S01]So we combine the strengths of the two.[2128.61] [2128.61][S02] All of these examples that you're giving[2130.777] [2130.777][S02]are really in lab settings.[2131.94] [2131.94][S02]I'm trying to think of the situations in which you would[2134.273] [2134.273][S02]really want a robot to be there, maybe after a natural disaster,[2137.19] [2137.19][S02]for instance.[2138.26] [2138.26][S02]How does it work taking this stuff out of the lab[2141.83] [2141.83][S02]and then putting it out into the real world?[2143.73] [2143.73][S02]What are the additional complications[2145.01] [2145.01][S02]that you need to handle?[2146.19] [2146.19][S01] Definitely all of our research right[2149.24] [2149.24][S01]now is still happening within our labs.[2152.41] [2152.41][S01]But we're super excited about the potential of bringing this[2155.09] [2155.09][S01]to the real world.[2156.357] [2156.357][S01]And there's a lot of additional things[2157.94] [2157.94][S01]we need to think about to do that.[2159.595] [2159.595][S01]Certainly, we're already thinking[2160.97] [2160.97][S01]about the aspect of safety when you bring these--[2164.85] [2164.85][S01]AI is actually moving physically, robots outside[2168.182] [2168.182][S01]and changing the world.[2169.14] [2169.14][S01]You want to think about all the safety aspects.[2171.39] [2171.39][S01]There's also the aspect that you might not[2173.21] [2173.21][S01]have internet access in any one of these locations.[2177.15] [2177.15][S01]And so it is very important that we think about,[2179.87] [2179.87][S01]can we have models that can run directly on the robot[2183.56] [2183.56][S01]and just be air gapped and completely on device?[2186.453] [2186.453][S01]And this might be useful in the case of a natural disaster[2188.87] [2188.87][S01]where there's no connection.[2190.2] [2190.2][S01]It might be useful for applications where there is[2192.29] [2192.29][S01]actually a lot of latency critical components,[2195.36] [2195.36][S01]like it has to respond very quickly and cannot wait[2199.79] [2199.79][S01]for a server connection.[2201.298] [2201.298][S02] Give me an example.[2202.59] [2202.59][S01] Well, I think actually[2204.215] [2204.215][S01]in any of these examples where the robot is operating[2207.17] [2207.17][S01]underground, it's simply not going[2210.29] [2210.29][S01]to be able to connect and wait for some more advanced reasoning[2215.06] [2215.06][S01]module to tell it what to do.[2216.42] [2216.42][S01]It actually has to decide right there and then how to behave.[2219.39] [2219.39][S01]But it loses a little bit of that generalization[2222.29] [2222.29][S01]and reasoning.[2223.1] [2223.1][S02] On that point about safety,[2225.21] [2225.21][S02]I guess, if you are giving robots the ability[2227.75] [2227.75][S02]to act in a physical world, then you[2230.3] [2230.3][S02]are opening up the possibility of different potential risks[2235.14] [2235.14][S02]like--[2235.64] [2235.64][S02]I don't know-- somebody getting into a robot's language model[2239.15] [2239.15][S02]and warping its reasoning.[2241.61] [2241.61][S02]How do you mitigate against those sort of risks?[2244.31] [2244.31][S01] We essentially have a pretty comprehensive[2247.57] [2247.57][S01]safety and security approach that[2250.12] [2250.12][S01]actually goes in multiple layers of the system.[2253.48] [2253.48][S01]Definitely, we think about software security as critical[2257.29] [2257.29][S01]for these robots so that no bad actor can actually[2259.84] [2259.84][S01]interfere and actually take control of the robot.[2262.52] [2262.52][S01]And in terms of safety, it happens[2264.73] [2264.73][S01]on many different levels.[2265.82] [2265.82][S01]So actually, safety for robotics has been there for decades.[2269.6] [2269.6][S01]There's quite a bit of work on making sure[2271.72] [2271.72][S01]that a robot doesn't collide with its environment[2274.13] [2274.13][S01]or doesn't put too strong impact forces on its environment,[2278.21] [2278.21][S01]or it actually walks stably.[2280.43] [2280.43][S01]And the Gemini robotic models can actually[2283.21] [2283.21][S01]just seamlessly interface with any of those safety[2286.69] [2286.69][S01]critical controllers.[2287.855] [2287.855][S01]The other thing that we do is that when[2289.48] [2289.48][S01]you have an AI controlling a robot,[2291.29] [2291.29][S01]you now have to be thinking about semantic physical safety.[2295.76] [2295.76][S01]And what I mean by that is, like,[2298.75] [2298.75][S01]if someone asks you to put the glass on the table,[2301.227] [2301.227][S01]you're not going to put it right at the edge[2303.06] [2303.06][S01]when it's about to fall.[2304.39] [2304.39][S01]You're going to put it actually somewhere in the middle.[2306.73] [2306.73][S01]Or for example, if you actually see[2310.738] [2310.738][S01]that there is something on the floor,[2312.28] [2312.28][S01]you might want to pick it up so that it avoids someone[2314.76] [2314.76][S01]falling or tripping over it.[2316.06] [2316.06][S01]The way we've done that is that we're actually introducing[2318.57] [2318.57][S01]a new data set called ASIMOV dataset,[2322.0] [2322.0][S01]which essentially contains a long list of scenarios[2327.21] [2327.21][S01]that the robot could encounter and how to reason through those.[2330.28] [2330.28][S01]And these are all physical safety scenarios,[2332.47] [2332.47][S01]and it's inspired by, essentially,[2333.91] [2333.91][S01]Asimov's three laws.[2335.26] [2335.26][S01]The first one is a robot may never hurt a human[2338.88] [2338.88][S01]or cause a human to come to harm by inaction.[2341.83] [2341.83][S01]The second one is that a robot should always[2343.89] [2343.89][S01]follow human orders unless it conflicts with the first law.[2347.29] [2347.29][S01]And the third one is that a robot[2349.41] [2349.41][S01]should protect its own existence unless it conflicts[2352.29] [2352.29][S01]with the first and second law.[2354.04] [2354.04][S01]And it was this very comical situation[2356.34] [2356.34][S01]where a robot was stuck between the three different laws.[2359.67] [2359.67][S01]So that's what inspired the ASIMOV dataset.[2363.0] [2363.0][S01]And it actually has quite a bit of information from US injuries[2368.42] [2368.42][S01]reported by hospitals.[2370.97] [2370.97][S01]And inspired by those examples, we actually[2373.61] [2373.61][S01]created a data set that has visual images, like images[2377.828] [2377.828][S01]of something that is about to happen,[2379.37] [2379.37][S01]and a question associated with it,[2380.86] [2380.86][S01]like, what action should you take in order for this[2383.6] [2383.6][S01]to be a safe situation?[2385.41] [2385.41][S01]And the idea is that we would present it to the community,[2388.11] [2388.11][S01]and everyone in the community can start testing their models[2390.68] [2390.68][S01]with respect to this data set.[2392.13] [2392.13][S02] So it turns out Asimov's original three[2393.83] [2393.83][S02]rules are not enough.[2395.13] [2395.13][S02]You need a little bit more.[2395.51] [2395.51][S01] Not enough.[2396.677] [2396.677][S01]Yes.[2397.177] [2397.177][S02] Give me some examples, though, of the kind[2399.427] [2399.427][S02]of things.[2400.13] [2400.13][S01] Some of the examples that we've seen there[2402.588] [2402.588][S01]is you cannot put a stuffed stuffie, plushy on a hot stove,[2409.202] [2409.202][S01]which is something that I wouldn't have thought about[2411.41] [2411.41][S01]making a law about that.[2412.83] [2412.83][S01]But certainly, it has happened, and therefore, it just[2416.26] [2416.26][S01]comes out in the data.[2417.74] [2417.74][S02] Then are we back in the same problem of you're[2420.25] [2420.25][S02]never going to be able to create an exhaustive list of everything[2422.29] [2422.29][S02]that it shouldn't be able to do?[2423.8] [2423.8][S01] I think that it will be really hard for human[2426.383] [2426.383][S01]to sit down and create the perfect law.[2428.84] [2428.84][S01]So part of what we're doing here is[2432.11] [2432.11][S01]leveraging AI to actually understand[2435.16] [2435.16][S01]a broad set of injury situations that[2437.95] [2437.95][S01]have happened in many different countries[2440.11] [2440.11][S01]and then transform that into a better, more succinct list.[2444.02] [2444.02][S01]And then obviously, that list will[2445.66] [2445.66][S01]have to be updated with some frequency.[2448.16] [2448.16][S01]And the idea here is that we derive an initial list,[2452.99] [2452.99][S01]but then humans can check it and decide how much of it[2456.22] [2456.22][S01]to include or not in order to keep the robot safe.[2460.13] [2460.13][S02] How much overlap is there[2461.95] [2461.95][S02]between this list and the work that's been done on safety[2465.43] [2465.43][S02]and agents, for instance?[2466.85] [2466.85][S01] Yeah, we inherit actually[2468.6] [2468.6][S01]all of the safety that happens already for general foundation[2473.59] [2473.59][S01]models like Gemini.[2474.82] [2474.82][S01]And part of what we do is try to take some of those problems,[2479.17] [2479.17][S01]and if they have a physical grounding to them,[2482.11] [2482.11][S01]then that's where we start to advance[2484.68] [2484.68][S01]the model's understanding.[2486.64] [2486.64][S01]So it's typically examples where there[2489.0] [2489.0][S01]might be a situation that if it's on a screen, it's OK.[2492.31] [2492.31][S01]But if it's now in the physical world,[2494.04] [2494.04][S01]it actually has consequences.[2495.91] [2495.91][S02] Are there some things that you would just never[2498.66] [2498.66][S02]really want a robot to perform?[2500.02] [2500.02][S02]I don't know.[2500.562] [2500.562][S02]Like, a massage, for example.[2504.7] [2504.7][S02]Are there some things that you just actually only want[2507.3] [2507.3][S02]a human to be able to do?[2508.342] [2508.342][S01] There are massage robots, I have to say.[2510.717] [2510.717][LAUGHTER][2511.5] [2511.5][S02] There are massage chairs definitely.[2513.0] [2513.0][S02]Are there massage robots as well?[2513.72] [2513.72][S01] There are massage robots, yes.[2515.04] [2515.04][S02] Wow.[2515.37] [2515.37][S02]That's another thing.[2516.49] [2516.49][S01] Yes.[2517.365] [2517.365][LAUGHTER][2519.33] [2519.33][S02] OK, bad example.[2520.75] [2520.75][S02]Are there some things that you think actually[2523.56] [2523.56][S02]should remain human?[2524.79] [2524.79][S02]Nursing, perhaps.[2526.137] [2526.137][S01] Yeah, I think in many ways,[2527.97] [2527.97][S01]what we think is that robots could be a collaborator that[2531.51] [2531.51][S01]can actually enable humans to pay more[2533.84] [2533.84][S01]attention to the human aspects of the job and less attention[2537.53] [2537.53][S01]to those that are about moving things around or picking things[2541.43] [2541.43][S01]up.[2541.98] [2541.98][S01]So you can imagine that if a nurse could have assistants that[2544.73] [2544.73][S01]could actually help it fetch things while they're paying[2547.88] [2547.88][S01]attention to the patient, then that[2549.86] [2549.86][S01]would enable a much better experience for that patient.[2552.54] [2552.54][S02] You said something really nice[2553.37] [2553.37][S02]at the beginning about how the robots that we've got now,[2555.745] [2555.745][S02]it's like looking at two-year-olds.[2557.25] [2557.25][S02]I mean, quite talented two-year-olds.[2559.7] [2559.7][S02]But I see what you're saying, that they're just demonstrating[2564.23] [2564.23][S02]the beginnings of something.[2566.03] [2566.03][S02]What kind of breakthroughs do you[2567.65] [2567.65][S02]think still need to happen before we[2570.41] [2570.41][S02]get to the adult version of these robots?[2573.243] [2573.243][S01] Yeah, I mean, there's quite a bit of work[2575.66] [2575.66][S01]to be done, definitely, in the aspects of capturing dexterity[2580.58] [2580.58][S01]with generalization, being able to do both of those things,[2584.01] [2584.01][S01]and just continuously grow without losing one or the other.[2588.18] [2588.18][S01]The other key area is that you want these robots[2590.18] [2590.18][S01]to learn on the job.[2592.5] [2592.5][S01]There's no way these robots are going[2594.167] [2594.167][S01]to learn everything they need to learn in the lab,[2596.25] [2596.25][S01]and then you put them out, and they just work.[2598.77] [2598.77][S01]I think the reality is that you would put them out.[2600.99] [2600.99][S01]They will experience new things, and you[2602.81] [2602.81][S01]want them to learn from those experiences[2604.7] [2604.7][S01]and get better and better over time.[2606.57] [2606.57][S01]So that's another area.[2607.83] [2607.83][S01]Also, robots that are more social.[2610.35] [2610.35][S01]I think, certainly, all of these foundation models[2613.94] [2613.94][S01]enable robots to have a lot better understanding[2616.7] [2616.7][S01]of semantics and the world, but they still lack social skills.[2620.85] [2620.85][S01]They still cannot read body language.[2623.46] [2623.46][S01]They cannot understand how to behave in a very cluttered[2626.24] [2626.24][S01]space, like a cocktail party.[2628.65] [2628.65][S01]So there's quite a bit of work there.[2630.36] [2630.36][S02] So how far away do you[2631.777] [2631.777][S02]think we are, then, from the kind of Rosie the Robot that you[2635.93] [2635.93][S02]saw in your childhood?[2637.343] [2637.343][S01] I don't think I have an exact date,[2639.51] [2639.51][S01]but I can tell you before we used to have discussions about[2643.16] [2643.16][S01]whether it would happen in our lifetime or even in our careers,[2647.28] [2647.28][S01]and now we have debates about whether it[2649.4] [2649.4][S01]would be 5 or 10 years.[2651.24] [2651.24][S01]So it's certainly shifted.[2653.99] [2653.99][S01]And it feels like the next two years[2655.96] [2655.96][S01]are going to be pretty defining for the field of robotics.[2658.97] [2658.97][S01]There's just a lot of things that[2660.7] [2660.7][S01]are coming together-- understanding, dexterity,[2664.49] [2664.49][S01]whole body control.[2666.05] [2666.05][S01]It is all started.[2667.28] [2667.28][S01]You can see how this could actually merge[2669.55] [2669.55][S01]into a very strong solution.[2671.54] [2671.54][S02] Do you think that's what[2672.1] [2672.1][S02]we're about to see, then, in the same way as we've[2673.66] [2673.66][S02]seen the explosion of large language models?[2675.74] [2675.74][S02]Do you think the next thing is the explosion of robotics?[2678.145] [2678.145][S01] Yes, absolutely.[2679.52] [2679.52][S01]And I think actually being better[2681.91] [2681.91][S01]at operating in the physical world actually[2684.46] [2684.46][S01]will make our LLMs and VLMs significantly stronger AI models[2689.36] [2689.36][S01]because they can now understand the space of humans.[2693.47] [2693.47][S02] Things are about to change.[2695.17] [2695.17][S02]Thank you so much.[2696.38] [2696.38][S02]Absolutely fascinating.[2697.52] [2697.52][S02]Really amazing.[2698.54] [2698.54][S01] Thank you for having me.[2699.29] [2699.29][S02] Thank you.[2700.207] [2700.207][S02]I don't know if you've ever noticed this little guy sitting[2703.06] [2703.06][S02]behind me.[2704.03] [2704.03][S02]These were the reinforcement learning kings.[2707.12] [2707.12][S02]For literally years, they wandered around[2710.14] [2710.14][S02]in little robot playpens trying and largely[2713.01] [2713.01][S02]failing to learn how to walk, how[2714.84] [2714.84][S02]to play football, how not to continually fall over[2717.54] [2717.54][S02]all of the time.[2718.72] [2718.72][S02]And now, almost overnight, once language and reasoning[2723.21] [2723.21][S02]and conceptual understanding arrived[2725.07] [2725.07][S02]as the missing pieces of the puzzle,[2727.72] [2727.72][S02]they have been confined to the shelves of podcast studios.[2732.6] [2732.6][S02]And in all that time, the researchers,[2735.22] [2735.22][S02]they were focused on the robot's body.[2737.05] [2737.05][S02]But it was advances in the mind that[2739.59] [2739.59][S02]made the biggest leaps forward.[2741.228] [2741.228][S02]You've been listening to "Google DeepMind--[2743.02] [2743.02][S02]The Podcast" with me, Professor Hannah Fry.[2745.18] [2745.18][S02]If you enjoyed this episode, then[2746.73] [2746.73][S02]do subscribe to our YouTube channel.[2748.42] [2748.42][S02]You can also find us on your favorite podcast platform.[2751.69] [2751.69][S02]And of course, we have plenty more episodes[2754.05] [2754.05][S02]on a whole range of topics to come, so do check those out.[2757.72] [2757.72][S02]See you next time.[2758.73] [2758.73][MUSIC PLAYING][2762.38]