[0.0][S01] There was a really nice quote that I came across,[2.02] [2.02][S01]which is that almost no prerequisite[3.72] [3.72][S01]to any major invention was made with that invention in mind.[7.14] [7.14][S01]Can you imagine a point in the future[9.76] [9.76][S01]where you are letting agents loose in these environments[13.04] [13.04][S01]without specifying an objective for them?[15.52] [15.52][S03] I think we as humans,[17.42] [17.42][S03]we decide what's interesting.[19.14] [19.14][S03]I think there is even an example where[20.96] [20.96][S03]the entire evolution of mathematics[22.96] [22.96][S03]was guided by people deciding what's next, what's interesting,[26.78] [26.78][S03]what's not interesting, and just the problem being hard[30.098] [30.098][S03]doesn't mean that it's interesting at all, right?[32.14] [32.14][S02] I believe very strongly[33.2] [33.2][S02]that we need simulation, and I also[35.182] [35.182][S02]believe very strongly that we won't[36.64] [36.64][S02]be able to build a simulator of the real world any other way.[40.038] [40.038][S02]So when you combine those two things,[41.58] [41.58][S02]I think, yes, it is a big step for my version of AGI.[45.327] [45.327][MUSIC PLAYING][47.662] [51.163][S01] Welcome back to "Google DeepMind--[53.08] [53.08][S01]The Podcast."[53.86] [53.86][S01]I'm Professor Hannah Fry.[55.72] [55.72][S01]Now the latest video generation models[58.2] [58.2][S01]have impressed the entire world.[59.8] [59.8][S01]They've created this near perfect imitation of reality.[64.0] [64.0][S01]But the limitations of video is that you are just[66.72] [66.72][S01]a viewer rather than a participant.[69.08] [69.08][S01]And that's not how humans experience the real world.[72.24] [72.24][S01]We instead can navigate environments we've never been to[75.36] [75.36][S01]and still have an expectation of what we're likely to encounter.[78.96] [78.96][S01]We can explore in every feasible direction, kind[82.16] [82.16][S01]of without limits, and interact with things that we chance upon[86.12] [86.12][S01]along the way.[87.52] [87.52][S01]And that is the next great frontier for this technology,[91.32] [91.32][S01]to move beyond generating a perfect recording of a scene[95.08] [95.08][S01]and towards building a dynamic simulation of a world[98.4] [98.4][S01]we can finally step into.[100.96] [100.96][S01]Enter Genie 3, a prototype world model[104.12] [104.12][S01]that can generate an unprecedented variety[107.32] [107.32][S01]of interactive environments.[109.4] [109.4][S01]It's already been described as a stepping stone towards AGI.[113.48] [113.48][S01]And with me today are two of its creators,[116.52] [116.52][S01]Shlomi Fruchter, research director,[118.46] [118.46][S01]and Jack Parker-Holder, research scientist.[120.69] [120.69][S01]Welcome to the podcast, both of you.[122.19] [122.19][S01]Can you sum this up in a sentence for me?[123.898] [123.898][S01]What is Genie 3?[124.98] [124.98][S02] It's a real time, interactive world model[127.73] [127.73][S02]that allows you to create diverse, visually interesting[130.97] [130.97][S02]worlds from a text prompt.[132.71] [132.71][S02]So there is no underlying game engine, no structure, no code.[137.27] [137.27][S02]It's just a neural network that's[139.09] [139.09][S02]predicting every single pixel in reaction[141.41] [141.41][S02]to inputs from the user and also the past.[144.45] [144.45][S02]And so the flexibility in the diversity[146.49] [146.49][S02]of things you can create in basically no time[149.21] [149.21][S02]is quite unprecedented.[150.91] [150.91][S01] You haven't had a whole army of artists sitting[153.45] [153.45][S01]in rooms constructing a world in order to be able to interact.[156.033] [156.033][S03] Yes, I think the point is that you can create[158.617] [158.617][S03]any world that you can imagine.[160.19] [160.19][S03]And that's not something that you can do with a game engine.[162.825] [162.825][S01] OK.[163.45] [163.45][S01]Let's have a look at it, because you've[164.45] [164.45][S01]got some demos for me, right?[165.89] [165.89][S03] Yeah.[166.807] [166.807][S03]So we have a few.[168.05] [168.05][S03]The first one, I think you might like it.[170.63] [170.63][S03]So it's basically playing a cat.[172.75] [172.75][S01] OK.[173.973] [173.973][S01]You've got me already.[174.89] [174.89][S03] A ginger cat.[176.73] [176.73][S01] Excellent.[177.792] [177.792][S03] Let me run it now.[179.25] [179.25][S03]We basically prompted it with being a cat.[181.71] [181.71][S01] So this is reacting to the inputs[184.21] [184.21][S01]that you're giving it.[185.35] [185.35][S03] Yes, exactly.[186.61] [186.61][S01] Is the light going to change as you go into the--[189.152] [189.152][S01]oh, look at that.[190.81] [190.81][S01]Look at that.[191.61] [191.61][S03] Yes.[192.485] [192.485][S03]So the model is basically trying to predict[194.77] [194.77][S03]what's going to happen next based on the sequence of inputs[198.29] [198.29][S03]that it gets, and it does it in real time.[200.155] [200.155][S01] So looking at this superficially,[202.03] [202.03][S01]I mean, this doesn't look that different from if I just[205.09] [205.09][S01]loaded up a video game, right?[206.99] [206.99][S01]How is it different?[208.15] [208.15][S03] Oh, I think the key difference is[209.45] [209.45][S03]that this is generated from text,[211.09] [211.09][S03]so every single pixel is generated from a model which[214.65] [214.65][S03]is just predicting the pixels.[216.467] [216.467][S01] I mean, there's also the detail that you're seeing[219.05] [219.05][S01]in this 3D environment.[220.51] [220.51][S01]It's also quite reminiscent of some of the stuff[222.77] [222.77][S01]that we're seeing with Veo if I didn't know that you[225.81] [225.81][S01]were interacting with it.[227.27] [227.27][S01]How is it different from that?[228.93] [228.93][S03] So when you create a video using Veo,[231.66] [231.66][S03]so you provide a prompt and then the model[233.41] [233.41][S03]is trying to figure out how to create[234.952] [234.952][S03]this entire video of, say, eight seconds from start to finish.[241.1] [241.1][S03]And once it's ready, then you cannot change how the camera[245.78] [245.78][S03]moves around, and definitely you cannot explore it much more than[250.54] [250.54][S03]just eight seconds.[251.332] [251.332][S01] Can you use an image to prompt this or is[253.54] [253.54][S01]it only text?[254.16] [254.16][S03] Yes, so we just found out[255.91] [255.91][S03]that we can actually use an image and videos to prompt[259.62] [259.62][S03]them all.[260.899] [260.899][S03]In this particular case, we found that we can actually[263.3] [263.3][S03]use paintings.[264.56] [264.56][S03]For example, this is "Nighthawks" by Edward Hopper.[266.927] [266.927][S01] Very famous picture.[268.26] [268.26][S03] Very famous picture, and from 1942.[271.56] [271.56][S03]And yeah, basically we asked Genie 3[274.9] [274.9][S03]to let us walk into the painting.[278.58] [278.58][S01] So this painting is of a very vivid image, a street[283.3] [283.3][S01]corner at nighttime.[285.16] [285.16][S01]You're looking in through the glass to see a man and a woman[289.06] [289.06][S01]leaning up against a bar, and then[291.5] [291.5][S01]someone serving drinks the other side of the counter.[293.92] [293.92][S01]It's got these rich greens.[295.92] [295.92][S01]The pavement underneath.[297.34] [297.34][S01]The way the light falls is really, really evocative.[300.58] [300.58][S01]But now that you're navigating this space and turning around,[304.46] [304.46][S01]I mean, this is really extraordinary.[307.64] [307.64][S01]It's like you've imagined what would be on either side.[309.973] [309.973][S01]Can you go around the back of the man and the woman?[312.14] [312.14][S01]I want to see the back of their heads.[313.78] [313.78][S01]And you can.[314.92] [314.92][S01]So it's like, got all dimensions of the original painting.[317.745] [317.745][S01]Let's turn around and see the rest of the street as well.[320.12] [320.12][S01]And in the distance, you can see these street lights just pinging[324.78] [324.78][S01]off into the distance as the road extends.[326.86] [326.86][S01]Some billboards above other shops and establishments[331.94] [331.94][S01]off into the distance.[333.26] [333.26][S01]And now we're looking back.[335.02] [335.02][S01]We've chosen to look back at the image of the original painting[338.1] [338.1][S01]to see it there.[339.26] [339.26][S01]That is really amazing.[340.92] [340.92][S01]It's all consistent, all completely consistent.[344.0] [344.0][S01]Have you got another one for us?[345.58] [345.58][S03] Yeah.[346.497] [346.497][S03]We have one that we're just kind of controlling the jet[349.5] [349.5][S03]ski around a few islands.[353.08] [353.08][S03]So let's see how it goes.[354.623] [354.623][S01] Tell me the original prompt.[356.29] [356.29][S03] So it's sailing a jet ski[358.04] [358.04][S03]through the waters around the islands of Kauai.[360.43] [360.43][S01] Ooh, sounds dreamy.[362.05] [362.05][S03] The waters have different ramps[364.35] [364.35][S03]that we can go up on.[365.85] [365.85][S03]Yeah.[366.35] [366.35][S01] OK.[366.81] [366.81][S03] Let's see.[367.49] [367.49][S01] All right, here we go.[368.49] [368.49][S01]OK.[368.99] [368.99][S01]So we are the POV of the person on the jet ski[373.33] [373.33][S01]We can see the hands in frame.[375.57] [375.57][S01]Both hands are consistent with each other, I should add.[378.43] [378.43][S01]The water is beautifully still.[380.41] [380.41][S01]You can see these islands in the background.[382.53] [382.53][S01]The sun is quite low in the sky and you[384.59] [384.59][S01]can see the reflection of its rays on the water.[388.31] [388.31][S01]Now, I see you're taking us up a ramp here.[390.21] [390.21][S03] Trying.[390.65] [390.65][S03]I'm a bit too slow.[391.61] [391.61][S03]I don't know.[392.425] [392.425][S01] What is it going to do on the way down?[394.55] [394.55][S03] Ah.[395.15] [395.15][S01] I mean--[396.11] [396.11][S01]oh, and it splashes when it hits the water.[398.41] [398.41][S03] Let's see what's-- let's look back.[399.77] [399.77][S01] And when you look round, when you look around[401.71] [401.71][S01]to the back, there's a trail in the water[403.85] [403.85][S01]exactly as you would expect from a real jet ski.[406.27] [406.27][S01]So are you seeing elements of where[408.03] [408.03][S01]it's understanding physics?[409.19] [409.19][S02] Definitely, yes.[410.69] [410.69][S02]There's some things we, I guess, refer to as emergent properties,[415.67] [415.67][S02]just from having general training[418.19] [418.19][S02]and seeing lots of different things.[420.51] [420.51][S02]When it sees a new scenario, it understands how smoke moves[423.51] [423.51][S02]or how water should flow.[425.15] [425.15][S02]But that's not maybe 100% accurate[427.59] [427.59][S02]in every single setting.[428.85] [428.85][S02]But it's got enough accuracy that you[430.51] [430.51][S02]do feel some sense of being in the scene,[432.61] [432.61][S02]and as humans, we can't obviously[434.27] [434.27][S02]spot these things that are wrong with it.[435.978] [435.978][S03] So I think, as Jack said,[437.728] [437.728][S03]there are definite limitations.[439.17] [439.17][S03]But on the other hand, after I've[441.23] [441.23][S03]been working on game engines in the past,[443.29] [443.29][S03]and I think we worked really hard to make[445.11] [445.11][S03]all of these kind of effects independently,[448.77] [448.77][S03]like the lens flare and the water simulation, everything.[451.213] [451.213][S03]And here we have basically a model[452.63] [452.63][S03]that can do all of that out of the box with some limitations,[455.81] [455.81][S03]of course.[456.31] [456.31][S01] Without even really trying.[457.65] [457.65][S02] And there's other things[459.483] [459.483][S02]that it can do that would be almost impossible to get[461.71] [461.71][S02]through other methods, right?[462.918] [462.918][S02]Like simulating other animals and people in the world.[467.25] [467.25][S02]I think that's something that's really exciting more[469.47] [469.47][S02]in the future as well, is to be able to interact[471.72] [471.72][S02]with other agents in the world.[473.452] [473.452][S01] Yeah.[474.16] [474.16][S01]I mean, this totally interactive environment[476.262] [476.262][S01]that ends up being consistent regardless of which direction[478.72] [478.72][S01]you push it in.[479.38] [479.38][S01]I mean, that's really extraordinary.[480.88] [480.88][S01]I mean, these demos demonstrate the proof of concept, I guess.[484.317] [484.317][S01]But how would you see this being used?[485.9] [485.9][S01]What are the kind of applications[486.8] [486.8][S01]that you'd be looking at?[487.78] [487.78][S03] One thing that we[488.64] [488.64][S03]are very excited about is using it[490.52] [490.52][S03]for actually the simulation environment for agents.[494.14] [494.14][S03]So, for example, you can imagine an agent that[497.6] [497.6][S03]wants to accomplish a goal.[499.84] [499.84][S03]And then we can put them in any environment that we can imagine,[504.02] [504.02][S03]maybe one that is more challenging for it.[507.513] [507.513][S03]And then it can explore the environment,[509.18] [509.18][S03]maybe try and accomplish a goal and learn from its mistakes.[511.7] [511.7][S03]Again, not without doing anything[513.08] [513.08][S03]in the real world, which is very expensive.[515.02] [515.02][S03]Another thing that we're excited about[516.76] [516.76][S03]is actually using those simulations for planning.[519.542] [519.542][S03]So if you have a robot or you have,[521.0] [521.0][S03]again, an agent that wants to accomplish a goal,[523.72] [523.72][S03]they can maybe do some rollouts in this simulation,[526.62] [526.62][S03]figure out what might happen.[528.1] [528.1][S03]For example, if they want to go across the road,[531.8] [531.8][S03]the agent can use the model to predict a few options.[535.3] [535.3][S03]Maybe there is a few scenarios.[537.7] [537.7][S03]Maybe a person going to cross its path,[539.718] [539.718][S03]maybe something else going to happen.[541.26] [541.26][S03]And then using those rollouts, it[543.28] [543.28][S03]can decide what's the next action it should take.[545.94] [545.94][S03]And then this is used for planning.[547.96] [547.96][S03]And beyond that, we just see a lot[549.56] [549.56][S03]of applications for education, for entertainment.[552.13] [552.13][S01] Just anchor this in a few examples for me.[554.38] [554.38][S01]So I mean, what's the idea here?[556.58] [556.58][S01]Are you going to, in a history lesson,[558.82] [558.82][S01]be able to create a world of Victorian England, for instance?[562.233] [562.233][S02] Exactly.[563.4] [563.4][S02]Yeah.[563.9] [563.9][S02]So imagine you are in front of a bunch of students.[567.235] [567.235][S02]They're obviously excited to learn about Victorian England,[569.9] [569.9][S02]but they've also got a lot of other distractions, things[571.88] [571.88][S02]that they're interested in as well.[573.36] [573.36][S02]Instead of just reading a textbook,[575.18] [575.18][S02]you can instead allow them to step into the world[577.38] [577.38][S02]and you can take them on a virtual tour,[579.24] [579.24][S02]in a sense, of what it would have been like to be there.[581.573] [581.573][S02]So for places that maybe are harder to access, maybe[584.96] [584.96][S02]for distant corners of the planet[586.52] [586.52][S02]or perspectives that you couldn't otherwise[588.65] [588.65][S02]get-- so being a jaguar, for example,[592.17] [592.17][S02]or other kind of animals, or being a shark,[595.036] [595.036][S02]or going back in the past.[596.382] [596.382][S02]These are things that you couldn't really[598.09] [598.09][S02]get any other way as an experience,[599.622] [599.622][S02]and it might make more visual learners in particular,[601.83] [601.83][S02]I think, resonate more with them.[603.323] [603.323][S01] And that's when you have the human who's[605.49] [605.49][S01]playing with the controls.[606.573] [606.573][S01]But as you say, if you then let an agent loose in this,[609.85] [609.85][S01]that opens up a whole other level of possibilities.[612.51] [612.51][S03] Yeah.[613.427] [613.427][S03]So an agent basically--[614.73] [614.73][S03]again, once you have a real or a very close[617.21] [617.21][S03]to real simulation of an environment,[619.35] [619.35][S03]then the agent can use it instead[621.53] [621.53][S03]of actually learning in the real world, which is very costly.[626.01] [626.01][S03]If an agent makes a mistake or if a robot makes[628.21] [628.21][S03]a mistake in the real world, it's much harder to fix.[634.29] [634.29][S03]Basically, this is a way for agents[636.49] [636.49][S03]to learn in a simulated environment[637.97] [637.97][S03]where we can control everything.[639.43] [639.43][S03]We can set up some environment that is maybe[641.37] [641.37][S03]more challenging or less predictable than what typically[645.49] [645.49][S03]the agent was trained to do.[647.49] [647.49][S03]And this way basically the agent can improve[650.49] [650.49][S03]in this safe simulation.[653.09] [653.09][S03]So we're very excited about that.[654.47] [654.47][S01] So let's say that you were maybe running a factory[657.053] [657.053][S01]and you wanted to install a robot with a particular task.[660.23] [660.23][S01]You can recreate the precise environment[662.05] [662.05][S01]it will find itself in and allow it to,[664.21] [664.21][S01]well, find its own mistakes.[666.27] [666.27][S02] Yeah.[666.37] [666.37][S02]I mean, it's a great example because this is already[668.537] [668.537][S02]something that we're close to being able to do already.[671.25] [671.25][S02]Robots are getting quite capable.[672.95] [672.95][S02]But I think what's even more exciting is things[674.912] [674.912][S02]that it's quite far away from being able to do,[676.87] [676.87][S02]that this could completely enable and unlock.[679.25] [679.25][S02]So having robots and embodied agents[682.13] [682.13][S02]actually in the real world.[683.93] [683.93][S02]The diversity of possible scenarios[686.17] [686.17][S02]is just quite hard to fathom, I think, for our current systems.[689.998] [689.998][S02]I think of this example of what would an embodied agent[692.29] [692.29][S02]do on Halloween, right?[693.723] [693.723][S02]Maybe one day a year it sees children running around[695.89] [695.89][S02]in costumes.[696.835] [696.835][S02]What would it do the first time it sees this?[698.71] [698.71][S02]It's quite a challenging scenario to prepare for.[701.453] [701.453][S02]And even if you've seen it before,[702.87] [702.87][S02]it might be different in the next year.[704.81] [704.81][S02]And so to really be able to simulate these rare events[707.26] [707.26][S02]and be able with text to describe any imaginable world,[710.812] [710.812][S02]to become robust to it, make sure the robots are safe[713.02] [713.02][S02]or the agents are safe.[714.78] [714.78][S02]They understand all these different things,[716.572] [716.572][S02]but they can also learn from their experience[718.447] [718.447][S02]as well, which we know is really important.[720.28] [720.28][S01] But then I also wonder,[721.16] [721.16][S01]I mean, all the examples that we've given so far[723.16] [723.16][S01]has been that experiencing the real world at the human level,[725.92] [725.92][S01]as it were, in terms of our size.[727.96] [727.96][S01]Could you shrink this down and create a simulated world[732.22] [732.22][S01]that was at the level of molecules or the human cell,[734.88] [734.88][S01]for instance?[735.44] [735.44][S03] Yeah, so we tried that[736.46] [736.46][S03]and we actually have a few examples we're moving around[739.1] [739.1][S03]like a blood vessel.[740.4] [740.4][S03]It's not necessarily always biologically accurate,[744.26] [744.26][S03]but this is not-- we don't think it's a fundamental limitation.[747.28] [747.28][S03]We could probably, if we had a more accurate simulation[752.82] [752.82][S03]that models can be trained on, then we see that in the future,[755.48] [755.48][S03]maybe we have other variants of models[757.22] [757.22][S03]that are able to specialize in this particular environment.[760.48] [760.48][S03]But we did try to focus more on the real world, the world[764.98] [764.98][S03]from the eyes of a person, because we[766.9] [766.9][S03]think that's the most widely applicable in terms[771.14] [771.14][S03]of the generality of the model.[773.38] [773.38][S01] Is this about just utilizing the existing[776.333] [776.333][S01]developments in artificial intelligence,[778.0] [778.0][S01]or is this also about stepping us[779.94] [779.94][S01]closer towards the goal of AGI?[781.232] [781.232][S02] We think this is definitely[783.19] [783.19][S02]a kind of a new kind of foundation model.[785.02] [785.02][S02]And that's why the breadth of applications[786.9] [786.9][S02]is so wide but also so nascent, because we've never really[792.42] [792.42][S02]had this kind of model before.[793.96] [793.96][S02]It blends and combines ideas from what[796.82] [796.82][S02]we've seen in language models and also video models,[799.46] [799.46][S02]with some of the techniques that we use.[801.68] [801.68][S02]And so it's kind of combining these different elements[804.02] [804.02][S02]in something quite new, which I think[806.58] [806.58][S02]is what's so exciting about it.[808.36] [808.36][S02]And this breakthrough that we've made[810.34] [810.34][S02]is that it might enable some completely new applications[812.86] [812.86][S02]that we didn't really have before.[814.58] [814.58][S02]And so it's still quite an early stage for this research,[817.28] [817.28][S02]but we're quite excited to see what the next few months bring.[820.16] [820.16][S01] Well, let me dig into that a bit deeper,[822.327] [822.327][S01]then, because I know that your background is much more in Veo[824.91] [824.91][S01]and the video side of things, but you were working on Genie 1[828.67] [828.67][S01]and Genie 2 before you and your team came on board.[833.27] [833.27][S01]What were you doing then?[834.41] [834.41][S01]What was the inspiration?[835.452] [835.452][S01]And how is it different from this iteration?[837.55] [837.55][S02] So before Genie,[839.05] [839.05][S02]I was working on open-ended learning.[841.77] [841.77][S02]So training agents in large simulated environments[844.59] [844.59][S02]where we could configure different components[846.75] [846.75][S02]of the world.[847.41] [847.41][S02]So we worked on the xLAM project,[849.15] [849.15][S02]and the idea there was to basically,[851.03] [851.03][S02]with procedural generation, generate[852.63] [852.63][S02]wide variety of environments that were still specified[855.35] [855.35][S02]in code and have the agent learn from these different experiences[858.47] [858.47][S02]and to become a generalist agent in simulation.[861.15] [861.15][S02]But ultimately, we were bottlenecked by the availability[863.51] [863.51][S02]of environments.[865.23] [865.23][S02]We were also, in my PhD, working a bit[867.11] [867.11][S02]on world models as well, but with[868.91] [868.91][S02]much more limited, constrained setting.[871.05] [871.05][S02]We were training world models from single environments,[873.59] [873.59][S02]typically quite low dimensional ones.[876.35] [876.35][S02]And the dream was really to combine these ideas,[878.548] [878.548][S02]to learn general world models that could be used,[880.59] [880.59][S02]simulators for any imaginable tasks,[882.61] [882.61][S02]and then train agents in them to solve completely new things[885.848] [885.848][S02]and have this kind of open-ended loop[887.39] [887.39][S02]where we generate new worlds and the agents[889.182] [889.182][S02]would learn from those.[890.55] [890.55][S02]We started with Genie 1 as a proof of concept, like,[893.15] [893.15][S02]could we do this at all?[894.17] [894.17][S02]Could we generate new worlds that were interactive at all?[897.13] [897.13][S02]And that was quite a major breakthrough.[899.11] [899.11][S02]And then with Genie 2, we scaled this to any 3D sort[901.79] [901.79][S02]of environment.[902.95] [902.95][S01] There were some things that emerged from--[906.03] [906.03][S01]emergent properties in that work.[907.435] [907.435][S01]Tell me a little bit about those.[908.81] [908.81][S01]Were they expected?[909.89] [909.89][S02] So for Genie 2, the question for us was,[912.63] [912.63][S02]can this idea scale?[913.75] [913.75][S02]Right?[914.25] [914.25][S02]Because Genie 1, it was quite a simple proof of concept.[918.422] [918.422][S02]It was, does this work at all?[920.69] [920.69][S02]Whereas Genie 2 was like, is this something[922.71] [922.71][S02]that could really scale to look more like what we see[925.95] [925.95][S02]in foundation models nowadays?[927.337] [927.337][S02]And we weren't sure if it would really work.[929.17] [929.17][S02]We weren't sure if it would be consistent for very long,[931.503] [931.503][S02]because Genie 1 only lasted a couple of seconds.[933.542] [933.542][S02]We weren't sure if it would work at higher resolution[935.75] [935.75][S02]because Genie 1 was 90p, which is very small images.[939.21] [939.21][S02]Genie 2 was 360p.[940.84] [940.84][S02]And then the type of diversity of environments[942.84] [942.84][S02]was a significant increase.[944.572] [944.572][S02]And so given all of that, we weren't sure[946.28] [946.28][S02]if it would be possible to have a single neural network that[948.78] [948.78][S02]could simulate anything within that domain.[952.16] [952.16][S02]And so when we actually got the model,[954.9] [954.9][S02]it was definitely an emergent property[956.68] [956.68][S02]for completely new worlds, and we used Imagen 3[959.4] [959.4][S02]to generate the starting frames back then.[962.28] [962.28][S02]It could do things simulate smoke[964.04] [964.04][S02]or, when you drove off the side of a cliff,[966.46] [966.46][S02]the car would have gravity, or if you landed in a puddle,[969.5] [969.5][S02]it splashed.[971.0] [971.0][S02]And it was quite surprising that this worked so well.[973.26] [973.26][S02]And that gave us the confidence that this next step with Genie 3[976.52] [976.52][S02]would be possible.[977.66] [977.66][S01] So Genie 1, 2D.[979.1] [979.1][S02] Yep.[980.1] [980.1][S01] Genie 2, 3D.[981.6] [981.6][S02] Mhm.[982.6] [982.6][S01] So with Genie 1, then, you're like,[984.4] [984.4][S01]right, we want to be able to create any sort of environment,[986.9] [986.9][S01]and you feed in--[987.892] [987.892][S01]I mean, it was platform games, wasn't it?[989.6] [989.6][S02] Yep.[990.08] [990.08][S01] Just tons and tons and tons and tons[992.08] [992.08][S01]of footage of platform games.[993.5] [993.5][S02] Exactly, yeah.[994.22] [994.22][S01] And then were there any emergent properties in that?[996.92] [996.92][S02] Well, the fact[997.52] [997.52][S02]that you could generate completely new ones, I think[999.92] [999.92][S02]was surprising.[1000.74] [1000.74][S02]I don't think that had really been shown before.[1002.86] [1002.86][S03] Even a painting, right?[1003.78] [1003.78][S02] Yeah.[1004.54] [1004.54][S02]Actually--[1004.8] [1004.8][S03] A drawing.[1005.44] [1005.44][S02] Yeah, I mean, you[1006.982] [1006.982][S02]know the work better than me.[1008.24] [1008.24][S02]We even had a picture of my dog in the park,[1010.6] [1010.6][S02]and you could move her left and right like a platform game.[1013.98] [1013.98][S02]But of course, she's not one.[1016.92] [1016.92][S02]We also had Jeff Clune was an advisor of the project.[1020.1] [1020.1][S02]His children did a bunch of drawings,[1022.04] [1022.04][S02]and we were able to animate those and move them[1024.44] [1024.44][S02]around like games.[1025.692] [1025.692][S02]And I think it's safe to say that was not[1027.4] [1027.4][S02]in the training data.[1028.3] [1028.3][S02]So I'm happy to call that an emergent property.[1032.44] [1032.44][S02]It looked quite different to what it was trained on.[1034.758] [1034.758][S02]And thanks, Shlomi, for reminding me,[1036.3] [1036.3][S02]because it's been a couple of years.[1037.938] [1037.938][S01] And then the step between Genie 1 and Genie 2 was[1040.48] [1040.48][S01]making it 3D.[1041.619] [1041.619][S02] Yeah, so it was increasing the diversity[1043.24] [1043.24][S02]of the things it could do.[1044.599] [1044.599][S02]So compared to Genie 1, which was just 2D platform games,[1048.98] [1048.98][S02]it was 2D games, but as well as 3D environments as well[1051.92] [1051.92][S02]in the same model.[1053.16] [1053.16][S02]It was also higher resolution, and there[1054.868] [1054.868][S02]was a lot more consistency so it would last a bit longer when[1057.41] [1057.41][S02]you interacted with it.[1058.75] [1058.75][S02]So it was really a step up in capability[1060.45] [1060.45][S02]in a few different dimensions, which[1062.33] [1062.33][S02]required a much more concentrated effort[1065.01] [1065.01][S02]to get that to be possible.[1066.77] [1066.77][S02]And we weren't sure if it would actually work at all.[1069.05] [1069.05][S02]So it was more of a proof of concept[1070.97] [1070.97][S02]at a slightly larger scale.[1072.69] [1072.69][S02]And that gave us confidence that what we have now[1075.01] [1075.01][S02]would be achievable.[1076.21] [1076.21][S01] So you were working on building an environment[1078.627] [1078.627][S01]that you could interact with, and meanwhile, in parallel, you[1081.57] [1081.57][S01]were working on video generation.[1083.23] [1083.23][S03] Yes.[1084.105] [1084.105][S03]So my background is actually in 3D game engines.[1086.91] [1086.91][S03]And a very long time ago, I used to work on simulations.[1090.49] [1090.49][S03]That was kind of where I started working on AI.[1094.297] [1094.297][S03]Back then we didn't use even AI.[1095.63] [1095.63][S03]We called it ML.[1097.49] [1097.49][S03]But in the last few years, I really[1100.17] [1100.17][S03]got more excited as the technology[1103.17] [1103.17][S03]evolved and then worked on image models, video models.[1106.81] [1106.81][S03]And yeah, in the last two years, video models, as we all know,[1110.13] [1110.13][S03]got to new levels of realism.[1112.51] [1112.51][S03]And I remember looking at one of the--[1114.33] [1114.33][S03]I think it was the Imagen video models and just saying,[1117.81] [1117.81][S03]how is it possible that there is a full simulation of the world[1121.09] [1121.09][S03]just in this model?[1122.15] [1122.15][S03]It's just mind blowing, if you think about the level of realism[1125.57] [1125.57][S03]that those models even initially achieved compared to simulation[1130.05] [1130.05][S03]using 3D graphics methods.[1133.09] [1133.09][S03]And then with Veo, what we've tried to do[1136.17] [1136.17][S03]is basically to build the best possible video model.[1140.41] [1140.41][S03]And when I saw the results, I started thinking, OK,[1143.95] [1143.95][S03]what happens if-- can we do that in real time?[1146.85] [1146.85][S03]And I was obviously following the work by Jack and team,[1150.93] [1150.93][S03]and I think that was also very inspiring.[1153.27] [1153.27][S03]And we just said, OK, we have to go to the next level.[1156.83] [1156.83][S01] So what were the elements from Veo[1158.85] [1158.85][S01]that you wanted to combine or learn from for this project?[1163.358] [1163.358][S01]Because it's not just the visual aesthetic.[1165.15] [1165.15][S03] The quality and the realism[1166.983] [1166.983][S03]is very something that we really invested in.[1169.45] [1169.45][S03]And I think there is a level of realism[1171.25] [1171.25][S03]where I think we kind of got to it with Veo 2 that--[1176.9] [1176.9][S03]the physics are not perfect, but it's good enough[1179.82] [1179.82][S03]to start to be useful so we can actually[1181.98] [1181.98][S03]create some scenes that are indistinguishable[1184.54] [1184.54][S03]from real footage.[1187.12] [1187.12][S03]Not everything.[1188.0] [1188.0][S03]Not all the time.[1188.72] [1188.72][S03]But it's starting to be there, right?[1190.48] [1190.48][S03]With Veo 3, we also added audio and other stuff.[1193.52] [1193.52][S03]So taking it to the next step of basically making[1196.82] [1196.82][S03]this interactive was the obvious next step.[1199.84] [1199.84][S03]But technically it's quite challenging, especially[1203.78] [1203.78][S03]terms of how fast we should create the next frames.[1210.46] [1210.46][S03]That's what was one of the core challenges[1212.38] [1212.38][S03]for the project for Genie 3.[1215.06] [1215.06][S03]And I think the approach that we have overall[1217.9] [1217.9][S03]is to try and understand how those models train,[1221.32] [1221.32][S03]how they're being trained, how they learn.[1225.36] [1225.36][S03]And the same kind of principles that helped us scale and improve[1230.84] [1230.84][S03]Veo, we found them to be useful for Genie 3 as well.[1233.253] [1233.253][S01] It seems to me--[1234.42] [1234.42][S01]OK.[1235.3] [1235.3][S01]As an outsider, it seems to me that the objective[1238.58] [1238.58][S01]of Genie and Veo are, I mean, even though they[1241.5] [1241.5][S01]look visually quite similar, the objectives of them[1243.82] [1243.82][S01]are quite different.[1244.78] [1244.78][S01]Veo, you're trying to create this very realistic,[1247.32] [1247.32][S01]non-interactive environment, whereas with Genie you[1250.62] [1250.62][S01]need to make this consistent, explorable world that you[1255.02] [1255.02][S01]can move around in.[1256.4] [1256.4][S01]Do you have to start from scratch[1258.38] [1258.38][S01]or is there kind of swaps that you can make?[1261.378] [1261.378][S03] So I think in a way,[1262.92] [1262.92][S03]if you think about the video model, you can tell it,[1265.36] [1265.36][S03]OK, I want to walk around maybe a volcano[1269.18] [1269.18][S03]or I want to-- the camera to move that way or another.[1272.04] [1272.04][S03]So what it does is basically looking at this entire video[1275.18] [1275.18][S03]and tries to create a coherent eight second or whatever[1279.82] [1279.82][S03]long video, and it can change the past and the future[1282.7] [1282.7][S03]at the same time.[1283.48] [1283.48][S03]I think that's the property of video models[1285.8] [1285.8][S03]that is very different from Genie 3.[1287.387] [1287.387][S02] Because it spits out the end result.[1289.72] [1289.72][S03] Exactly.[1290.762] [1290.762][S03]And you can think of it like as a painting[1292.91] [1292.91][S03]that you can change all the time everything on the canvas.[1297.27] [1297.27][S03]And while it's much, in a way, easier[1300.95] [1300.95][S03]than doing it in what we call autoregressive[1303.55] [1303.55][S03]or just extending one frame at a time.[1306.57] [1306.57][S03]So I think that's the fundamental difference.[1308.65] [1308.65][S03]There's still a lot of similar aspects.[1312.237] [1312.237][S03]For example, the way that we have eventually[1314.07] [1314.07][S03]to take some text input and convert it[1315.95] [1315.95][S03]to some kind of visual output.[1318.05] [1318.05][S03]So that's somewhat similar.[1319.61] [1319.61][S03]So I would say definitely we don't start from scratch.[1323.91] [1323.91][S03]We build on top of, I think, a lot of work in this space,[1326.73] [1326.73][S03]but there are definitely some novel properties[1329.07] [1329.07][S03]that we had to figure out.[1330.153] [1330.153][S01] Well, that's super interesting, actually,[1332.362] [1332.362][S01]the idea that time is the key component in this,[1334.73] [1334.73][S01]that you have to understand the past[1336.23] [1336.23][S01]and march forward to the future.[1337.85] [1337.85][S01]That's where the autoregressive stuff comes in, right?[1340.15] [1340.15][S02] Exactly.[1340.65] [1340.65][S02]So essentially every frame you see[1342.67] [1342.67][S02]is generated from scratch at that point in time.[1346.71] [1346.71][S02]So things that happen later in the interaction[1349.83] [1349.83][S02]aren't known yet, and things that happen at the beginning[1352.99] [1352.99][S02]have to all be remembered by the model.[1355.67] [1355.67][S02]So essentially, if we're doing 24 frames per second,[1358.67] [1358.67][S02]it's like doing image generation 24 times per second,[1362.99] [1362.99][S02]each one completely generating from scratch,[1365.63] [1365.63][S02]given all of the past and the actions[1367.55] [1367.55][S02]of the agent or human player.[1370.057] [1370.057][S01] There's an analogy here, though.[1371.89] [1371.89][S01]I mean, that's sort of the way that language models work,[1374.31] [1374.31][S01]right?[1374.68] [1374.68][S03] Yes, exactly.[1375.93] [1375.93][S03]So I think it's really a good example.[1377.57] [1377.57][S03]So we know that language models are basically[1379.75] [1379.75][S03]being trained to predict the next token or word.[1382.13] [1382.13][S03]So they look at text and try to guess, OK,[1384.73] [1384.73][S03]what's the distribution of words or tokens that happen to follow?[1389.27] [1389.27][S03]And what we have with autoregressive word models,[1392.11] [1392.11][S03]we actually have a similar problem.[1394.37] [1394.37][S03]We want to basically predict the next observation, which[1398.87] [1398.87][S03]is pretty much visual, the next frame, given[1401.51] [1401.51][S03]what was already seen.[1403.77] [1403.77][S03]And the nice thing, I think, in this kind of parallel to LLMs[1408.27] [1408.27][S03]is that LLMs learn from that very simple task,[1411.64] [1411.64][S03]a very rich, potentially, representation of the world[1415.16] [1415.16][S03]or how people think, how people solve problems, for example.[1420.0] [1420.0][S03]I think the reason that word models,[1421.74] [1421.74][S03]especially autoregressive word models,[1423.38] [1423.38][S03]are kind of exciting for us is because maybe[1426.64] [1426.64][S03]through that task, that is pretty much[1428.68] [1428.68][S03]simple task that anyone can understand,[1431.5] [1431.5][S03]they have to learn the dynamics of the world.[1434.26] [1434.26][S03]And I think it's nice, if you really think about it,[1438.2] [1438.2][S03]it's kind of a superset of intelligence.[1440.96] [1440.96][S03]Because through the world, if you say,[1443.02] [1443.02][S03]OK, now I'm playing chess with some grandmaster[1445.28] [1445.28][S03]and it was the next visual or the next frame would actually[1449.64] [1449.64][S03]be their next maybe move.[1452.3] [1452.3][S03]So of course, our model is not capable of doing that,[1455.22] [1455.22][S03]but at the limit, it kind of goes very, very far.[1457.558] [1457.558][S01] But it's these ideas of understanding[1459.6] [1459.6][S01]the past, the context, and being able to predict the next move[1462.76] [1462.76][S01]in the future, which is--[1464.16] [1464.16][S03] Yes.[1464.56] [1464.56][S02] Yeah.[1464.78] [1464.78][S02]It's also quite powerful, because it[1466.28] [1466.28][S02]means you can start in the same location[1468.24] [1468.24][S02]and do many very different things.[1470.72] [1470.72][S02]So from an agent's perspective, there[1473.08] [1473.08][S02]could even be quite a simple task,[1474.74] [1474.74][S02]but you want to get really good at it.[1476.48] [1476.48][S02]And so you can simulate various different scenarios.[1479.86] [1479.86][S02]This is very analogous to reinforcement learning paradigm,[1482.36] [1482.36][S02]where you have this end reset function that brings you back[1485.92] [1485.92][S02]to the same state.[1486.773] [1486.773][S02]And then you want to get more experience from there.[1488.94] [1488.94][S01] Well, let's inch through this then.[1490.898] [1490.898][S01]So the very first frame.[1492.88] [1492.88][S01]You can have an image, like you did with the painting,[1496.64] [1496.64][S01]but you can also have a text input, right?[1498.61] [1498.61][S03] Yes, of course.[1499.5] [1499.5][S01] I mean, I know you asked me[1501.2] [1501.2][S01]earlier to describe somewhere I'd been,[1503.72] [1503.72][S01]and I got to go to a hunter's lodge in Siberia once.[1506.42] [1506.42][S01]So it was the most impressive thing I could come up with.[1509.928] [1509.928][S01]And this is what I sent to you.[1511.22] [1511.22][S01]I went to hunter's lodge, sat on a reindeer skin in the woods[1514.96] [1514.96][S01]outside of Yakutsk in Siberia, drank vodka,[1517.52] [1517.52][S01]and ate frozen calf liver.[1519.117] [1519.117][S03] Yes, so--[1520.2] [1520.2][S01] Tell me what you did with that.[1521.22] [1521.22][S03] So first, we were surprised with the prompt,[1523.762] [1523.762][S03]but then we just tried to--[1525.0] [1525.0][S03]we put it into the system, and the system is basically able[1528.81] [1528.81][S03]to add a few more details to the prompt that you provided.[1534.75] [1534.75][S03]But still it does follow the key elements that you provided.[1539.85] [1539.85][S03]So for example, I think it's reasonably--[1543.09] [1543.09][S03]I'm not sure if it's a reindeer, but it's definitely--[1546.97] [1546.97][S03]so, and then here you can see the table.[1549.477] [1549.477][S01] It's really amazing.[1550.81] [1550.81][S01]The light, again.[1551.91] [1551.91][S01]You guys are very--[1552.97] [1552.97][S01]you love making the light late afternoon,[1555.37] [1555.37][S01]a beautiful golden hour.[1557.05] [1557.05][S02] You also missed the calf liver.[1559.175] [1559.175][S02]And it's frozen, too.[1560.37] [1560.37][S02]Frozen calf liver.[1561.51] [1561.51][S02]It's not even just calf liver.[1563.65] [1563.65][S01] That first frame is generated in the same way[1566.29] [1566.29][S01]as you might find in Veo.[1568.253] [1568.253][S03] Yeah.[1569.17] [1569.17][S03]So the model doesn't treat it like in any special way.[1571.55] [1571.55][S03]It's just like you give it a text,[1572.967] [1572.967][S03]and it just starts outputting frames[1577.05] [1577.05][S03]and it doesn't do any preparation before it.[1579.51] [1579.51][S03]It just throws you into the world[1581.13] [1581.13][S03]and you can go wherever you want.[1582.63] [1582.63][S01] And then it's from that first frame[1583.93] [1583.93][S01]that the prediction backwards and forwards.[1585.67] [1585.67][S02] Exactly, yeah.[1586.23] [1586.23][S02]So it's like text to first frame,[1588.19] [1588.19][S02]then first action to next frame and so on[1590.77] [1590.77][S02]and so forth from that point onwards, basically.[1593.05] [1593.05][S01] And how critical is the exact wording of the prompt[1595.85] [1595.85][S01]here?[1596.35] [1596.35][S01]I mean, can you get better images, better worlds[1598.53] [1598.53][S01]with better prompts?[1599.79] [1599.79][S02] Yeah, I think that's definitely true.[1601.19] [1601.19][S02]There's an art to prompting, I think,[1603.09] [1603.09][S02]all of these modern models, and some people are better at it[1606.37] [1606.37][S02]than others.[1606.87] [1606.87][S02]And fortunately, we have some people[1608.37] [1608.37][S02]who are much better than me at this.[1610.47] [1610.47][S03] This one worked pretty much out of the box.[1612.97] [1612.97][S02] This one, yeah.[1614.428] [1614.428][S02]Often they work pretty well, especially when[1616.41] [1616.41][S02]you have very vivid description, like the table with the vodka[1619.93] [1619.93][S02]and the frozen calf's liver.[1622.17] [1622.17][S02]But sometimes you can try something and not quite capture[1627.85] [1627.85][S02]exactly what you wanted first time,[1629.35] [1629.35][S02]and then you can iterate a little bit on the prompt[1631.01] [1631.01][S02]and then get something that's much more like what you wanted.[1633.35] [1633.35][S01] And does that require[1634.725] [1634.725][S01]you to regenerate the world?[1636.59] [1636.59][S01]Or given that it's marching forwards,[1638.51] [1638.51][S01]can you add things in on the fly?[1640.07] [1640.07][S03] So we do have ways[1641.528] [1641.528][S03]to add things on the fly, what we call promptable world events.[1644.94] [1644.94][S03]And this is just something that you can say, OK,[1647.36] [1647.36][S03]now I want, for example, a balloon to fly in[1651.82] [1651.82][S03]or some other character to show up.[1653.883] [1653.883][S03]This is something we're very excited about because it also[1656.3] [1656.3][S03]allows the environment to be more interesting for people,[1659.52] [1659.52][S03]but also, as we mentioned, more relevant for training[1663.82] [1663.82][S03]agents in simulation, because then we[1666.34] [1666.34][S03]can throw in something that happens in the world[1668.98] [1668.98][S03]that they have to adapt to.[1670.718] [1670.718][S01] So in that example, maybe[1672.26] [1672.26][S01]have a reindeer coming through.[1673.66] [1673.66][S03] Right.[1674.618] [1674.618][S03]We can have a reindeer, or we can[1676.14] [1676.14][S03]have another person is walking into the scene[1679.38] [1679.38][S03]and then we have to-- if it's an agent,[1682.1] [1682.1][S03]basically, it can respond to it, and if it's just[1685.18] [1685.18][S03]for entertainment purposes, then it's[1687.137] [1687.137][S03]much more interesting than just walking in a world[1689.22] [1689.22][S03]that nothing happens.[1690.095] [1690.095][S01] And this is all a key advantage[1691.887] [1691.887][S01]of the autoregressive part, I guess,[1693.46] [1693.46][S01]that you do have control over the future.[1695.52] [1695.52][S02] Exactly.[1695.98] [1695.98][S02]Yeah.[1696.48] [1696.48][S02]So you can just inject things on the fly[1699.38] [1699.38][S02]as you're generating things in real time.[1701.203] [1701.203][S01] The thing that I find extraordinary about this[1703.62] [1703.62][S01]is the consistency.[1704.94] [1704.94][S01]Right?[1705.44] [1705.44][S01]The memory of the system.[1707.42] [1707.42][S01]You turn away and then you turn back,[1709.04] [1709.04][S01]and it's exactly how you left it.[1710.76] [1710.76][S01]But presumably, if you're turning in a direction[1713.0] [1713.0][S01]you haven't yet turned in, then it's[1714.5] [1714.5][S01]sort of a stochastic process.[1716.58] [1716.58][S01]So how do you balance it, that it's sometimes a memory[1720.74] [1720.74][S01]and sometimes generated statistically?[1723.862] [1723.862][S02] Well, it's kind of a mixture of things.[1726.32] [1726.32][S02]So you can, in the text prompt, specify[1728.482] [1728.482][S02]things that are out of sight, which[1729.94] [1729.94][S02]I think is quite powerful compared to image prompting.[1732.96] [1732.96][S02]Because you can say, on the right is x, y, z,[1736.86] [1736.86][S02]and then when you actually look round to the right,[1739.04] [1739.04][S02]it's there when you actually play or interact in the world.[1742.885] [1742.885][S02]But then there is also an element[1744.26] [1744.26][S02]of the model using its world knowledge to generate things.[1746.68] [1746.68][S02]So in the Hopper example with the artwork,[1750.38] [1750.38][S02]it does generate the street that we haven't seen before.[1753.978] [1753.978][S02]And you can't be exactly certain what it's going to generate.[1756.52] [1756.52][S02]The model uses its own sort of intuition[1758.94] [1758.94][S02]of what should be there.[1760.54] [1760.54][S01] And that's intuition based on, I mean,[1762.73] [1762.73][S01]having watched, I mean, incredible amounts[1765.39] [1765.39][S01]of video footage in advance.[1767.093] [1767.093][S03] Yeah.[1768.01] [1768.01][S03]So basically the model is trying to just generate[1771.67] [1771.67][S03]a sequence of frames that is representative of the world.[1775.31] [1775.31][S03]So if it already saw some part or generated[1778.35] [1778.35][S03]some part of the world, then the right thing for the model to do[1781.15] [1781.15][S03]would be to make--[1782.47] [1782.47][S03]to recall this memory and use it.[1784.99] [1784.99][S03]So if I look back to where I've already been,[1787.99] [1787.99][S03]then the right thing for the model[1789.79] [1789.79][S03]would just to use the same thing.[1791.41] [1791.41][S03]But when you look to a new area from the model's perspective,[1794.17] [1794.17][S03]again, it can allow itself to generate something new[1796.69] [1796.69][S03]because it hasn't been seen.[1798.17] [1798.17][S03]So the model doesn't really treat it[1799.67] [1799.67][S03]in a fundamentally different way.[1802.49] [1802.49][S03]So the model, it learns to basically balance[1806.59] [1806.59][S03]the two aspects.[1808.17] [1808.17][S03]And again, all comes back to anchoring all of the generation[1812.55] [1812.55][S03]to the prompt or what the user provided.[1815.273] [1815.273][S03]That's where the information to the generation comes from.[1817.69] [1817.69][S01] But then I guess going back[1818.39] [1818.39][S01]to the language analogy that we were talking about earlier.[1820.89] [1820.89][S01]It's not that surprising that once[1822.445] [1822.445][S01]a statement has been established in a conversation,[1824.57] [1824.57][S01]that it remains consistent when you refer back to it.[1826.61] [1826.61][S02] Exactly, yeah.[1828.027] [1828.027][S02]And we've seen with language models, this ability[1830.47] [1830.47][S02]to have memory and consistency has been something that's[1834.51] [1834.51][S02]improved a lot recently, especially with the latest[1837.03] [1837.03][S02]Gemini models now versus two years ago.[1838.973] [1838.973][S03] Yeah.[1839.89] [1839.89][S03]I think what's interesting here is that while,[1842.11] [1842.11][S03]if you think about the number of the size of the memory[1846.31] [1846.31][S03]or the level of detail.[1847.61] [1847.61][S03]So if we think about I'm talking to Gemini,[1852.27] [1852.27][S03]and after a few sentences, it might refer to something[1854.99] [1854.99][S03]I've said before.[1855.97] [1855.97][S03]That's great.[1856.81] [1856.81][S03]But the number of details that we have in those visual words[1861.83] [1861.83][S03]is just like, it's staggering, right?[1863.49] [1863.49][S03]The quality of the memory, considering how much detail[1867.63] [1867.63][S03]and how much information it has to actually remember.[1870.61] [1870.61][S03]Yeah.[1871.11] [1871.11][S01] Well, how do you do that?[1872.17] [1872.17][S01]Does this require you having a sort of 3D representation[1874.91] [1874.91][S01]of the world that you're in?[1876.482] [1876.482][S02] We don't use anything[1878.19] [1878.19][S02]like that for this version of the model.[1880.56] [1880.56][S02]It's largely just learnt as an emergent property just from this[1884.817] [1884.817][S02]autoregressive prediction.[1885.9] [1885.9][S01] You guys with your emergent properties.[1887.34] [1887.34][S01]My goodness me.[1888.2] [1888.2][S02] Yeah.[1889.242] [1889.242][S03] We are very good students of the Beatles.[1891.68] [1891.68][S02] Yeah, I think it's similar to,[1893.68] [1893.68][S02]if you're predicting the next frame,[1895.18] [1895.18][S02]you just have to learn to remember these critical things[1897.52] [1897.52][S02]in the past.[1898.68] [1898.68][S02]And obviously the model has some representations[1900.68] [1900.68][S02]that prioritize the important details,[1903.4] [1903.4][S02]but it really is just predicting the next frame.[1905.46] [1905.46][S01] And is this analogous, then, to the fact[1907.627] [1907.627][S01]that language models have this conceptual understanding[1910.2] [1910.2][S01]and can describe similar things in--[1913.36] [1913.36][S01]the same things in different ways?[1914.943] [1914.943][S01]Is it the transformer architecture[1916.36] [1916.36][S01]here that's allowing you to do that?[1917.86] [1917.86][S03] Yes.[1918.735] [1918.735][S03]So the architecture is--[1919.76] [1919.76][S03]I think almost everything today is a transformer.[1922.34] [1922.34][S03]So yes, this is also a transformer.[1925.56] [1925.56][S03]And looking back, basically the model, we[1928.52] [1928.52][S03]will see what was already generated,[1931.68] [1931.68][S03]including in the user provided inputs, and based on that,[1935.32] [1935.32][S03]makes the prediction.[1936.54] [1936.54][S03]So I think the interesting question about explicit 3D is[1941.28] [1941.28][S03]that the model probably had to learn some representation,[1945.46] [1945.46][S03]but it's just not an explicit representation.[1948.3] [1948.3][S03]So we see that the ability of the model to understand 3D[1952.04] [1952.04][S03]environments is very strong.[1953.96] [1953.96][S03]And I think the most, to me, an emergent, if you want,[1958.2] [1958.2][S03]capability, is that it actually works[1959.96] [1959.96][S03]where it was not trained for.[1961.26] [1961.26][S03]For example, in taking a painting,[1964.14] [1964.14][S03]an oil painting in 1942 and actually making it into some[1968.84] [1968.84][S03]kind of a 3D environment.[1969.96] [1969.96][S03]That's pretty much out of its distribution.[1972.86] [1972.86][S01] Absolutely.[1973.88] [1973.88][S01]I want to go back to this idea of understanding physics.[1977.96] [1977.96][S01]Jack, if you've got this world that you can put an agent in,[1982.28] [1982.28][S01]does that mean that you can test how well it knows physics?[1985.46] [1985.46][S01]I mean, could you get a hammer and a feather, for instance,[1988.16] [1988.16][S01]and allow it to drop it at the same time?[1990.018] [1990.018][S02] You definitely could do that.[1992.06] [1992.06][S02]I think that would probably be close to the frontier[1995.0] [1995.0][S02]of the model's capabilities at this point.[1997.77] [1997.77][S02]We've seen it's quite good at visual things and things that[2001.09] [2001.09][S02]are more general concepts.[2002.31] [2002.31][S02]So you can imagine water occurs in quite[2005.09] [2005.09][S02]a lot of different scenarios it's seen before.[2008.21] [2008.21][S02]Gravity has probably occurred in many of those,[2010.87] [2010.87][S02]but probably not these exact objects.[2013.47] [2013.47][S02]But I think if you were to specify in the text prompt,[2016.11] [2016.11][S02]like in the world, there is a feather that[2018.81] [2018.81][S02]has less gravity and a hammer that is heavier,[2022.19] [2022.19][S02]maybe then it would work.[2023.51] [2023.51][S03] Yeah.[2024.427] [2024.427][S03]I think there are a lot of--[2025.65] [2025.65][S03]those models are inherently visual.[2027.943] [2027.943][S03]They don't know anything about the world[2029.61] [2029.61][S03]except for what we can see.[2032.01] [2032.01][S03]And that's a limitation, I would say.[2033.95] [2033.95][S03]Even for video models, some things[2036.53] [2036.53][S03]sometimes don't make sense.[2038.05] [2038.05][S03]And the model has to guess if something is heavy, for example.[2041.63] [2041.63][S03]There is no real way for a model to look at an image[2044.81] [2044.81][S03]and say, OK, that's how much this thing weigh.[2047.95] [2047.95][S03]So it kind of makes up the weight[2050.17] [2050.17][S03]and then try to simulate what would have happened.[2052.79] [2052.79][S03]And sometimes it breaks.[2054.69] [2054.69][S03]And we know that from video models, even the best video[2057.61] [2057.61][S03]models.[2058.31] [2058.31][S03]And I think what we have is basically we[2060.969] [2060.969][S03]solve an even harder problem because we have to, as we said,[2064.75] [2064.75][S03]not just-- we can't fix the past.[2066.77] [2066.77][S03]We have to go--[2067.732] [2067.732][S03]once the model generates something,[2069.19] [2069.19][S03]it has to roll with it.[2071.17] [2071.17][S03]And I think we have really good progress in terms of simulation,[2076.01] [2076.01][S03]like fluid dynamics.[2077.969] [2077.969][S03]But maybe some other aspects of the world physically[2081.25] [2081.25][S03]would not be accurate because of those limitations.[2083.78] [2083.78][S01] Well, tell me some of the work[2085.53] [2085.53][S01]that you've been doing with agents so far, because actually[2087.988] [2087.988][S01]on a previous episode, we got to talk about SIMA, the Scalable[2091.81] [2091.81][S01]Instructable Multiworld Agent.[2093.802] [2093.802][S02] Exactly.[2094.969] [2094.969][S01] Which they were putting into existing computer[2097.53] [2097.53][S01]game environments.[2098.77] [2098.77][S01]But I mean, you can now take SIMA[2102.01] [2102.01][S01]and put it into these generated environments too.[2104.47] [2104.47][S02] Exactly.[2105.637] [2105.637][S02]So the cool thing with this is that we've got--[2107.65] [2107.65][S02]at Google DeepMind, we have these agents[2109.45] [2109.45][S02]that are trained to be general.[2111.41] [2111.41][S02]The M is multiworld, as you just said.[2114.82] [2114.82][S02]And so what we're able to do is take all the worlds we generate[2117.58] [2117.58][S02]and then test whether these agents and the latest[2119.7] [2119.7][S02]versions of them are able to already use them[2122.42] [2122.42][S02]for agent training or collecting experiences or evaluations[2126.62] [2126.62][S02]as they are.[2127.42] [2127.42][S02]So you can say to it something like, navigate to the robot[2131.94] [2131.94][S02]over there, and then you can give it an image from the world[2136.44] [2136.44][S02]and it can take the first action.[2138.16] [2138.16][S02]And then from that point onwards,[2139.56] [2139.56][S02]it's interacting in the world through actions.[2142.1] [2142.1][S02]But the Genie 3 model does not know what the goal is.[2145.46] [2145.46][S02]So that makes it like a genuine simulation,[2148.56] [2148.56][S02]rather than if you were to tell Genie 3 that the SIMA agent is[2151.7] [2151.7][S02]trying to achieve this goal.[2153.46] [2153.46][S02]It might make the experience not authentic,[2155.58] [2155.58][S02]because it might make it happen in an incorrect way.[2157.888] [2157.888][S01] For it.[2158.68] [2158.68][S02] Yeah, exactly.[2159.52] [2159.52][S01] I guess if you've got a robot in the real world,[2161.32] [2161.32][S01]the world isn't helping the robot.[2163.19] [2163.19][S02] Exactly.[2163.28] [2163.28][S03] Exactly.[2164.322] [2164.322][S03]You can't say, OK, the robot has to go to fetch the red cube,[2167.09] [2167.09][S03]and then it looks to the left and there is a red cube.[2169.34] [2169.34][S03]It's a bit making it.[2170.215] [2170.215][S02] Yeah, and this[2171.632] [2171.632][S02]is a problem that we've encountered in other scenarios,[2174.16] [2174.16][S02]but you don't really get this problem[2175.74] [2175.74][S02]if you have this separation between the agent[2177.5] [2177.5][S02]and the environment.[2178.38] [2178.38][S02]And that's the really nice thing of working with agent teams that[2181.62] [2181.62][S02]are focused on building really capable agents,[2183.56] [2183.56][S02]but then having them access our environment as[2185.82] [2185.82][S02]if it's any other environment.[2187.38] [2187.38][S02]So the new worlds that are created by Genie 3[2190.5] [2190.5][S02]look the same as the existing worlds[2192.78] [2192.78][S02]that the SIMA agent was trained on.[2194.993] [2194.993][S01] But then what if there isn't a red cube?[2197.16] [2197.16][S01]What if it searches around forever[2198.66] [2198.66][S01]and there is no red cube?[2199.802] [2199.802][S03] So yeah, I think that's part of the things[2202.26] [2202.26][S03]that we're looking into is how we[2203.9] [2203.9][S03]can add more details as we go.[2206.76] [2206.76][S03]For example, if you put the agent in some room and then[2210.46] [2210.46][S03]it has to open maybe a drawer and find something in there.[2213.28] [2213.28][S03]So we want to be able to inject kind of events into the world[2220.9] [2220.9][S03]and control it.[2221.72] [2221.72][S03]So I think the interesting frontier here[2225.58] [2225.58][S03]is how you can make the world look very realistic[2229.62] [2229.62][S03]and also control what's happening[2232.31] [2232.31][S03]in the world in a way that still makes sense.[2234.847] [2234.847][S03]So I think what we've seen is that with our prompt about world[2237.43] [2237.43][S03]events, we can add things that happen in the world.[2240.29] [2240.29][S03]But if you just want something to pop into the world,[2242.81] [2242.81][S03]then it's not necessarily a plausible thing.[2245.41] [2245.41][S03]If you look at--[2246.367] [2246.367][S03]I don't know, if you're in the desert,[2247.95] [2247.95][S03]and then you're going to ask it, OK, now I[2249.7] [2249.7][S03]want to see an elephant, then where is this elephant[2253.098] [2253.098][S03]going to come from?[2253.89] [2253.89][S03]So maybe it's going to come from the side[2255.07] [2255.07][S03]or when you look to the left.[2256.29] [2256.29][S03]So I think there is something really interesting in what[2258.75] [2258.75][S03]does it mean to change a world, because the model[2262.59] [2262.59][S03]has some assumptions.[2263.65] [2263.65][S03]And when it comes to agents, this[2265.43] [2265.43][S03]is definitely an important capability[2267.91] [2267.91][S03]to be able to inject this new event into the world.[2270.61] [2270.61][S01] So you've done this already.[2272.51] [2272.51][S02] Yeah, we've got signs of life for this.[2274.968] [2274.968][S02]I think we haven't really say we've[2276.67] [2276.67][S02]got a full agent training loop where[2278.75] [2278.75][S02]we're already doing some large scale[2280.878] [2280.878][S02]training in these environments.[2282.17] [2282.17][S02]But what we can already do is test the agents in them[2284.83] [2284.83][S02]and see how they do.[2286.01] [2286.01][S02]And I think it's quite remarkable[2287.47] [2287.47][S02]that, given these things weren't developed together,[2290.65] [2290.65][S02]we just dropped the agent in and it can already do things.[2293.657] [2293.657][S02]And you can imagine all the different things[2295.49] [2295.49][S02]you could now use this for.[2296.71] [2296.71][S01] Color that in for me.[2297.57] [2297.57][S01]Give me an example.[2298.53] [2298.53][S03] If you have a factory[2300.59] [2300.59][S03]where you have some robots and you[2302.15] [2302.15][S03]want to introduce maybe a new--[2304.19] [2304.19][S03]maybe a very boring example, but a new machine, right?[2307.57] [2307.57][S03]And that wasn't there.[2308.65] [2308.65][S03]Or you changed somehow the structure of the building[2312.09] [2312.09][S03]and you want to test the robot before you actually[2314.482] [2314.482][S03]put them in the new building.[2315.69] [2315.69][S03]So again, this is like you can simulate the world that[2319.67] [2319.67][S03]basically is a variant of what maybe the agent has[2323.87] [2323.87][S03]seen in the past and see if it breaks.[2325.67] [2325.67][S03]Right?[2326.47] [2326.47][S03]Everything can happen in simulated environments[2328.99] [2328.99][S03]and not necessarily break your new machine.[2330.85] [2330.85][S03]So that's one example, I would say.[2333.265] [2333.265][S01] Find the unintended consequences.[2335.14] [2335.14][S03] Yeah, yeah.[2335.91] [2335.91][S03]And the evaluation of the model.[2337.49] [2337.49][S03]So that's even not training the agent,[2339.95] [2339.95][S03]just testing how well it adapts, maybe,[2341.95] [2341.95][S03]to a new variation of an environment.[2344.655] [2344.655][S01] All of these examples[2346.03] [2346.03][S01]you've given so far are where the agent has a specified[2349.04] [2349.04][S01]objective, which I know is the point of SIMA thus far.[2352.96] [2352.96][S01]But what about if you had agents that didn't have an objective?[2357.018] [2357.018][S01]There was a really nice quote that I came across,[2359.06] [2359.06][S01]which is that almost no prerequisite[2360.8] [2360.8][S01]to any major invention was made with that invention in mind.[2364.96] [2364.96][S01]Can you imagine a point in the future[2367.68] [2367.68][S01]where you are letting agents loose in these environments[2370.96] [2370.96][S01]without specifying an objective for them?[2373.203] [2373.203][S02] Yeah, exactly.[2374.62] [2374.62][S02]So the quote comes from "Why Greatness Cannot Be Planned,"[2377.74] [2377.74][S02]which is from Ken Stanley and Joel Lehman.[2379.9] [2379.9][S02]It's a great book.[2381.48] [2381.48][S02]And the general idea there is that searching[2384.0] [2384.0][S02]for interestingness might actually[2386.12] [2386.12][S02]lead to things that are more useful for practical goals[2389.383] [2389.383][S02]than if you just directly optimize for the practical goals[2391.8] [2391.8][S02]themselves.[2392.73] [2392.73][S02]And clearly, the bigger the domain[2395.8] [2395.8][S02]and the space for discovery, the more interesting things[2398.6] [2398.6][S02]could happen.[2399.74] [2399.74][S02]So they had this really nice example[2402.4] [2402.4][S02]quite a while ago with this paper called Picbreeder, where[2405.44] [2405.44][S02]essentially they allowed people to select images and combine[2410.08] [2410.08][S02]them to create new images that were mutations of those two.[2413.56] [2413.56][S02]People weren't directly optimizing for specific end[2416.24] [2416.24][S02]goals, but by just choosing what they found interesting,[2419.8] [2419.8][S02]they ended up discovering really cool, structured pictures,[2423.7] [2423.7][S02]like a skull or a butterfly, that weren't obvious[2426.56] [2426.56][S02]how you would reach that from the starting points.[2429.12] [2429.12][S02]And some of the stepping stones along the way[2431.2] [2431.2][S02]didn't really look much like the final goal,[2433.04] [2433.04][S02]and they wouldn't have been things you obviously[2435.04] [2435.04][S02]would have chosen to go for if you had those goals in mind.[2438.32] [2438.32][S02]And there are lots of examples of this in the real world.[2442.06] [2442.06][S02]If you are, for example, trying to reach the moon,[2444.66] [2444.66][S02]you wouldn't build a bigger ladder.[2446.28] [2446.28][S02]So optimizing along one dimension with maybe[2450.0] [2450.0][S02]a greedy, myopic approach doesn't always[2452.48] [2452.48][S02]lead you to make these big leaps.[2453.98] [2453.98][S01] Well, I mean, evolution itself[2455.73] [2455.73][S01]is like the classic example of iteration without objective.[2460.52] [2460.52][S02] Yeah, and we see this a lot in research.[2463.02] [2463.02][S03] Yeah.[2463.937] [2463.937][S03]My perspective on that is that I think we as humans,[2467.55] [2467.55][S03]we decide what's interesting.[2469.27] [2469.27][S03]I think there is even an example where[2471.05] [2471.05][S03]the entire evolution of mathematics[2473.39] [2473.39][S03]was guided by people deciding what's next, what's interesting,[2476.83] [2476.83][S03]what's not interesting, and just the problem being hard[2480.21] [2480.21][S03]doesn't mean that it's interesting at all.[2482.762] [2482.762][S03]And I think in a way, when we think[2484.89] [2484.89][S03]about generating new things in science,[2487.71] [2487.71][S03]it's like the goal or the-- it doesn't have to be a goal,[2491.31] [2491.31][S03]but there is some aspects of maybe beauty or interest that[2495.45] [2495.45][S03]is coming from us, and models might maybe[2498.89] [2498.89][S03]learn to simulate that.[2500.315] [2500.315][S03]But I think it's really important[2501.69] [2501.69][S03]to remember that we ultimately decide[2504.09] [2504.09][S03]what's interesting as our preferences as people.[2508.75] [2508.75][S01] I mean, you haven't had the move 37, as it were,[2511.25] [2511.25][S01]in this method.[2512.25] [2512.25][S03] Even in this case, the goal of Go or the game[2517.73] [2517.73][S03]was designed in a certain way that people[2519.93] [2519.93][S03]find it interesting to play, otherwise--[2522.32] [2522.32][S03]and it's a very ancient game.[2524.03] [2524.03][S03]So I'm just saying that the setup, even when there is--[2528.13] [2528.13][S03]or getting to the moon, this goal[2530.17] [2530.17][S03]was kind of maybe made by people.[2532.99] [2532.99][S03]So I'm just saying, I think there is still[2535.37] [2535.37][S03]broader constraints set by us, in a way.[2540.81] [2540.81][S03]If a machine comes up with a problem to solve,[2545.062] [2545.062][S03]we still have to say, is this an interesting problem?[2547.27] [2547.27][S03]Because otherwise we're just like, OK,[2548.59] [2548.59][S03]I don't care about that.[2549.77] [2549.77][S03]So I think there is still an aesthetic part to anything[2552.29] [2552.29][S03]that's open-ended to me.[2554.15] [2554.15][S02] Yeah, I think[2554.49] [2554.49][S02]there was this quote from Dennis a while ago about the levels[2557.17] [2557.17][S02]of creativity, and it was like, interpolation is one.[2559.97] [2559.97][S02]Like, you see a new cat and you can identify it as a cat.[2562.61] [2562.61][S02]Extrapolation was one where it's like, given the rules of Go,[2565.59] [2565.59][S02]can you discover a new move like move 37?[2567.93] [2567.93][S02]And then the third level is generating completely[2570.41] [2570.41][S02]new things.[2571.11] [2571.11][S02]So, could you actually invent Go, is what he said.[2573.43] [2573.43][S02]And we actually had this as a motivation for the Genie project[2576.15] [2576.15][S02]at the beginning.[2576.858] [2576.858][S02]It was like, can you create completely new things?[2580.29] [2580.29][S02]And I think we're starting to see that happen.[2582.49] [2582.49][S02]Since it's a completely new kind of model, someone on the team[2586.06] [2586.06][S02]will do something like create a certain kind of world,[2589.227] [2589.227][S02]and then immediately other members of the team are like,[2591.56] [2591.56][S02]that's really interesting.[2593.02] [2593.02][S02]And then they start evolving that idea themselves.[2596.58] [2596.58][S02]And then we post it on social media[2598.96] [2598.96][S02]and we see the reaction to some things,[2600.62] [2600.62][S02]and then we know that's interesting.[2602.12] [2602.12][S02]So then we create new things that way.[2604.08] [2604.08][S02]And that's just with a very limited access to the model.[2607.48] [2607.48][S02]So clearly you can see if we open this up a bit more[2610.74] [2610.74][S02]in the future, that it could lead[2612.66] [2612.66][S02]to some sort of open-ended creativity that way.[2615.382] [2615.382][S03] It's kind of like but still the evolution.[2617.84] [2617.84][S03]It's like an evolution with the criteria[2619.42] [2619.42][S03]that people find interesting.[2620.36] [2620.36][S02] Yeah, exactly.[2621.777] [2621.777][S02]There's people in the loop guiding the interestingness.[2624.5] [2624.5][S01] Well, then, OK, could you take--[2625.66] [2625.66][S01]I'm just thinking back to the conversation[2626.98] [2626.98][S01]that we had with Dave Silver earlier in the series[2629.063] [2629.063][S01]where he was saying, actually, there's[2631.34] [2631.34][S01]one way you remove humans from the equation[2633.34] [2633.34][S01]and actually you get even more surprising results, potentially.[2637.74] [2637.74][S01]OK, just play along with me here for a moment.[2640.3] [2640.3][S01]But could you get to a point where you just[2642.58] [2642.58][S01]simulate the first, I don't know, single-celled organism[2645.62] [2645.62][S01]and allow it to evolve inside of Genie[2649.34] [2649.34][S01]and actually watch the process of evolution[2651.62] [2651.62][S01]happen in a virtual environment?[2654.387] [2654.387][S02] That's a great question.[2656.22] [2656.22][S02]And that's the dream of an A life, open-ended evolution[2659.58] [2659.58][S02]community.[2661.62] [2661.62][S02]I think maybe the worlds that we create[2664.18] [2664.18][S02]are not fully rich enough, but I definitely[2666.46] [2666.46][S02]think that we're getting along that path.[2669.16] [2669.16][S02]So open-ended evolution and A life,[2671.123] [2671.123][S02]they've been designing worlds that[2672.54] [2672.54][S02]could facilitate this kind of thing, typically in code.[2675.86] [2675.86][S02]And so this could be an alternative approach[2679.18] [2679.18][S02]to getting maybe richer real world simulations.[2683.34] [2683.34][S02]And so in theory, I mean, we've made quite a lot of progress[2686.98] [2686.98][S02]pretty fast.[2687.92] [2687.92][S02]But if you've got the simulation to be fully like the real world,[2690.96] [2690.96][S02]and it had the kind of objectives and constraints[2693.3] [2693.3][S02]that lead to these kind of evolutionary steps,[2696.78] [2696.78][S02]then it's definitely plausible.[2699.04] [2699.04][S02]But I can't say it's definitely there yet.[2701.38] [2701.38][S03] It's not a direct answer,[2703.13] [2703.13][S03]but I actually tried.[2705.39] [2705.39][S03]I think there is maybe a very basic example of Game of Life.[2711.327] [2711.327][S01] John Conway's.[2712.41] [2712.41][S03] Yeah.[2713.327] [2713.327][S03]It has four rules, and then I actually[2715.19] [2715.19][S03]tried using Veo to simulate it.[2717.97] [2717.97][S03]You give it an image and it doesn't work.[2720.71] [2720.71][S03]Like, it does look like it's evolving.[2723.35] [2723.35][S03]And if you don't know the rules, you[2725.07] [2725.07][S03]would look like, yeah, it looks reasonable.[2727.323] [2727.323][S03]Different pixels light up, but it doesn't follow[2731.83] [2731.83][S03]the four rules of the game.[2733.89] [2733.89][S03]I think this is a good example for what our current models are[2738.47] [2738.47][S03]able to do, and what they are less-- still[2741.27] [2741.27][S03]limited in their ability to follow specific rules.[2744.35] [2744.35][S03]To actually evolve maybe life forms,[2747.01] [2747.01][S03]I think you need much more ability to do both,[2750.63] [2750.63][S03]to also simulate the physical world,[2752.41] [2752.41][S03]but also follow some basic rules of physics[2756.43] [2756.43][S03]in a very accurate way.[2757.803] [2757.803][S01] Constrained way.[2758.97] [2758.97][S03] Yes, and constrained way.[2759.95] [2759.95][S03]And I think we're not-- like, we see some glimpse of it,[2762.29] [2762.29][S03]but it's definitely very far from being able to have[2765.67] [2765.67][S03]evolution on a GPU.[2767.092] [2767.092][S01] Well, thank you for going philosophical with me[2769.55] [2769.55][S01]for a moment there.[2770.85] [2770.85][S01]I enjoyed that.[2771.77] [2771.77][S01]Let's come back down to Earth with a thump,[2773.97] [2773.97][S01]though, because, I mean, there are safety implications[2776.79] [2776.79][S01]with this.[2777.29] [2777.29][S01]What are your main concerns?[2778.572] [2778.572][S02] I think there's[2780.03] [2780.03][S02]different levels of concern.[2781.61] [2781.61][S02]There's the known things, and they're quite obvious.[2786.567] [2786.567][S02]I mean, things like violence, maybe we[2788.15] [2788.15][S02]wouldn't want to occur in the world in new ways.[2792.95] [2792.95][S02]And that's something that we can already start addressing.[2796.51] [2796.51][S02]But there's also maybe some more gray areas[2799.63] [2799.63][S02]where we're not actually really sure how[2802.582] [2802.582][S02]we feel about these things, like historical settings,[2804.79] [2804.79][S02]for instance.[2806.05] [2806.05][S02]Some of them may be unsavory for subtle reasons,[2810.35] [2810.35][S02]and those are just things that we as a team[2812.59] [2812.59][S02]can see quite clearly.[2814.595] [2814.595][S02]But I think there's probably also some things[2816.47] [2816.47][S02]that we haven't considered.[2817.73] [2817.73][S02]And we'd rather get those things right[2820.12] [2820.12][S02]by limiting our early access and getting feedback,[2823.857] [2823.857][S02]which we're already doing.[2824.94] [2824.94][S02]And we've already learned a lot from the folks[2826.52] [2826.52][S02]that we brought in a few weeks ago and the folks[2828.56] [2828.56][S02]that we're still interacting with.[2830.54] [2830.54][S01] Like what?[2831.72] [2831.72][S02] We found a lot of new use cases[2833.36] [2833.36][S02]that I didn't think of.[2834.46] [2834.46][S02]Vocational training could actually be quite impactful.[2837.44] [2837.44][S02]Lots of people can't go into the role[2839.8] [2839.8][S02]of things like firefighting, for instance.[2842.583] [2842.583][S02]What does it feel like to actually[2844.0] [2844.0][S02]be there without having a visceral experience?[2848.06] [2848.06][S02]It's probably something that you would benefit a lot from being[2850.92] [2850.92][S02]able to simulate in advance.[2852.34] [2852.34][S02]Even if it's not perfectly correct from a simulation[2854.56] [2854.56][S02]perspective, there may be elements to it[2857.12] [2857.12][S02]that just getting a sense of what[2859.04] [2859.04][S02]it's like to be situated in that specific circumstance,[2862.588] [2862.588][S02]that it's quite nice to be able to simulate in advance.[2864.88] [2864.88][S03] Minus the heat.[2865.98] [2865.98][S02] Yeah.[2867.022] [2867.022][S02]Minus the heat and the smoke.[2868.68] [2868.68][S01] And the genuine jeopardy.[2870.26] [2870.26][S02] Yes.[2871.28] [2871.28][S01] But actually, you raised another interesting point[2873.863] [2873.863][S01]there though, because you said even if it's not[2876.04] [2876.04][S01]perfectly realistic.[2877.76] [2877.76][S01]Is there also another danger about[2879.52] [2879.52][S01]this gap between what's simulated and what's real?[2883.04] [2883.04][S01]How do you make that as small as possible?[2885.22] [2885.22][S01]This is something-- the sim to real gap[2886.36] [2886.36][S01]is something we spoke about in this podcast lots of times[2888.24] [2888.24][S01]before.[2888.86] [2888.86][S01]Let's say you've got your example[2890.235] [2890.235][S01]where you've got a robot in a factory and it's moving around.[2893.052] [2893.052][S03] In terms of reliability.[2894.76] [2894.76][S01] It's interacting.[2895.66] [2895.66][S01]You can't directly take that and map it into the real world,[2898.22] [2898.22][S01]right?[2899.12] [2899.12][S03] Yeah.[2900.037] [2900.037][S03]So I think over time, we'll see more control,[2902.96] [2902.96][S03]basically being able to take maybe a real environment,[2906.04] [2906.04][S03]map it into the model so the model can base its generation[2912.0] [2912.0][S03]on the real environment.[2913.18] [2913.18][S03]And we see it to an extent, whether we're[2915.36] [2915.36][S03]starting from an image or starting from a video.[2918.2] [2918.2][S03]And now the question is, would it be perfectly the same[2922.0] [2922.0][S03]as the real world?[2922.82] [2922.82][S03]Probably not.[2923.54] [2923.54][S03]I don't think it's even well defined what does it mean.[2926.0] [2926.0][S03]But I think the gap is definitely narrowing.[2929.3] [2929.3][S03]So we can take environments.[2930.72] [2930.72][S03]In the past, we know a lot of the RL environments[2933.56] [2933.56][S03]look very far from anything that's[2936.17] [2936.17][S03]photorealistic or real world, but now we can go even closer.[2940.31] [2940.31][S03]But definitely there still remains a gap,[2942.27] [2942.27][S03]and we will have to see what are the implications.[2945.03] [2945.03][S03]And we're still not using it for any real world[2947.33] [2947.33][S03]deployment, of course.[2948.51] [2948.51][S03]Yeah.[2949.09] [2949.09][S02] Yeah, I think it's[2949.81] [2949.81][S02]kind of an iterative approach.[2951.06] [2951.06][S02]I don't think we're saying, right, now, we've got Genie 3,[2953.59] [2953.59][S02]we've solved simulation for any possible embodied task.[2956.69] [2956.69][S02]But I think what we can do is combine it[2959.65] [2959.65][S02]with other techniques.[2961.17] [2961.17][S02]So we still would train our agents in the same way[2963.81] [2963.81][S02]that we already did without this,[2965.27] [2965.27][S02]and we'd use it to augment the training process.[2968.37] [2968.37][S02]And then another element to it is[2970.81] [2970.81][S02]it's really important that it has some diversity.[2973.85] [2973.85][S02]So if it's always wrong in the same way,[2976.29] [2976.29][S02]then agents might learn to exploit that inaccuracy.[2979.395] [2979.395][S02]Whereas if the model can generate[2980.77] [2980.77][S02]quite diverse different worlds, then what we can do[2984.77] [2984.77][S02]is really test the breadth of agents' capabilities[2987.45] [2987.45][S02]and make sure that there's no scenario where[2989.33] [2989.33][S02]it does something really wrong.[2990.813] [2990.813][S02]That might actually be a strength.[2992.23] [2992.23][S02]So same in sim to real.[2994.31] [2994.31][S02]We want to do domain randomization.[2996.07] [2996.07][S02]Maybe by having a model that is a generative model, being[2999.25] [2999.25][S02]able to search the space of possibilities[3001.93] [3001.93][S02]and check that all of them, the agents[3004.29] [3004.29][S02]do something sensible might be a good thing.[3007.09] [3007.09][S02]But that doesn't mean you want to completely train it[3009.423] [3009.423][S02]that that is the real world.[3010.59] [3010.59][S02]Maybe you want to use it make it more adversarially robust[3014.41] [3014.41][S02]rather than learn specifics.[3016.582] [3016.582][S01] Let me make sure I understand that, then.[3018.79] [3018.79][S01]So if it's wrong but wrong in unpredictable ways,[3021.75] [3021.75][S01]then actually that might end up making the agent more[3024.25] [3024.25][S01]robust in the long run.[3025.85] [3025.85][S02] Yeah.[3026.17] [3026.17][S02]So what you want to do-- it's similar to domain[3028.13] [3028.13][S02]randomization-- is you want to make sure[3029.797] [3029.797][S02]that there's no plausible scenario where the agent could[3032.41] [3032.41][S02]do something really unsafe.[3034.65] [3034.65][S02]It's quite a different objective than if you[3036.73] [3036.73][S02]had one specifically incorrect scenario and then[3039.53] [3039.53][S02]you told the agent exactly how to behave from that one.[3042.81] [3042.81][S02]Instead, it's make it so that in any possible future world,[3046.71] [3046.71][S02]the agent should be able to do something sensible.[3048.93] [3048.93][S01] That's so interesting, then,[3050.597] [3050.597][S01]because I was sort of imagining that you were trying[3053.22] [3053.22][S01]to nudge this towards it being more realistic to more[3057.62] [3057.62][S01]towards getting more reliable outcomes towards closing[3060.94] [3060.94][S01]that gap of sim to real.[3062.24] [3062.24][S01]But I mean, the way you're describing[3063.782] [3063.782][S01]this is that not necessarily.[3066.212] [3066.212][S03] I think the question is,[3067.92] [3067.92][S03]what do you mean by reliable?[3069.52] [3069.52][S03]Because reliable, to me, is that is probably[3073.5] [3073.5][S03]a lot about following the instructions that we[3075.9] [3075.9][S03]provide the model.[3077.2] [3077.2][S03]So if we want the model to simulate a specific environments[3081.06] [3081.06][S03]and we describe it with a lot of detail,[3083.385] [3083.385][S03]we want the model to follow that.[3084.76] [3084.76][S03]If there is something not so plausible in this description,[3090.0] [3090.0][S03]the models should still follow that.[3091.7] [3091.7][S03]I think the challenge is that sometimes we[3094.94] [3094.94][S03]as people, or for various reasons,[3097.04] [3097.04][S03]we're interested in the less plausible scenarios.[3100.1] [3100.1][S03]We're just looking at the tree in the middle of-- for example,[3103.64] [3103.64][S03]in some of the examples we saw today,[3106.18] [3106.18][S03]you want some vodka and the calf liver.[3109.42] [3109.42][S03]All right?[3109.92] [3109.92][S03]So that's not a very--[3111.46] [3111.46][S03]it's not if you just sample from all[3113.3] [3113.3][S03]of the possible tables in Siberia, probably[3115.98] [3115.98][S03]that's not in the middle of the distribution.[3118.68] [3118.68][S03]So I think that the reliability to me[3121.3] [3121.3][S03]comes mostly from following the description that we provide[3124.86] [3124.86][S03]the model with and simulating the world in a way that[3127.58] [3127.58][S03]would be close to that.[3128.76] [3128.76][S02] I think that's a really good point,[3129.72] [3129.72][S02]actually.[3130.22] [3130.22][S02]I think it's in underspecified environments you want diversity,[3133.02] [3133.02][S02]because you want to be able to adapt[3134.78] [3134.78][S02]to anything within the plausible distribution.[3137.437] [3137.437][S02]But if you have a very well specified environment,[3139.52] [3139.52][S02]then you want it to be accurate.[3141.867] [3141.867][S02]And I think we're kind of seeing improvement[3143.7] [3143.7][S02]on both those dimensions, but we're probably not fully[3146.54] [3146.54][S02]there yet.[3147.638] [3147.638][S01] Let me go back to that AGI question--[3149.68] [3152.196][S01]if I may.[3153.86] [3153.86][S01]The end question that everyone always wants to ask.[3156.34] [3156.34][S01]Do you think that this is a step towards it, Jack?[3159.56] [3159.56][S02] I think AGI is something itself, which[3161.98] [3161.98][S02]is relatively subjective and people[3164.82] [3164.82][S02]have different interpretations of what you mean by AGI.[3167.9] [3167.9][S02]So I think it would be quite maybe grandiose to say[3171.83] [3171.83][S02]our model is the key thing in the whole field that[3174.39] [3174.39][S02]will enable AGI.[3175.75] [3175.75][S02]But I think for me, an AGI needs to be embodied[3178.63] [3178.63][S02]and be able to act in the physical world.[3180.64] [3180.64][S02]That's what really excites me.[3181.89] [3181.89][S02]I think that could really improve people's quality of life[3185.83] [3185.83][S02]in any demographic anywhere in the world.[3189.09] [3189.09][S02]And so with that framing, I definitely[3191.35] [3191.35][S02]think this is an important tool.[3192.83] [3192.83][S02]I can't see how an embodied AGI or AGI that is embodied[3196.71] [3196.71][S02]would be able to operate any scenario in the world[3199.75] [3199.75][S02]without being able to simulate it,[3201.33] [3201.33][S02]to gather experience and learn from its own experience.[3203.97] [3203.97][S02]Because that's the paradigm that we've used in other settings[3206.55] [3206.55][S02]to get superhuman capabilities or even[3208.83] [3208.83][S02]just robust capabilities.[3210.79] [3210.79][S02]And so I believe very strongly that we need simulation,[3214.47] [3214.47][S02]and I also believe very strongly that we[3216.27] [3216.27][S02]won't be able to build a simulator of the real world[3218.87] [3218.87][S02]any other way.[3219.908] [3219.908][S02]So when you combine those two things,[3221.45] [3221.45][S02]I think, yes, it is a big step for my version of AGI.[3224.993] [3224.993][S03] Yeah.[3225.91] [3225.91][S03]And I think it's a really good answer.[3228.19] [3228.19][S03]On top of that, I would just say that our current generation[3231.23] [3231.23][S03]of AI is limited to connect the digital world.[3236.67] [3236.67][S03]For AI to be useful for us, definitely it[3238.63] [3238.63][S03]has to have some kind of real world interaction.[3242.11] [3242.11][S03]So I think, again, it's a small step towards that,[3245.35] [3245.35][S03]towards that embodied AI.[3248.27] [3248.27][S03]And there are definitely a lot of gaps to get there,[3251.15] [3251.15][S03]so I think we need much better signals for, for example, robots[3255.31] [3255.31][S03]get while they walk through the world.[3257.03] [3257.03][S03]They need to get some physical response.[3259.61] [3259.61][S03]It's not enough just to have a visual input and output.[3263.91] [3263.91][S03]So I just think that this is definitely[3265.83] [3265.83][S03]a step towards that vision, yeah.[3267.21] [3267.21][S01] Because there is still a lot that this can't do.[3269.01] [3269.01][S01]I mean, the additional sensors being one of them.[3271.73] [3271.73][S01]But also doesn't handle people that well at the moment, Jack?[3274.272] [3274.272][S02] Exactly, and I[3275.688] [3275.688][S02]think that's really the key thing is I both think[3277.87] [3277.87][S02]this is one of the most promising technologies[3280.31] [3280.31][S02]to achieve sociable and socially aware robots,[3284.11] [3284.11][S02]and embodied agents, but also, I think[3285.95] [3285.95][S02]that's the biggest limitation, probably,[3288.28] [3288.28][S02]of the current iteration of the model[3290.32] [3290.32][S02]is that it doesn't do this perfectly.[3291.88] [3291.88][S02]Right?[3292.38] [3292.38][S02]Because our standards have raised,[3293.78] [3293.78][S02]and that's the thing that we now think[3295.12] [3295.12][S02]is something that isn't good enough.[3296.72] [3296.72][S02]But I think it's really critical that we do have that, right?[3299.82] [3299.82][S02]Because even if our robots and embodied agents[3302.56] [3302.56][S02]fully understand physics, I think[3304.52] [3304.52][S02]physics are fairly consistent around the world[3306.7] [3306.7][S02]but people are not, right?[3308.26] [3308.26][S02]And we want these agents, robots,[3311.24] [3311.24][S02]whatever form factor they come in,[3312.76] [3312.76][S02]to be able to really augment humans[3314.52] [3314.52][S02]and work with humans to make our quality of life better.[3317.8] [3317.8][S02]And so they need to understand how humans think,[3320.54] [3320.54][S02]work, interact, and be able to work with us on things.[3324.575] [3324.575][S02]And so that's what I think we're really[3326.2] [3326.2][S02]excited about, as one of the things that[3327.867] [3327.867][S02]might be enabled by our model.[3329.56] [3329.56][S03] Yeah, I think there are definitely[3331.685] [3331.685][S03]a lot of limitations that are in terms[3333.64] [3333.64][S03]of the quality of the generation.[3336.12] [3336.12][S03]But what I mostly--[3338.16] [3338.16][S03]I'm very excited about this, the pace.[3340.44] [3340.44][S03]If you think about it, we had Genie 2, Veo 2 in December,[3344.88] [3344.88][S03]and we definitely feel the pace, the impact on it[3348.8] [3348.8][S03]in our personal lives.[3350.24] [3350.24][S03]But the field is just moving fast.[3352.42] [3352.42][S03]And if we remember just, what, less than two years ago, we[3355.64] [3355.64][S03]had images generated with six fingers[3359.6] [3359.6][S03]and that was a big thing, and nobody[3361.16] [3361.16][S03]is speaking about that anymore.[3362.84] [3362.84][S03]So I don't see why we won't be able to generate people[3365.8] [3365.8][S03]in much higher fidelity and with everything that follows.[3369.662] [3369.662][S01] Is the goal here, then,[3371.12] [3371.12][S01]to have a foundational model?[3372.68] [3372.68][S01]To essentially do for simulated worlds what LLMs[3376.64] [3376.64][S01]have done for language?[3377.843] [3377.843][S02] Yeah, exactly.[3379.26] [3379.26][S02]I think you put it better than probably than I could.[3381.96] [3381.96][S02]I think this is really a step change as a foundation[3384.52] [3384.52][S02]model in terms of the breadth and generality and capabilities.[3387.94] [3387.94][S02]And I think this is probably similar to what Shlomi alluded[3390.56] [3390.56][S02]to as we've seen with images recently,[3392.38] [3392.38][S02]where there were things obviously[3393.84] [3393.84][S02]like the fingers to them now being, I mean,[3396.62] [3396.62][S02]pretty incredible.[3397.56] [3397.56][S02]We saw the same thing with video maybe in the past year[3400.04] [3400.04][S02]where, once we had something like Veo 2,[3402.14] [3402.14][S02]it's looking pretty amazing at this point.[3405.35] [3405.35][S02]And we saw this with language models[3406.85] [3406.85][S02]maybe three or four years ago, where[3408.53] [3408.53][S02]they started to get really capable,[3410.29] [3410.29][S02]and we wanted to get to that point[3411.73] [3411.73][S02]for this new kind of foundation model, sort[3413.97] [3413.97][S02]of an autoregressive world model.[3415.97] [3415.97][S02]Now we're there, there's a whole host[3417.73] [3417.73][S02]of different potential things that could be used for[3420.37] [3420.37][S02]and have impact on, and we're still fairly early in that right[3423.45] [3423.45][S02]now.[3423.95] [3423.95][S01] But are there elements of simulation too?[3426.62] [3426.62][S01]Do you think that you will ever be able to use this kind of idea[3431.17] [3431.17][S01]to recreate a lived experience, rather than just a visual one?[3435.17] [3435.17][S03] So we have many, many senses[3437.25] [3437.25][S03]that we're not even aware of.[3438.93] [3438.93][S03]So, for example, a proprioceptive perception[3443.13] [3443.13][S03]that we are basically we feel where[3445.29] [3445.29][S03]we are and we have this kind of notion[3447.49] [3447.49][S03]of where we are in the world.[3449.89] [3449.89][S03]When we think about actually putting people in a simulation[3453.33] [3453.33][S03]to really feel kind of immersed in it,[3455.75] [3455.75][S03]I think this is a huge part of it,[3457.43] [3457.43][S03]and basically the constraint to visual and maybe audio[3460.09] [3460.09][S03]is still too much of a constraint.[3463.09] [3463.09][S03]I think that there is definitely a potential for that,[3466.51] [3466.51][S03]but it goes through multiple technologies[3469.61] [3469.61][S03]that we have to build to actually get there.[3472.87] [3472.87][S03]So before that, I expect people to be[3475.17] [3475.17][S03]able to interact with photorealistic environments,[3479.39] [3479.39][S03]but still through some kind of an interface that will[3482.81] [3482.81][S03]be like a hybrid interface.[3485.57] [3485.57][S03]Like maybe they can feel some sensation[3487.65] [3487.65][S03]for maybe gloves or something.[3489.303] [3489.303][S02] I think I would also[3490.97] [3490.97][S02]say that there is definitely something[3492.553] [3492.553][S02]about being interactive in real time that[3494.69] [3494.69][S02]does make a big difference for the experience.[3496.89] [3496.89][S02]We've had members of the team say that they visited childhood[3500.01] [3500.01][S02]locations, for example, and did actually get a sense for it[3503.69] [3503.69][S02]that you couldn't really get from an image or a video.[3506.21] [3506.21][S02]So there is already some sort of degree of experience[3508.93] [3508.93][S02]that you can gain from this kind of model already.[3512.17] [3512.17][S02]And obviously, we're working hard[3513.69] [3513.69][S02]to make it an even more capable model in the future,[3517.27] [3517.27][S02]so maybe that will extend.[3518.733] [3518.733][S03] Yeah.[3519.65] [3519.65][S03]Actually, it reminds me of a project[3521.15] [3521.15][S03]that we had earlier in the year that the team at Google[3527.5] [3527.5][S03]used Veo, actually, to help people[3530.94] [3530.94][S03]with early onset of dementia to go back to their childhood[3534.78] [3534.78][S03]memories and reconstruct them.[3537.2] [3537.2][S03]So I can imagine that that might be,[3539.32] [3539.32][S03]for example, a potentially therapeutic tool[3542.82] [3542.82][S03]as well, that it can not only look at the video,[3545.64] [3545.64][S03]but maybe actually relive or remember some things[3549.82] [3549.82][S03]from their childhood.[3550.92] [3550.92][S03]So I think that even before, we don't[3552.78] [3552.78][S03]need to go very far for things to have[3555.34] [3555.34][S03]positive impact on the world.[3557.02] [3557.02][S01] Amazing.[3558.1] [3558.1][S01]That was absolutely fascinating.[3559.52] [3559.52][S01]Thank you so much.[3560.44] [3560.44][S02] Thanks for having us.[3561.28] [3561.28][S03] Thanks for having us, Hannah.[3562.58] [3562.58][S01] I think the most impressive part of this[3564.747] [3564.747][S01]is not what you're looking at on the screen.[3567.04] [3567.04][S01]It's how that is generated.[3569.64] [3569.64][S01]It's this change that this model represents,[3572.18] [3572.18][S01]from creating realistic images or videos of the real world,[3575.54] [3575.54][S01]as though they were frozen moments in time,[3579.34] [3579.34][S01]into something that can actually handle time in the way[3582.5] [3582.5][S01]that we experience it, with an arrow that's[3584.82] [3584.82][S01]pointing in only one direction, where effect follows cause[3588.94] [3588.94][S01]to build this consistent, forward moving world[3593.42] [3593.42][S01]where the present is a direct result of the past.[3597.5] [3597.5][S01]And that is why I think that this[3599.34] [3599.34][S01]is an early hint of something that's much bigger.[3602.48] [3602.48][S01]This is not just a new way to design games[3604.5] [3604.5][S01]or beautiful environments.[3606.26] [3606.26][S01]This here is the bedrock for machines[3609.3] [3609.3][S01]that can genuinely plan and reason about our world.[3614.905] [3614.905][S01]You have been listening to "Google DeepMind--[3616.78] [3616.78][S01]The Podcast" with me, Professor Hannah Fry.[3618.973] [3618.973][S01]Now we're going to take a little bit of a pause over the summer,[3621.64] [3621.64][S01]but we're going to be back with more episodes in the autumn[3624.38] [3624.38][S01]from Google HQ in California.[3626.86] [3626.86][S01]And in the meantime, do take a look[3628.34] [3628.34][S01]at our extensive back catalog, which covers everything[3631.22] [3631.22][S01]from tools for creators to AI for drug discovery.[3634.68] [3634.68][S01]See you soon.[3636.45]