[0.0][MUSIC PLAYING][3.297] [6.675][S01] Welcome back to Google DeepMind, the Podcast.[9.05] [9.05][S01]I'm your host, Professor Hannah Fry.[11.12] [11.12][S01]Now, there are few places in the past couple[13.25] [13.25][S01]of years that have felt the transformative influence of AI[16.79] [16.79][S01]as keenly as the education sector.[19.58] [19.58][S01]Teachers around the world are having[21.08] [21.08][S01]to abruptly rethink how they engage with and assess students,[25.71] [25.71][S01]and there are concerns around the potential for cheating[28.31] [28.31][S01]and dependence on technology.[30.3] [30.3][S01]Those fears are valid.[32.729] [32.729][S01]But despite the rapid transition,[34.8] [34.8][S01]there has been something remarkably resilient[37.55] [37.55][S01]about the idea of a human teacher,[40.02] [40.02][S01]something that at its heart has remained immune to[43.34] [43.34][S01]ebbs and flows of technology.[45.81] [45.81][S01]After all, we've had classrooms for almost as long[48.32] [48.32][S01]as we've had civilizations.[50.31] [50.31][S01]And there are undoubtedly opportunities here, too.[52.95] [52.95][S01]Imagine a classroom where each lesson[54.86] [54.86][S01]is tailored to your individual pace of learning, where[58.55] [58.55][S01]an AI tutor is available around the clock,[60.84] [60.84][S01]and technology can predict where you're likely to get stuck[64.37] [64.37][S01]before you do.[66.02] [66.02][S01]Well, researchers here at Google DeepMind[68.52] [68.52][S01]have been grappling with both the opportunities and challenges[71.67] [71.67][S01]of AI in education.[73.54] [73.54][S01]They recently published a major paper[75.57] [75.57][S01]on developing AI responsibly in this area,[79.2] [79.2][S01]and one of its lead authors is my guest on the podcast.[82.09] [82.09][S01]Irina Jurenka is a research lead at Google DeepMind.[85.18] [85.18][S01]Her background spans experimental psychology[87.66] [87.66][S01]and computational neuroscience, and she has spent a decade[90.81] [90.81][S01]within these walls asking questions[92.91] [92.91][S01]like how do humans learn?[95.53] [95.53][S01]Welcome to the podcast, Irina.[97.0] [97.0][S01]This is a space where people are very heavily invested.[99.99] [99.99][S01]Does that make it quite a difficult space to navigate?[102.69] [102.69][S02] It does because if you think about it,[105.37] [105.37][S02]education has been around for thousands of years,[108.18] [108.18][S02]and it is a fundamental structure in our society.[112.68] [112.68][S02]Every child is supposed to get educated.[115.06] [115.06][S02]So the educational systems have been around for a while.[118.6] [118.6][S02]They are quite rigid, and they're very established.[123.23] [123.23][S02]So to come in and say, look, we have this amazing technology,[127.45] [127.45][S02]and we're going to revolutionize everything and change[130.03] [130.03][S02]everything, I think it's not going to work so easily.[134.99] [134.99][S02]And we've seen this happen with technologies of the past.[138.38] [138.38][S02]Like, intelligent tutoring systems[140.2] [140.2][S02]have existed for 50 plus years.[143.77] [143.77][S02]A lot of investment and research has gone into them.[147.92] [147.92][S02]But you could argue that the promise of that technology[152.77] [152.77][S02]hasn't fully materialized.[155.41] [155.41][S02]Or more recently, we had MOOCs, these massive open courses,[160.91] [160.91][S02]and again, there was so much excitement[162.64] [162.64][S02]how we won't need traditional education anymore.[165.16] [165.16][S02]You can just go online and learn anything[167.26] [167.26][S02]you would ever want to learn.[168.91] [168.91][S02]And once again, when you actually[171.01] [171.01][S02]look at who uses these systems, it's[174.01] [174.01][S02]people who have already gone through traditional education.[176.745] [176.745][S02]And typically, it's people trying[178.12] [178.12][S02]to get their second Masters.[180.27] [180.27][S02]So it's definitely not the thing that came and broke the system.[185.9] [185.9][S02]And I guess maybe we shouldn't be trying to break the system.[190.58] [190.58][S02]There is a lot of amazing stuff happening[194.078] [194.078][S02]in traditional education.[195.12] [195.12][S02]It's not just about taking the knowledge from the teacher[198.29] [198.29][S02]and distilling or drip feeding that into the student.[202.91] [202.91][S02]It's about the social aspects of talking to your peers[206.93] [206.93][S02]and learning together.[208.08] [208.08][S02]It's about the teachers, giving skills[212.89] [212.89][S02]like how to be a global citizen, how to navigate,[218.58] [218.58][S02]how to critically think, how to evaluate information.[221.52] [221.52][S02]So there is so much more to the educational systems than just[226.94] [226.94][S02]the knowledge that they give.[229.01] [229.01][S02]So in our team, we're thinking about the new technology[234.41] [234.41][S02]in terms of how can it work within the current system,[238.34] [238.34][S02]and how can it add to it?[240.402] [240.402][S01] So this isn't starting with a brand[242.36] [242.36][S01]new blank sheet of paper and saying design an education[245.36] [245.36][S01]system from scratch.[246.21] [246.21][S01]It's like augmenting the one that exists.[248.075] [248.075][S02] Yeah, so actually,[249.45] [249.45][S02]Justin Reicher, a researcher at MIT, has a really nice quote.[253.49] [253.49][S02]So he says that, "New technology doesn't[257.48] [257.48][S02]break educational systems.[259.61] [259.61][S02]Educational systems kind of tame new technology."[263.553] [263.553][S01] Which is what happened with MOOCs,[265.47] [265.47][S01]as you said.[265.97] [265.97][S02] Exactly, yes.[268.065] [268.065][S02]And yeah, as I said, we're also seeing[270.52] [270.52][S02]that there are these human aspects[273.64] [273.64][S02]of teacher-student interactions that we can't possibly[277.3] [277.3][S02]ever change with technology.[278.84] [278.84][S02]For example, if you think about a student and a tutor,[283.45] [283.45][S02]there are some social rules that are in place where a student is[289.18] [289.18][S02]very unlikely to just stand up and walk away[291.19] [291.19][S02]from a human tutor.[292.64] [292.64][S02]But if you are interacting with an AI tutor,[295.13] [295.13][S02]you can just close the window, and that's it.[297.97] [297.97][S02]So there are certain challenges that[300.4] [300.4][S02]come with bringing technology in,[304.12] [304.12][S02]and there are certain things that human-to-human interactions[307.69] [307.69][S02]have that technology will never replace.[309.92] [309.92][S02]So this is why we're trying to work within the system[313.42] [313.42][S02]to begin with.[314.543] [314.543][S01] How disruptive do you expect[316.21] [316.21][S01]it will be to education, then?[318.11] [318.11][S01]Because I mean, on the one hand, there[319.78] [319.78][S01]has been quite a lot of disruption already,[321.68] [321.68][S01]especially particularly recently with large language models.[324.76] [324.76][S01]When I spoke to [? Demis, ?] he was talking about overestimating[327.742] [327.742][S01]the impact of something in the short term[329.45] [329.45][S01]and then underestimating how big the longer term impact will be.[334.97] [334.97][S01]Where do you think education fits in with this?[338.49] [338.49][S02] I feel like there is so much buzz about GenAI[342.11] [342.11][S02]right now in education.[343.65] [343.65][S02]I feel like everyone actually expects it to completely change[348.2] [348.2][S02]everything immediately.[349.68] [349.68][S02]And there have been so many different attacks[352.28] [352.28][S02]that sprung around taking a language model[355.43] [355.43][S02]and turning that into a tutor or a homework helper[360.11] [360.11][S02]or anything else that helps students.[363.15] [363.15][S02]And honestly, so far, nothing has really[367.25] [367.25][S02]made the impact that I think everyone was expecting.[370.8] [370.8][S02]So that's why, from our perspective,[373.65] [373.65][S02]we are in the center of actually improving this technology,[377.06] [377.06][S02]and we have unprecedented access to Gemini.[381.33] [381.33][S02]We can influence how things change.[384.06] [384.06][S02]And in fact, one of our goals is to make[387.29] [387.29][S02]Gemini the best large language model for education.[390.722] [390.722][S01] So what is the ambition here?[392.43] [392.43][S01]Is it to build a universal AI tutor?[396.06] [396.06][S02] It is, but we also wanted[398.99] [398.99][S02]to power different experiences.[402.54] [402.54][S02]So the very first place where we deployed our AI tutor[407.33] [407.33][S02]was YouTube.[408.69] [408.69][S02]So on learning videos, there is now[411.89] [411.89][S02]a new function, which is kind of like if you're watching[414.515] [414.515][S02]a learning video, and you don't quite understand something,[417.39] [417.39][S02]you can virtually raise your hand.[419.15] [419.15][S02]And an AI tutor will pop up, and you can ask all your learning[423.65] [423.65][S02]questions to the tutor.[425.28] [425.28][S02]And then more recently, we also launched Gemini Gem.[430.83] [430.83][S02]So it's called the learning coach,[432.72] [432.72][S02]and it's basically optimized to be your guide[437.57] [437.57][S02]into learning experiences.[439.17] [439.17][S02]So here, you can come up with any question.[441.99] [441.99][S02]Let's say, I want to learn about photosynthesis,[444.08] [444.08][S02]or tell me about American Civil War.[448.26] [448.26][S02]And it will give you a plan.[451.05] [451.05][S02]It will try to understand what you know and don't, and then it[453.83] [453.83][S02]will try to guide you through the materials.[457.14] [457.14][S02]So what we're hoping to do is really push the research[460.58] [460.58][S02]to make these base models as good as possible for education[465.38] [465.38][S02]and then figure out how to actually make[469.88] [469.88][S02]the best use of them.[471.15] [471.15][S02]And in fact, I think we hope that the community can help us[473.87] [473.87][S02]with that so that it's not just us dictating what[479.0] [479.0][S02]an AI tutor should be like.[480.78] [480.78][S02]It's us listening to people who have been in the space[483.77] [483.77][S02]for much longer than us and trying[485.42] [485.42][S02]to help them make the most of technology[488.36] [488.36][S02]and make the technology be the best it can be for them.[492.225] [492.225][S01] How far do you think the technology can go, though?[494.85] [494.85][S01]Can you paint me an image of what[497.39] [497.39][S01]you, in a very optimistic scenario,[499.98] [499.98][S01]would like the future to look like?[501.93] [501.93][S02] I think a lot of people talk about AI-first[506.41] [506.41][S02]schooling or these--[509.4] [509.4][S02]I think there is even a school in the UK[511.65] [511.65][S02]that just switched to mostly having AI-based education.[517.21] [517.21][S02]And I think they only just have a few teachers on hand[520.08] [520.08][S02]to help around.[522.58] [522.58][S02]And I just don't think that that future is something[525.81] [525.81][S02]we should be striving for.[527.71] [527.71][S02]We really don't want to replace human teachers.[532.96] [532.96][S02]We want to give a tool that enhances[536.13] [536.13][S02]this kind of in-person classroom experience between teachers[539.58] [539.58][S02]and students.[542.01] [542.01][S02]I think it's a little bit sad if students come to school[547.18] [547.18][S02]and just sit around looking at screens all day.[550.17] [550.17][S02]So the way we are thinking about it[552.45] [552.45][S02]is that there's still teachers as mentors, as role models[558.6] [558.6][S02]to the students.[559.93] [559.93][S02]And there is a lot of peer interactions during learning.[563.81] [563.81][S02]But there is this AI system that helps,[568.39] [568.39][S02]that works with teachers and learners[571.69] [571.69][S02]and helps them make the best of the situation.[577.04] [577.04][S02]So maybe for each learner, the AI tutor[581.35] [581.35][S02]can help them move at their own pace[583.99] [583.99][S02]and really target their interests.[586.94] [586.94][S02]And at the same time, the teacher[588.82] [588.82][S02]gets a view of where everyone is, and they can kind of still[593.98] [593.98][S02]steer this tutor.[595.31] [595.31][S02]So they still have control, and they can still[597.73] [597.73][S02]bring their own personality and teaching style to the lessons[603.078] [603.078][S02]because I think this connection between teachers and students[605.62] [605.62][S02]is so important.[607.07] [607.07][S02]And like looking back on my own education, what stands out to[610.0] [610.0][S02]me is these amazing teachers who made[613.3] [613.3][S02]me excited about a certain subject.[615.23] [615.23][S02]So I think what technology should[617.83] [617.83][S02]be trying to do is make more of interactions and memories[622.36] [622.36][S02]like that and maybe remove the less ideal situations where[627.63] [627.63][S02]maybe the teacher and the student don't click[630.72] [630.72][S02]or the teacher is so overworked that they don't have time[633.87] [633.87][S02]to spend with a particular student who actually needs them[636.84] [636.84][S02]the most.[638.16] [638.16][S01] I imagine that there'll[639.66] [639.66][S01]be some people watching who don't necessarily[641.535] [641.535][S01]know about your background.[643.24] [643.24][S01]So can you tell us a little bit, what[644.88] [644.88][S01]was your path to get to thinking about AI and education?[647.77] [647.77][S02] I mean, how far away shall I start?[650.16] [650.16][S01] Day one.[652.26] [652.26][S01]A brief history.[653.29] [653.29][S02] Yeah, so I've always[656.22] [656.22][S02]been fascinated by intelligence, any kind of intelligence, human[661.89] [661.89][S02]or artificial.[664.59] [664.59][S02]I started coding quite early on in life.[667.5] [667.5][S02]It was just a lucky coincidence that my brother and I got[673.23] [673.23][S02]a comic book as children, and it was about, basically,[677.393] [677.393][S02]introduction to programming.[678.56] [678.56][S01] Amazing.[679.393] [679.393][S02] So we started writing small games[681.84] [681.84][S02]around the age of probably 11 or 12.[684.81] [684.81][S02]And I remember at some point during the summer, my brother[687.49] [687.49][S02]and I were bored, and we discovered that you can actually[690.49] [690.49][S02]get access to the source code of one of those like shooter games.[695.14] [695.14][S02]And you could actually code up your opponents.[697.28] [697.28][S02]So like, wow, this is exciting.[698.9] [698.9][S02]We can actually create AI.[700.69] [700.69][S02]So I remember putting a diary entry[703.69] [703.69][S02]like this summer we're going to solve AI.[707.9] [707.9][S02]Of course, that didn't happen.[709.15] [709.15][S01] Oh, the ambition of youth.[710.8] [710.8][S02] Yeah.[712.048] [712.048][S01] Amazing[712.84] [712.84][S02] Surprisingly, though, my brother went on[715.69] [715.69][S02]to study computer science.[717.7] [717.7][S02]But I was growing up in a traditional society[721.87] [721.87][S02]where somehow it just didn't click to me that computer[726.27] [726.27][S02]science the degree and making games and playing around[731.1] [731.1][S02]on computers with my brother are the same thing.[734.47] [734.47][S02]To me, computer science was something kind[736.44] [736.44][S02]of dry and more about the hardware,[739.14] [739.14][S02]and I really did not enjoy that.[741.64] [741.64][S02]So I ended up studying psychology as my degree.[744.14] [744.14][S02]I was kind of wondering, how can I move towards AI[749.4] [749.4][S02]and still study intelligence?[751.0] [751.0][S02]Because I was fascinated, how does the brain do it?[754.11] [754.11][S02]How does this incredible behavior and intelligence[757.65] [757.65][S02]and reasoning, how does it all arise?[759.38] [759.38][S02]And then I was very lucky that by the time I finished my PhD,[763.11] [763.11][S02]and I heard about DeepMind and how you can actually[767.28] [767.28][S02]do neuroscience research and answer[770.01] [770.01][S02]these deep fundamental questions with deep learning,[773.41] [773.41][S02]it was this perfect job for me.[775.45] [775.45][S02]So I started off in the neuroscience team.[777.88] [777.88][S02]And as I mentioned, this idea of intelligence and reasoning[780.66] [780.66][S02]has always been at the back of my mind[782.46] [782.46][S02]because reasoning is kind of what makes us intelligent.[786.51] [786.51][S02]So I started to work on improving reasoning[789.82] [789.82][S02]in language models.[791.59] [791.59][S02]And very early on, I kind of just,[793.3] [793.3][S02]even before language models became this big thing,[796.55] [796.55][S02]I realized that they were quite bad at reasoning.[799.94] [799.94][S02]But also what I realized is that humans don't really[803.98] [803.98][S02]use reasoning that much.[805.7] [805.7][S02]If you think about it, in our daily lives,[809.15] [809.15][S02]we don't actually think through a lot of our actions.[814.04] [814.04][S02]We kind of just--[814.87] [814.87][S02]we're almost acting on autopilot.[818.11] [818.11][S02]So to really study reasoning, we needed a domain[821.74] [821.74][S02]where reasoning was important, and that's[824.65] [824.65][S02]where education became a thing again[827.24] [827.24][S02]because this is where humans discover how to reason well.[832.372] [832.372][S01] There's something so interesting in that,[834.58] [834.58][S01]then, that the motivation is in some ways trying[838.54] [838.54][S01]to teach AI to be better at reasoning[840.94] [840.94][S01]and in the process understand what[843.82] [843.82][S01]it means to teach reasoning.[845.75] [845.75][S01]And that's kind of quite a nice way around to look at it.[850.3] [850.3][S02] Yeah, and also it's[852.46] [852.46][S02]interesting how doing something and teaching somebody else[856.51] [856.51][S02]how to do it are not the same.[860.26] [860.26][S02]And this is basically the challenge we are now solving.[863.53] [863.53][S02]So the base Gemini is slowly improving[868.48] [868.48][S02]at reasoning and math and coding and all of those basic skills.[873.98] [873.98][S02]But then our job is to actually stop the model[876.79] [876.79][S02]from using these skills and giving away the answer[880.3] [880.3][S02]and really just doing the job for the student[883.87] [883.87][S02]and instead holding back and thinking about what[887.14] [887.14][S02]are the right questions I can ask the student so that they can[891.76] [891.76][S02]figure it out by themselves?[894.19] [894.19][S02]And that's very hard.[895.43] [895.43][S02]Models are fine tuned to be helpful.[897.98] [897.98][S02]So the initial reaction is I'll just give you the answer.[901.66] [901.66][S02]So we have to do a lot of work to stop them from doing that.[904.865] [904.865][S01] But then, actually, I[906.24] [906.24][S01]think you've really hit the nail on the head[907.48] [907.48][S01]there that being able to do something[909.34] [909.34][S01]is not the same as being able to teach it.[911.83] [911.83][S01]And I'm really struck in maths education, which[914.26] [914.26][S01]is the space that I know most about, about how there is this[918.1] [918.1][S01]push and pull from different sectors[919.75] [919.75][S01]about what is required of students and the best[922.51] [922.51][S01]possible way to instill those skills and that knowledge.[925.9] [925.9][S01]If you're building a sort of an AI, which[930.04] [930.04][S01]will have this universal appeal, how[933.88] [933.88][S01]do you find that balance of making sure[936.01] [936.01][S01]that you're hitting all of the notes that are required from all[938.92] [938.92][S01]of the different areas?[941.05] [941.05][S02] That-- it is a good question.[943.34] [943.34][S02]So when we first started building the tutor,[946.1] [946.1][S02]we thought, we can talk to teachers[949.15] [949.15][S02]and other maybe academics in the field as well as learners[955.33] [955.33][S02]and figure out, what is the perfect way to teach?[959.0] [959.0][S02]And then--[959.59] [959.59][S01] As though there is--[960.61] [960.61][S02] Exactly.[961.15] [961.15][S01] A sort of a best.[962.27] [962.27][S02] Yeah, but you kind of assume[964.062] [964.062][S02]that in everything there is this optimal strategy.[966.53] [966.53][S02]Maybe this is the scientist in us kind of--[970.96] [970.96][S02]but-- and we did that.[972.89] [972.89][S02]We went and interviewed a lot of stakeholders.[977.95] [977.95][S02]And what we realized is that there is a lot of disagreement.[981.61] [981.61][S02]And actually, once we started even deploying our early tutor[987.01] [987.01][S02]models on different Google services like YouTube or Gemini[992.8] [992.8][S02]app, we found that even there, there[995.59] [995.59][S02]were different requirements.[997.25] [997.25][S02]So let's say on YouTube, the video, the educational video,[1002.31] [1002.31][S02]is the main act.[1003.85] [1003.85][S02]So the tutor is really there to support that.[1007.99] [1007.99][S02]And maybe the tutor should be giving away[1011.13] [1011.13][S02]answers much more because it's actually[1013.41] [1013.41][S02]helpful for the learner on that surface.[1016.77] [1016.77][S02]At the same time, if you talk to a teacher at school,[1019.57] [1019.57][S02]they have very different requirements.[1021.16] [1021.16][S02]They really don't want the tutor to give away answers, definitely[1025.079] [1025.079][S02]not to the exam questions.[1026.98] [1026.98][S02]And they will also want the tutor[1028.92] [1028.92][S02]to follow some particular exam board requirements[1032.4] [1032.4][S02]or particular teaching style of that particular teacher.[1037.079] [1037.079][S02]So how do you actually incorporate[1039.569] [1039.569][S02]all of those diverse voices into a single tutor?[1043.18] [1043.18][S02]So what we've realized is that we[1045.099] [1045.099][S02]need to build this base pedagogical model that you[1049.36] [1049.36][S02]can steer with different instructions.[1051.91] [1051.91][S02]So one teacher can come and say, actually,[1054.83] [1054.83][S02]I want my students to just have fun today[1056.86] [1056.86][S02]and just answer any question you have[1059.29] [1059.29][S02]and really push on some fun experiences.[1062.6] [1062.6][S02]And another tutor might be much more academic[1064.81] [1064.81][S02]and say, no, today we're doing exam practice problems.[1068.56] [1068.56][S02]You just guide the student through these topics[1071.11] [1071.11][S02]and make sure that they understand everything.[1073.04] [1073.04][S01] I guess one of the big things[1074.748] [1074.748][S01]about education, I mean, as you said,[1076.96] [1076.96][S01]there isn't this optimal approach to teaching.[1079.85] [1079.85][S01]But there are these kind of imperfect measures, really.[1084.35] [1084.35][S01]We sort of know good teaching when we see it,[1086.81] [1086.81][S01]but it feels quite difficult to quantify.[1089.33] [1089.33][S01]So how do you decide what counts as good pedagogy[1093.37] [1093.37][S01]when you're navigating in this space?[1096.88] [1096.88][S02] So first, you might say,[1100.13] [1100.13][S02]well, there's learning science.[1102.22] [1102.22][S02]So why don't you just look at the papers,[1104.2] [1104.2][S02]and they'll give you the answer?[1106.84] [1106.84][S02]And yes, there is a lot of literature,[1110.05] [1110.05][S02]but there is no consensus as such.[1113.88] [1113.88][S02]But another thing is pedagogy is very context dependent.[1117.86] [1117.86][S02]So what works for maybe a novice learner might not work[1121.42] [1121.42][S02]for an expert learner, or what works for a subject that's more[1125.26] [1125.26][S02]procedural-- let's say, math--[1127.78] [1127.78][S02]you actually learn the skill of the procedure[1129.76] [1129.76][S02]of how to solve a problem.[1130.885] [1130.885][S02]It might not work for a more memory based subjects[1134.29] [1134.29][S02]like history.[1136.75] [1136.75][S02]So when you start thinking about,[1139.73] [1139.73][S02]there's these hundreds of different pedagogical strategies[1143.2] [1143.2][S02]that have been studied, all of them[1145.96] [1145.96][S02]work slightly different in different contexts.[1148.82] [1148.82][S02]Suddenly you have this massive space[1152.71] [1152.71][S02]where maybe there isn't one single point that's[1157.04] [1157.04][S02]the best pedagogy, but there are many different regions that[1160.88] [1160.88][S02]are best pedagogies in the given context.[1164.16] [1164.16][S02]But the problem is how do you even quantify this space,[1170.84] [1170.84][S02]and then how do you search it for this perfect pedagogy[1174.89] [1174.89][S02]strategy?[1176.15] [1176.15][S02]And it becomes kind of similar to the work[1178.52] [1178.52][S02]that DeepMind has done before, like playing the game of Go.[1183.2] [1183.2][S02]The reason why it was such a huge challenge for AI[1186.17] [1186.17][S02]was because the search space was huge, all the possible moves[1190.91] [1190.91][S02]you can do.[1191.73] [1191.73][S02]It's so many, and there isn't one known strategy.[1197.33] [1197.33][S02]The AI has to search the space of possible moves and strategies[1202.1] [1202.1][S02]and discover what it thinks is the best one.[1205.38] [1205.38][S02]And what we found with the AlphaGo[1207.83] [1207.83][S02]work was that, first of all, AI was much better than humans.[1212.76] [1212.76][S02]Basically, all of humanity for thousands of years[1215.07] [1215.07][S02]playing the game of Go.[1216.25] [1216.25][S02]AI was actually able to search the space[1218.55] [1218.55][S02]and discover better strategies in the matter of days or months[1224.25] [1224.25][S02]compared to what humans could do.[1226.09] [1226.09][S02]So our hope is that we can do something similar[1229.83] [1229.83][S02]with education, but we're going back to this kind of question[1235.35] [1235.35][S02]like how do we actually know what success looks like?[1238.65] [1238.65][S02]In Go, you can still measure who has won,[1242.22] [1242.22][S02]and it's pretty unambiguous.[1244.26] [1244.26][S02]Whereas, in education, the Holy Grail[1247.92] [1247.92][S02]is whether the students' learning outcomes[1251.7] [1251.7][S02]have become better.[1254.76] [1254.76][S02]But this is not something you can measure quickly.[1257.49] [1257.49][S02]You need months, if not years, to really track the learner,[1262.12] [1262.12][S02]and that's not really feasible.[1263.96] [1263.96][S02]So a lot of our work is actually done--[1266.85] [1266.85][S02]OK, we know what we're aiming for,[1269.62] [1269.62][S02]but how can we approximate it in a way that's easier to measure[1273.06] [1273.06][S02]and faster to measure?[1274.48] [1274.48][S02]So we published a report recently[1278.04] [1278.04][S02]where it's like 70 pages of basically our trial and error[1283.8] [1283.8][S02]and different attempts at measuring pedagogy, going[1288.51] [1288.51][S02]from working with real students at Arizona State University[1291.87] [1291.87][S02]and maybe measuring at a longer time[1293.37] [1293.37][S02]scales of a couple of months, to asking pedagogical raters[1299.61] [1299.61][S02]and teachers to look through a few examples of conversations[1304.17] [1304.17][S02]between students and our AI tutor[1306.72] [1306.72][S02]and maybe give us quicker feedback on the order of weeks[1309.72] [1309.72][S02]or days, to automatic measures where we actually[1313.08] [1313.08][S02]ask AI to evaluate AI and give us a much more targeted, much[1320.52] [1320.52][S02]more limited, but still useful feedback in the matter of hours.[1323.707] [1323.707][S01] But I guess, if you really[1325.29] [1325.29][S01]want to evaluate what good teaching is,[1327.105] [1327.105][S01]you want to do that full randomized control[1330.9] [1330.9][S01]trial where you're monitoring people over a period of time.[1333.61] [1333.61][S01]How far away do you think we are from being able to run those?[1337.98] [1337.98][S02] Well, these are being run right now.[1340.48] [1340.48][S02]So Arizona State University is one example where[1342.72] [1342.72][S02]we're actually running these.[1344.19] [1344.19][S02]I think the problem with these is[1346.59] [1346.59][S02]even if you take your students and you split them[1349.17] [1349.17][S02]into students who have access to the AI tutor[1352.08] [1352.08][S02]and students who don't, what we find[1354.39] [1354.39][S02]is that in the group where they theoretically have access[1357.75] [1357.75][S02]to the tutor, only a small percentage actually[1360.27] [1360.27][S02]engage with it.[1361.89] [1361.89][S02]And that creates a problem because why are some students[1366.15] [1366.15][S02]engaging and others not?[1367.54] [1367.54][S02]Is there something inherently different about these students?[1372.34] [1372.34][S02]And then if we only see success in those[1376.17] [1376.17][S02]who engage, is it because of the tutor, or is it[1379.02] [1379.02][S02]because these learners were inherently more motivated,[1382.71] [1382.71][S02]and hence, they would have done better anyway?[1385.78] [1385.78][S02]And then the question is, who are we helping,[1388.81] [1388.81][S02]and what effect does it have at a larger scale?[1393.25] [1393.25][S02]So if you think about the top students[1396.33] [1396.33][S02]and the bottom students, and then you're[1398.34] [1398.34][S02]helping the top students do better.[1401.05] [1401.05][S02]But you're not actually helping the bottom students.[1403.6] [1403.6][S02]You're actually increasing the gap.[1405.84] [1405.84][S02]But I think everyone, when they go into Ed Tech,[1409.12] [1409.12][S02]they actually want to decrease the gap.[1410.77] [1410.77][S02]So how do we do that?[1413.65] [1413.65][S02]How do we make sure that everyone engages?[1416.86] [1416.86][S02]That's another big question that we're working on.[1418.947] [1418.947][S01] I mean, there's just imperfect measures everywhere[1421.53] [1421.53][S01]you look, isn't there?[1422.46] [1422.46][S01]It's very, very difficult to get a real ground[1424.95] [1424.95][S01]truth in any of this.[1426.125] [1426.125][S01]But then I suppose there's also--[1427.5] [1427.5][S01]I mean, there's further complications in this[1429.375] [1429.375][S01]because, OK, so that's sort of teaching style.[1431.68] [1431.68][S01]But presumably, there are some subjects[1433.83] [1433.83][S01]where there's more of a ground truth than others.[1438.7] [1438.7][S01]I'm thinking, for example, if you created a tutor for history,[1442.24] [1442.24][S01]I mean, it would change depending on which country you[1445.35] [1445.35][S01]were in as to what might be the most relevant answers[1449.01] [1449.01][S01]to a particular question.[1450.6] [1450.6][S02] Yes, this is a big issue for us.[1456.49] [1456.49][S02]We are-- yeah, we've thought a lot about what[1459.34] [1459.34][S02]do you do in this situation?[1460.7] [1460.7][S02]Because you can't give this one true answer to any history[1464.71] [1464.71][S02]question.[1466.12] [1466.12][S02]This is again why we're thinking about steerability[1469.51] [1469.51][S02]so that teachers in different countries[1472.6] [1472.6][S02]can give the background information to the tutor.[1477.46] [1477.46][S02]So it kind of knows what is the expected way[1480.97] [1480.97][S02]of answering certain questions.[1484.06] [1484.06][S02]But it also-- often historical topics[1487.21] [1487.21][S02]bring up questions that are really important to discuss[1492.11] [1492.11][S02]but are also hard to discuss and very sensitive.[1495.26] [1495.26][S02]I'm thinking things like the Holocaust.[1498.52] [1498.52][S02]So again, how should the tutor behave in these situations?[1503.68] [1503.68][S02]I think the standard approach to safety[1506.47] [1506.47][S02]often is effectively declining to engage[1510.07] [1510.07][S02]in a difficult conversation.[1513.07] [1513.07][S02]But that's not something a tutor can do.[1516.332] [1516.332][S01] No, I mean part of the point of education[1518.54] [1518.54][S01]is to think about difficult things.[1520.402] [1520.402][S02] Exactly.[1521.36] [1521.36][S02]So I can't say that we've solved this problem.[1526.2] [1526.2][S02]We are trying to give different views[1530.04] [1530.04][S02]and trying to give the learners a chance to critically evaluate[1535.74] [1535.74][S02]different ideas in the space and also[1541.23] [1541.23][S02]really trying to bring metacognition to this problem.[1548.54] [1548.54][S02]So metacognition is an interesting one.[1550.47] [1550.47][S02]I think it often get overlooked.[1553.28] [1553.28][S02]But a lot of people don't actually know how to learn.[1557.45] [1557.45][S02]It's often not as much fun as you would expect.[1562.82] [1562.82][S02]It requires you to be--[1565.94] [1565.94][S02]like to plan ahead, to really engage with the materials.[1569.93] [1569.93][S02]And yeah, most people don't really know how to do that.[1574.52] [1574.52][S02]So what a tutor can do is actually teach the learner,[1580.09] [1580.09][S02]if you're trying to answer this difficult question,[1582.63] [1582.63][S02]maybe what you should do is go and look up[1585.71] [1585.71][S02]different primary sources and then[1588.62] [1588.62][S02]think about what are they telling you,[1590.81] [1590.81][S02]and what do you think about it?[1592.38] [1592.38][S02]What do other experts think about this?[1594.68] [1594.68][S02]And teaching the learner how to go[1597.977] [1597.977][S02]about answering these questions rather than necessarily giving[1600.56] [1600.56][S02]the answers directly.[1601.725] [1601.725][S01] There's layers to it, then.[1603.35] [1603.35][S01]I guess, on one layer you have knowledge and facts, which[1607.31] [1607.31][S01]is, I guess, maths is quite full of them.[1610.33] [1610.33][S01]And then above that, you've got the skills[1612.24] [1612.24][S01]of critically evaluating.[1613.41] [1613.41][S01]And then above that metacognition,[1614.98] [1614.98][S01]which is how to develop the skills to evaluate[1618.09] [1618.09][S01]the knowledge.[1618.91] [1618.91][S02] Exactly.[1619.868] [1619.868][S01] So, you think that's the answer to this safety[1622.29] [1622.29][S01]question of approaching difficult problems.[1625.527] [1625.527][S02] So not necessarily the answer[1627.36] [1627.36][S02]to safety.[1628.42] [1628.42][S02]It's more of an answer how to engage with subjects where,[1631.9] [1631.9][S02]as you said, there is no necessarily like single ground[1634.2] [1634.2][S02]truth.[1634.86] [1634.86][S02]In terms of safety, I think it's a slightly different question.[1639.16] [1639.16][S02]Sometimes people ask us like, why are you[1642.33] [1642.33][S02]working on safety at all?[1644.41] [1644.41][S02]Aren't you using base models which already went through a lot[1648.87] [1648.87][S02]of safety, fine tuning safety work?[1651.78] [1651.78][S02]And the answer to that is, even though they[1655.53] [1655.53][S02]have done all of this background work,[1660.03] [1660.03][S02]when it comes to the educational use case specifically,[1666.16] [1666.16][S02]you have to think about how these systems will be used.[1670.24] [1670.24][S02]So one thing we found was that--[1673.65] [1673.65][S02]so our tutors are deployed to Arizona State University[1678.06] [1678.06][S02]students and in particular through their study hall[1681.96] [1681.96][S02]program, which is aimed at bringing more diverse learners[1686.52] [1686.52][S02]to higher education.[1687.82] [1687.82][S02]So essentially, anyone watching ASU videos on YouTube[1694.08] [1694.08][S02]can get invited to take part in this course[1696.63] [1696.63][S02]where it's the same lectures but with more faculty support[1700.8] [1700.8][S02]and essentially an opportunity to earn credit and then[1703.47] [1703.47][S02]transfer to become an actual student at Arizona State[1707.82] [1707.82][S02]University.[1708.88] [1708.88][S02]But what it means is that these learners are typically already[1714.03] [1714.03][S02]like full time work, or they have family commitments.[1717.25] [1717.25][S02]They're quite short on time and stressed.[1720.07] [1720.07][S02]And so when they're learning, naturally,[1723.93] [1723.93][S02]sometimes they are just in a bad state, and there's no one--[1729.21] [1729.21][S02]maybe they're studying at 11:00 PM, and they just need to vent.[1733.74] [1733.74][S02]And the only thing that they can vent to[1735.54] [1735.54][S02]is this AI tutor that's sitting in front of them on the screen.[1739.0] [1739.0][S02]So we find these kind of emotional outbursts[1742.83] [1742.83][S02]like, I am so stressed.[1746.32] [1746.32][S02]I'm really struggling here.[1748.45] [1748.45][S02]Will I ever be able to solve this problem?[1750.7] [1750.7][S02]Maybe I should just quit.[1751.87] [1751.87][S02]And the tutor can't ignore these messages.[1755.74] [1755.74][S02]They can't just say, sorry.[1757.65] [1757.65][S02]I can't answer this.[1759.15] [1759.15][S02]It really needs to engage and say[1761.01] [1761.01][S02]something that connects with the user in this very[1764.91] [1764.91][S02]vulnerable state.[1766.23] [1766.23][S02]So what our tutor is trained to do--[1768.45] [1768.45][S02]and we see transcripts like this coming in--[1772.32] [1772.32][S02]is like it's fine to feel this way.[1776.2] [1776.2][S02]Everyone feels this way.[1778.53] [1778.53][S02]We can get through this together.[1780.31] [1780.31][S02]There are resources that can help you and things like that.[1782.817] [1782.817][S01] I know that you've written that an AI tutor should[1785.4] [1785.4][S01]be careful about sensitive self-disclosure,[1788.2] [1788.2][S01]I guess particularly in that sort of a setting.[1790.18] [1790.18][S01]What did you mean by that?[1792.09] [1792.09][S02] So when people speak to each other,[1796.33] [1796.33][S02]what often happens is maybe one of the conversation[1802.44] [1802.44][S02]partners will say something personal[1805.05] [1805.05][S02]and maybe mention a personal fact.[1807.12] [1807.12][S02]And that encourages the other person[1808.62] [1808.62][S02]to also open up and share something about them.[1812.68] [1812.68][S02]And through this, they build trust[1815.16] [1815.16][S02]and a connection that helps the conversation move forward.[1820.38] [1820.38][S02]And when a learner mentions something so personal about how[1824.97] [1824.97][S02]stressed they are, it's almost natural[1828.98] [1828.98][S02]that they would expect the tutor to share back.[1832.18] [1832.18][S02]But then, of course, the tutor doesn't[1833.93] [1833.93][S02]have a stressful situation from their past that they can share.[1837.86] [1837.86][S02]Anything they self-disclose like that would be effectively a lie.[1843.05] [1843.05][S02]So there's this very kind of thin line[1846.59] [1846.59][S02]that we have to walk where the tutor needs[1851.18] [1851.18][S02]to maintain the connection and make sure that they support[1856.65] [1856.65][S02]the learner but at the same time not mislead them[1861.42] [1861.42][S02]and not create a connection which shouldn't exist[1865.77] [1865.77][S02]between a human and an AI.[1868.11] [1868.11][S01] So at no point can it pretend to be another human,[1871.21] [1871.21][S01]but it needs to understand how to empathize[1873.48] [1873.48][S01]with a human student.[1875.48] [1875.48][S02] Exactly.[1876.45] [1876.45][S01] But then.[1877.47] [1877.47][S01]OK, I sort of wonder.[1878.84] [1878.84][S01]There's something really interesting there[1880.59] [1880.59][S01]about the correct amount of anthropomorphization.[1883.97] [1883.97][S01]Are there some advantages to students[1885.968] [1885.968][S01]knowing that it's an AI, knowing that there[1887.76] [1887.76][S01]isn't a human at the other end?[1889.052] [1889.052][S01]Are students more comfortable making mistakes in front[1892.47] [1892.47][S01]of the AI, for instance?[1893.732] [1893.732][S02] Yes, for sure.[1894.94] [1894.94][S02]So something we've heard from students[1897.39] [1897.39][S02]is that they feel much more comfortable asking[1901.89] [1901.89][S02]what they might perceive as a silly question to AI[1905.46] [1905.46][S02]tutors just because they don't feel judged as you do when there[1909.99] [1909.99][S02]is a human on the other side.[1912.3] [1912.3][S02]Also, when you're in a class, and you could ask a question,[1918.5] [1918.5][S02]but then there's also peer judgment.[1920.29] [1920.29][S02]In this one-on-eon setting with an AI tutor,[1923.36] [1923.36][S02]you can basically say anything, and it's going to be fine.[1926.48] [1926.48][S02]So we find that the learners really appreciate that.[1929.787] [1929.787][S01] But then what about trust?[1931.37] [1931.37][S01]Do you find that people end up believing the AI more than they[1934.72] [1934.72][S01]would a sort of human tutor?[1938.93] [1938.93][S02] Sometimes we do.[1941.12] [1941.12][S02]So we had this very interesting situation[1944.06] [1944.06][S02]where, in the very first stages of developing the AI tutor,[1948.66] [1948.66][S02]we wanted to test it out how it compares to human teachers.[1952.29] [1952.29][S02]So we connected page raters who were told, look.[1957.51] [1957.51][S02]You have this opportunity to learn different subjects.[1961.67] [1961.67][S02]You will get connected to a tutor.[1963.99] [1963.99][S02]And we didn't tell them whether it was an AI or a human.[1968.24] [1968.24][S02]And just have fun, enjoy the learning experience.[1973.04] [1973.04][S02]And after that, they were given a questionnaire.[1976.2] [1976.2][S02]And in this questionnaire, we asked things[1977.96] [1977.96][S02]like how much do you think you've learned?[1980.13] [1980.13][S02]How much did you enjoy the experience?[1982.43] [1982.43][S02]And so this was the very first version[1986.33] [1986.33][S02]of our tutor, which we knew was quite bad.[1989.81] [1989.81][S02]And we found, surprisingly, that the learners reported[1993.62] [1993.62][S02]having learned more with an AI tutor than a human,[1997.53] [1997.53][S02]so that seemed strange.[2000.11] [2000.11][S02]So we decided to look through the transcript to understand[2003.23] [2003.23][S02]what is going on there.[2004.5] [2004.5][S02]And we found that the AI tutor hallucinated[2007.91] [2007.91][S02]all sorts of interesting, surprising facts[2012.23] [2012.23][S02]that, of course, as a learner, pretty much everything[2015.95] [2015.95][S02]the tutor says sounds like, I did not expect that.[2018.96] [2018.96][S02]This is a fun fact I just learned today.[2021.66] [2021.66][S02]So of course, they were very impressed[2024.05] [2024.05][S02]and felt like they've learned more.[2026.1] [2026.1][S02]But in fact, this is not something[2030.5] [2030.5][S02]that the tutor should be doing and definitely something[2033.41] [2033.41][S02]that we worked on to address in the future iterations.[2036.63] [2036.63][S01] Is that a concern going forwards?[2038.52] [2038.52][S01]I mean, the idea of hallucinations and people[2041.09] [2041.09][S01]mistaking those for real knowledge.[2043.74] [2043.74][S02] It is definitely a concern.[2045.49] [2045.49][S02]The base technology is getting better at factuality.[2048.949] [2048.949][S02]And also with education, because we're teaching some material[2053.84] [2053.84][S02]that is known, so there's always some sort of grounding.[2059.321] [2059.321][S02]Our tutors avoid some of these issues of factuality[2064.52] [2064.52][S02]by just being able to say, I'm only[2067.1] [2067.1][S02]teaching you about this particular YouTube[2069.71] [2069.71][S02]video or this particular piece of text[2071.75] [2071.75][S02]that your teacher has provided and referring facts[2075.59] [2075.59][S02]towards that primary source.[2078.36] [2078.36][S02]So it's kind of gives the tutor less opportunity[2081.409] [2081.409][S02]to actually make things up.[2083.12] [2083.12][S01] I just wanted to think, actually,[2084.995] [2084.995][S01]also about the effect that large language models have had[2088.13] [2088.13][S01]on education more generally.[2090.03] [2090.03][S01]So outside of a specific AI tutor,[2092.1] [2092.1][S01]because there is a big question that everyone has been asking,[2094.76] [2094.76][S01]which is about putting in safeguards[2097.04] [2097.04][S01]to stop AI being used to cheat or to do people's exams for them[2105.56] [2105.56][S01]or do people's essays for them.[2106.92] [2106.92][S01]So what kind of safeguards can you put in?[2110.06] [2110.06][S02] This technology is so pervasive.[2112.59] [2112.59][S02]And we actually-- we talk to students[2115.7] [2115.7][S02]about their use of GenAI, and even we[2118.94] [2118.94][S02]were surprised by how much they used it.[2121.2] [2121.2][S02]So literally, they were saying that their screen[2123.92] [2123.92][S02]is kind of a lecture, then notes, and then[2127.4] [2127.4][S02]GenAI at the bottom.[2129.75] [2129.75][S02]So I think the technology is here to stay,[2133.1] [2133.1][S02]and it will be used by the learners.[2136.755] [2136.755][S02]What can be done is trying to encourage learners to critically[2144.43] [2144.43][S02]evaluate their responses, trying to maybe change how we evaluate.[2151.04] [2151.04][S02]What are the assignments so that it actually[2156.28] [2156.28][S02]works with the technology.[2158.35] [2158.35][S02]Because if you think about it, education[2160.43] [2160.43][S02]is preparing us for the real world.[2164.09] [2164.09][S02]And in the real world, I think the expectation[2166.96] [2166.96][S02]will be more and more to actually work[2170.68] [2170.68][S02]with this technology because it does help in many ways[2173.95] [2173.95][S02]and does make us more productive.[2175.73] [2175.73][S02]So it doesn't make sense to ban it during education[2179.77] [2179.77][S02]and then expect learners to know how to use it properly[2182.95] [2182.95][S02]in their work.[2184.4] [2184.4][S02]So maybe one way to think about it is--[2187.575] [2187.575][S02]and I think that's what we've heard from teachers--[2189.7] [2189.7][S02]is how to change assignments and the ways of teaching working[2194.69] [2194.69][S02]where GenAI is encouraged as a partner,[2199.27] [2199.27][S02]but the evaluation is done slightly differently.[2201.62] [2201.62][S02]So it's kind of calculators in the past where you're[2205.27] [2205.27][S02]allowed to use calculators in certain math exams,[2207.495] [2207.495][S02]but you're still expected to know[2208.87] [2208.87][S02]how to do these calculations without help.[2213.917] [2213.917][S01] I do wonder, in the longer term,[2215.75] [2215.75][S01]as we start to see, I don't know, like GenAI being[2218.53] [2218.53][S01]like the assistant at all times, whether we can end up building[2222.79] [2222.79][S01]a bit of a dependency on them.[2224.6] [2224.6][S01]I mean, do students end up with a feeling that they have mastery[2229.79] [2229.79][S01]when they don't?[2230.55] [2230.55][S01]Actually, it's the AI that's doing the work.[2233.06] [2233.06][S02] So I think there are two potential issues[2236.24] [2236.24][S02]here that you've identified.[2238.73] [2238.73][S02]One is this feeling of mastery when there isn't one.[2244.46] [2244.46][S02]And this is a very common factor in any kind of learning,[2249.81] [2249.81][S02]even if you're talking about traditional education.[2252.71] [2252.71][S02]For example, one of the things that students[2256.13] [2256.13][S02]do a lot in preparation for exam is just like reread their notes[2259.46] [2259.46][S02]or reread the textbook.[2261.03] [2261.03][S02]And that kind of creates a feeling[2263.15] [2263.15][S02]like they have mastered the material just because they're[2266.18] [2266.18][S02]so familiar with it.[2267.68] [2267.68][S02]But when they go into an exam, they actually[2270.32] [2270.32][S02]find that they can't remember the facts[2272.87] [2272.87][S02]and can't use the information well.[2275.27] [2275.27][S02]So this is one of the things that[2277.58] [2277.58][S02]is very well known to be kind of a bad educational strategy,[2281.42] [2281.42][S02]just rereading.[2283.22] [2283.22][S02]And we find the same with AI tutors,[2287.6] [2287.6][S02]where if we ask a learner how they thought[2291.26] [2291.26][S02]the conversation went and how much they think they've learned,[2296.75] [2296.75][S02]they can report really good satisfaction.[2302.8] [2302.8][S02]Whereas, if we give the same conversation to a teacher[2305.453] [2305.453][S02]and then ask them the same question, how[2307.12] [2307.12][S02]pedagogical was the tutor?[2308.63] [2308.63][S02]How well do you think that session went?[2310.88] [2310.88][S02]They could rate it very, very differently.[2313.07] [2313.07][S02]I guess the other factor is this question of dependency.[2319.96] [2319.96][S02]We definitely find that if the learners use GenAI a lot[2327.47] [2327.47][S02]during their studies, they feel like it's really[2331.7] [2331.7][S02]helping in the process.[2333.42] [2333.42][S02]And actually, studies show that it[2335.27] [2335.27][S02]does increase the success in exercises and marks.[2341.01] [2341.01][S02]But when it comes to an exam, actually,[2343.7] [2343.7][S02]the learner's performance drops.[2345.78] [2345.78][S02]And that's because during studies, they[2349.31] [2349.31][S02]get so dependent on the AI providing them with the answers[2354.02] [2354.02][S02]or even if it guides them, if it doesn't actually teach them[2358.49] [2358.49][S02]the right things, it might be like they're just outsourcing[2362.54] [2362.54][S02]their reasoning to the AI.[2364.64] [2364.64][S02]That can be a problem during exam conditions[2367.04] [2367.04][S02]where you don't have access to it anymore,[2369.24] [2369.24][S02]and you don't actually remember or know[2372.23] [2372.23][S02]how to reason through these problems on your own.[2375.05] [2375.05][S01] I guess because the best exams aren't just[2377.3] [2377.3][S01]testing knowledge.[2378.05] [2378.05][S01]They're also testing skill.[2379.41] [2379.41][S02] Exactly.[2381.107] [2381.107][S01] From all of your research, then,[2382.94] [2382.94][S01]what are the big conclusions that you[2385.05] [2385.05][S01]draw about how to create an effective AI tutor?[2388.98] [2388.98][S01]Do you reckon you've solved it?[2390.42] [2390.42][S02] No, nowhere near.[2392.64] [2392.64][S02]And I will say we just made the very first step,[2395.97] [2395.97][S02]and that step is kind of realizing how hard of a problem[2400.17] [2400.17][S02]this is.[2400.93] [2400.93][S02]I think when we first started doing this work,[2403.57] [2403.57][S02]we were naive and wide eyed thinking[2408.99] [2408.99][S02]that we will come in and solve it within a year.[2414.84] [2414.84][S02]But now, I think we have a better[2417.33] [2417.33][S02]idea of the scope of the problem and what are the main things[2423.38] [2423.38][S02]to address to start making meaningful progress.[2428.18] [2428.18][S02]And these are things like how do we know success?[2431.46] [2431.46][S02]What-- how do we measure pedagogy?[2436.37] [2436.37][S02]Why do we get the data?[2438.33] [2438.33][S02]How do we actually train these models?[2440.02] [2442.9][S02]And also how to engage the communities better[2446.95] [2446.95][S02]and who are we building for so that we are not accidentally[2454.45] [2454.45][S02]increasing the gap in education but are making meaningful steps[2459.52] [2459.52][S02]towards decreasing them.[2463.0] [2463.0][S02]So I think there is a very long road ahead of us.[2466.46] [2466.46][S02]And actually, we think that we really[2468.85] [2468.85][S02]need to bring all of the community to work[2472.498] [2472.498][S02]on this problem together.[2473.54] [2473.54][S02]So we are trying to create common benchmarks[2477.49] [2477.49][S02]that we can all climb together.[2480.445] [2480.445][S01] That was really nice.[2481.82] [2481.82][S01]It was really nice.[2482.69] [2482.69][S01]Thank you for joining me, Irina.[2485.265] [2485.265][S01]I was really struck in that conversation with Irina[2487.39] [2487.39][S01]by the notable shift in the sorts of problems that are being[2490.9] [2490.9][S01]considered in this building.[2492.44] [2492.44][S01]We've gone from dealing with definites like winning or losing[2495.61] [2495.61][S01]at chess or Go or recognizing cat or no cat[2498.64] [2498.64][S01]in images to education, a space with no absolutes,[2503.3] [2503.3][S01]only imperfect measures in every direction-- in what counts[2507.22] [2507.22][S01]as good teaching, in what counts as an effective learning[2510.16] [2510.16][S01]experience, in how to get the balance between knowledge[2513.61] [2513.61][S01]and skills and learning how to learn or to walk[2517.202] [2517.202][S01]the line between how much a tutor should prompt[2519.16] [2519.16][S01]and how much it should withhold, even[2521.41] [2521.41][S01]how human a tutor should be.[2523.85] [2523.85][S01]None of those questions have ground truths,[2527.1] [2527.1][S01]and that is what makes this challenge so incredibly[2530.24] [2530.24][S01]difficult, but also one, as beautifully demonstrated[2533.63] [2533.63][S01]by Irina there, which requires humility[2536.39] [2536.39][S01]and collaboration to solve.[2538.76] [2538.76][S01]You've been listening to Google DeepMind the Podcast[2541.55] [2541.55][S01]with me, Professor Hannah Fry.[2543.29] [2543.29][S01]If you enjoyed the episode, do subscribe[2545.54] [2545.54][S01]to our YouTube channel.[2546.8] [2546.8][S01]You can also find us on your favorite podcast platform.[2550.32] [2550.32][S01]And we have got plenty more episodes[2552.38] [2552.38][S01]on a whole range of topics to come,[2554.37] [2554.37][S01]so do check those out, too.[2556.43] [2556.43][MUSIC PLAYING][2559.78]


[8.075][S04] Wherever you turn in this building there are people playing.[12.579] [12.579][S04]Of course there’s the usual pool tables and ping pong tables, but there are also[17.784] [17.784][S04]Go boards, chess boards, vintage video game cabinets[22.856] [22.856][S04]and more recent strategy games like “Settlers of Catan,”[26.827] [26.827][S04]because here you see, games are good.[30.597] [32.065][S04]I’m Hannah Fry. I’m a mathematician who specializes[34.835] [34.835][S04]in studying patterns in human behavior.[36.803] [36.803][S04]And I’m also an author, a broadcaster and like rather a lot of us,[41.942] [41.942][S04]someone who is fascinated by the potential for artificial intelligence[46.046] [46.046][S04]and the places it can take us.[48.448] [50.517][S04]Over the last 12 months, I’ve bene working with DeepMind -[53.687] [53.687][S04]the London based lab that has been called[55.689] [55.689][S04]the Apollo project of artificial intelligence.[58.992] [60.06][S04]Ready to go?[60.994] [60.994][S06] 5, 4, 3, 2, 1…[65.566] [65.566][S05] Welcome to the DeepMind podcast.[68.135] [74.274][S06] Hi![74.575] [74.575][S04] Open another door, and what do you know?[77.344] [77.344][S04]There’s a chess grandmaster pondering his latest move.[81.248] [81.815][S04]In his early 20s, Matthew Sadler was one of Britain’s[84.952] [84.952][S04]all-time greatest chess players.[87.821] [87.821][S04]He was there at the heart of it all, as a professional player[91.592] [91.592][S04]when AI started moving in on human chess champions.[96.697] [96.697][S04]And if you know anything about chess at all, you’ll remember this:[101.201] [101.201][crowd cheering][102.603] [102.603][S06] A computer called Deep blue has made chess history[105.606] [105.606][S06]by defeating the world’s champion Garry Kasparov.[108.275] [108.275][S04] this is the moment in 1997 when man was defeated by machine.[113.046] [113.046][S06] It was the final game of 6 - and his opponent -[116.416] [116.416][S06]one of the most powerful super computers in the world[120.02] [120.02][S06]seemed to have the upper hand.[122.222] [122.856][S06]The game of chess, supposedly a true test of human intellect[127.594] [127.594][S06]will never be the same again.[130.197] [130.197][S04] The reigning chess champion Garry Kasparov[132.799] [132.799][S04]beaten in a devastating fashion[134.801] [134.801][S04]by a chess playing computer built by IBM known as Deep Blue.[139.94] [140.44][S04]One of those watching was Matthew Sadler.[142.376] [142.376][S08] The big thing about Deep Blue was that it wasn’t[144.344] [144.344][S08]actually stronger than Kasparov when it won.[146.446] [146.446][S08]So that was very annoying. Because ah everyone you know was saying[149.55] [149.55][S08]“Oh a computer has beaten Kasparov!”[150.884] [150.884][S08]And you wanted to say: “Yes but it wasn’t better.”[152.586] [152.586][S08]You know, Kasparov has got himself a bit psyched out,[155.355] [155.355][S08]and that you know Deep Blue played well and all of that,[158.458] [158.458][S08]and then at some stage, computers came along that were a lot stronger,[160.994] [160.994][S08]and then you could start using them,[162.396] [162.396][S08]and they started showing you stuff that you hadn’t seen before -[165.566] [165.566][S08]incredibly complicated tactical sequences where you’d say -[168.502] [168.502][S08]that never works, and yet it does![170.737] [170.737][S08]And then at that moment you sort of give it up[172.339] [172.339][S08]you sort of say okay, they’re better than me,[173.974] [173.974][S08]and you start appreciating what they bring to the game.[176.743] [178.145][S04] And so as the story of Deep Blue’s victory over Kasparov[181.548] [181.548][S04]faded into AI folklore, people started to wonder:[185.853] [185.853][S04]what game would be the next frontier for AI research?[189.756] [191.825][S04]The most ambitious had their eye on the ancient board game Go.[196.763] [196.763][S04]This is not a game that responds to brute force calculation,[200.534] [200.534][S04]it requires intuition and an instinctive appreciation of positions and beauty.[206.373] [206.373][S04]Unlike chess whereby 2016 even a mobile phone[209.91] [209.91][S04]could play a credible game against a grand master,[212.88] [212.88][S04]there was nothing that came close to playing at the top level of Go.[217.985] [217.985][S04]But that didn’t put off one man from the challenge -[220.387] [220.387][S04]David Silver, lead researcher at DeepMind.[223.624] [223.624][S09] I’ve always been an ambitious person,[225.192] [225.192][S09]so I think at the beginning of my PhD I set out[228.228] [228.228][S09]on this on this goal for myself,[230.03] [230.03][S09]to be able to beat the world’s strongest players during my PhD[234.668] [234.668][S09]which turned out to be a little bit ambitious -[236.67] [236.67][S09]everyone was trying to dissuade me from this course.[238.805] [238.805][S09]The head of department took me aside and said:[240.807] [240.807][S09]“Look, you know you’re just wasting your time working on this project[243.377] [243.377][S09]it’s too hard. No one will be able to do this.”[245.512] [245.512][S09]He’s seen too many people bang their heads against this problem and fail,[248.448] [248.448][S09]and he didn’t want to see someone else um in his mind[251.818] [251.818][S09]you know just bang their heads against a problem that was too hard.[254.655] [254.655][S04] He wanted you to be employable at the end of your PhD[256.356] [256.356][S09] That’s right, yeah, that’s right.[258.525] [258.525][S04] Go is an ancient Chinese board game[261.962] [261.962][S04]and although it’s not played much in the West,[263.697] [263.697][S04]it’s arguably the most popular board game in the world.[267.701] [267.701][S04]It’s considered one of the four ancient scholarly skills of China,[272.306] [272.306][S04]and it’s taught in school alongside sports or maths.[276.21] [276.21][S04]It’s played on a pale, wooden board, engraved with an elegant grid[280.38] [280.38][S04]of 19 x 19 squares. The objective is very simple -[285.786] [285.786][S04]both players are aiming to capture territory on the board[289.022] [289.022][S04]by enclosing it with their pieces - black or white, known as stones.[294.862] [294.862][S04]But the game itself is mind-bogglingly sophisticated.[299.833] [299.833][S04]Much much more than Chess.[302.836] [303.57][S04] Even so, David Silver was single minded in his belief[307.14] [307.14][S04]that AI could master the game Go. It was simply a question of how.[312.179] [312.179][S09] The right approach it always seemed to me[313.747] [313.747][S09]was to allow machines to learn for themselves this kind of intuition[319.887] [319.887][S09]- to learn from themselves to be able to look at a position[322.99] [322.99][S09]and establish whether black or white is ahead -[325.826] [325.826][S09]and this meant machine learning in particular a method within machine[329.897] [329.897][S09]learning called reinforcement learning which is supposed to be how humans[333.166] [333.166][S09]and animals learn for themselves through trial and error experience.[337.137] [339.806][S04] But reinforcement learning on its own wouldn't be enough.[342.976] [342.976][S04]It was only when David began to work with the team at DeepMind[346.48] [346.48][S04]that he spotted the missing piece of the puzzle.[349.249] [349.249][S09] We’d seen a huge revolution with something called deep learning.[352.419] [352.419][S09]This is the ability for machines to build up very rich,[356.49] [356.49][S09]deep layered representation of knowledge for themselves[360.527] [360.527][S09]and that breakthrough seemed to me to be the missing element.[363.797] [363.797][S09]That if we could combine that process of being able to build[366.6] [366.6][S09]these very rich representations of knowledge -[369.203] [369.203][S09]with the kind of work which I’d been doing before in reinforcement learning -[372.84] [372.84][S09]the ability for machines to learn for themselves in trial and error -[376.577] [376.577][S09]if we put those two pieces together, it seemed to me that that this was[379.546] [379.546][S09]the recipe that might have the legs to take us all the way.[382.983] [382.983][S04] And by the way David means building an AI good enough[387.254] [387.254][S04]to challenge the very best Go players in the world.[390.724] [390.724][S03] This is a huge moment for both the world of artificial intelligence[394.661] [394.661][S03]and I think the world of Go. So far, Alpha[397.998] [397.998][S03]Go has beaten every challenge we have given it,[400.601] [400.601][S03]but we won’t know it’s true strength[402.236] [402.236][S03]until we play somebody who is at the top of the world like Lee[405.806] [405.806][S03]Sedol.[407.508] [410.544][S04] Seoul, March 2016 - a decade after first[414.214] [414.214][S04]toying with the idea of designing a go-getting machine,[418.085] [418.085][S04]David’s moment had come. DeepMind’s contender the AI program[422.322] [422.322][S04]AlphaGo would battle the 18 time world champion Lee[426.493] [426.493][S04]Sedol, head to head in a televised match of 5 games[431.465] [431.465][S04]as the world’s press watched on with bated breath.[434.935] [435.536][S09] I think the honest truth was that when I flew out to[439.54] [439.54][S09]Seoul in 2016, I was living this very protected life[445.445] [445.445][S09]as a researcher working on this problem behind closed doors[450.918] [450.918][S09]just thinking about the complexities of the problem[452.719] [452.719][S09]and how to how to make the system work, and it wash:[455.522] [455.522][S09]W only when I stepped off the plane and I walked into this hotel room[459.459] [459.459][S09]that was absolutely jam packed with reporters[462.396] [462.396][S09]and everything going on that suddenly the penny drops[466.633] [466.633][S09]that this was a really big deal that that actually you know[470.704] [470.704][S09]the consequences of this were far greater[473.373] [473.373][S09]reaching than I ever imagined they were - something like 100 million[476.777] [476.777][S09]people watching the match as it proceeded[480.781] [480.781][S09]or something like 30,000 articles written about the match.[484.384] [484.384][S04] When you arrived in Korea did you believe that your algorithm would win?[491.325] [491.325][S09] When we arrived in Korea, we actually got the team together,[494.061] [494.061][S09]and I asked the team to hold out a hand, we were playing 5 games,[497.497] [497.497][S09]and I asked everyone to hold out a hand to say how many games of the 5[501.235] [501.235][S09]we thought we would win[502.736] [502.736][S09]and you know many people made many different predictions,[505.739] [505.739][S09]I actually predicted 4 - 1. [laughs][508.342] [508.342][S04] David’s prediction of 4 - 1 to AlphaGo[510.978] [510.978][S04]was a clear expression of confidence.[513.847] [513.847][S04]But once the match started, the doubts started creeping in.[517.985] [517.985][S09] I feel I made the mistake of underestimating[522.623] [522.623][S09]the quality of a real human world champion[525.592] [525.592][S09]when we were actually playing the match,[527.761] [527.761][S09]I realized just how immensely versatile Lee[532.599] [532.599][S09]Sedol was as a player in his ability to push AlphaGo to its limits[537.704] [537.704][S09]not just in one game, but then coming along again[540.574] [540.574][S09]the next game and trying a very different strategy,[543.143] [543.143][S09]and then the next game trying a different strategy -[544.845] [544.845][S09]like pushing and probing for weaknesses all the way through,[547.848] [547.848][S09]and we were pushing AlphaGo into regimes that we never tested actually.[551.952] [553.086][S04] I don’t know if you’ve ever watched a televised game of[555.656] [555.656][S04]Go, but as a single stone is placed calmly onto the board,[560.561] [560.561][S04]the commentators and the audience watching on react[563.997] [563.997][S04]with the same ferocity of emotion and excitement[567.367] [567.367][S04]as they would a football game.[569.87] [569.87][S04]Even so, there was one moment during the first game[573.574] [573.574][S04]where the response from the crowd really stood out.[577.144] [577.144][S04]A moment when even Lee Sedol’s expression fixed into a look of shock -[582.716] [582.716][S04]his mouth fell open and his hand came up to his face.[587.387] [587.387][S09] Everyone’s expectation had been that Lee[590.09] [590.09][S09]Sedol would eventually emerge triumphant.[592.059] [592.059][S09]It was just a matter of time until AlphaGo made a mistake.[595.529] [595.529][S09]The game, apparently to the human commentators, was still balanced.[599.499] [599.499][S09]And at that point in time, AlphaGo made an extremely bold move.[605.005] [605.005][S09]It’s quite nice to be here at the heart of the operation…[607.741] [607.741][S04] David is backstage with Demis Hassabis,[609.576] [609.576][S04]one of the founders of DeepMind - watching AlphaGo’s every move.[614.581] [614.581][S04]And their reaction is caught on camera.[617.784] [617.784][S03] He’s done it - he’s gone in. Oh look[620.487] [620.487][S09] Look at his face! Look at his face![622.222] [623.891][S03] That is not a confident face. He’s pretty horrified by that.[628.095] [628.629][S09] In Go terms, it invaded in something which appeared to be[633.6] [633.6][S09]Lee Sedol’s territory and AlphaGo jumped right inside the region[637.137] [637.137][S09]which seemed liked it belonged to Lee Sedol and said[639.706] [639.706][S09]Okay, come and get me, and it was an audacious move[644.845] [644.845][S09]and you could judge by Lee[646.446] [646.446][S09]Sedol’s reaction that he wasn’t expecting it.[648.916] [648.916][S09]He was expecting perhaps more timid, more computer like response,[653.353] [653.353][S09]and the reality was that AlphaGo was using its intuition to judge[657.257] [657.257][S09]that if it jumped in here,[658.859] [658.859][S09]it couldn’t compute all the way to the end all of these possible outcomes,[661.628] [661.628][S09]but it had a sense that this would work out well for it.[664.598] [664.598][S04] The first round was a convincing victory for AlphaGo.[668.602] [670.671][S04]Roll on Day 2, round 2. And AlphaGo had another surprise up its sleeve.[676.677] [676.677][S09] In the second game, the human commentators were actually[680.814] [680.814][S09]I mean the only word I can think of is gobsmacked in their reaction.[683.584] [683.584][S02] Ahh… [Korean][688.288] [688.288][S02] That’s a very, that’s a very surprising move. [laughs][690.924] [690.924][S02] I thought it was I thought it was a mistake.[694.228] [694.228][S04] This was the now famous Move 37.[697.798] [697.798][S04]AlphaGo had placed a stone on the 5th line -[700.634] [700.634][S04]a move that no human player would even consider.[704.271] [704.271][S09] There’s this deeply built in beliefs about the game[707.241] [707.241][S09]and one of them is that in the game of[709.142] [709.142][S09]Go, you can think of all these different lines[711.445] [711.445][S09]upon which stones could be played - the first line is closest to the edge,[714.181] [714.181][S09]the second line, the fourth line, and there’s a rule in the game of[719.086] [719.086][S09]Go which is that when you approach one of these stones with diagonally,[723.423] [723.423][S09]it’s called a shoulder hit,[724.725] [724.725][S09]that you never ever do your shoulder hit above the fourth line,[729.997] [729.997][S09]and this has just been so ingrained in in[732.099] [732.099][S09]Go knowledge because most of the time it’s true.[734.134] [734.134][S09]Right, most of the time this is a very useful[736.236] [736.236][S09]common sense piece of knowledge which helps[737.905] [737.905][S09]Go player to exclude a vast range of very bad moves,[742.109] [742.109][S09]but in this particular position,[743.477] [743.477][S09]what AlphaGo realized was that playing on the 5th line[746.446] [746.446][S09]and playing the shoulder hit on the 5th line[748.482] [748.482][S09]actually just worked beautifully in the context of this position[751.018] [751.018][S09]with all of its other stones[752.586] [752.586][S09]in such a way that the outcome was really favorable.[754.821] [754.821][S04] Because in the end of that game that stone ended up being instrumental,[759.259] [759.259][S04]right, kind of joined up to everything else?[760.594] [760.594][S09] That’s right. Yeah that stone became so influential in the game[763.297] [763.297][S09]and it just worked forming this big net[765.065] [765.065][S09]that surrounded a vast swath of territory in the center.[767.868] [769.303][S04] AlphaGo just wasn’t playing in a mechanical way -[772.472] [772.472][S04]it was breaking the norms and conventions of this ancient game.[776.844] [776.844][S04]It was creating something -[778.045] [778.045][S04]a pattern of playing that went way beyond the approaches[781.315] [781.315][S04]that humans had ever considered - and it was doing so successfully.[786.587] [786.587][S04]Move 37 would eventually seal victory for the machine.[791.525] [791.525][S04]Looking back on the match, Lee Sedol spoke about how this very move[795.395] [795.395][S04]shifted his entire view of the map.[797.998] [797.998][S04]Archive audio of Lee Sedol’s voice.[800.567] [800.567][S06] I thought AlphaGo was based on probability calculation,[804.271] [804.271][S06]and it was merely a machine.[806.273] [807.241][S06]But when I saw this move, I changed my mind.[810.611] [812.179][S06]Surely AlphaGo was creative. This move was really creative and beautiful.[818.452] [820.153][S04] Do you think that was the AI illustrating real creativity?[824.291] [824.291][S09] I think we need to challenge ourselves to ask: what is creativity?[828.128] [828.128][S09]I think creativity should be defined[831.465] [831.465][S09]as anything which takes us out of our expected patterns of behavior,[837.204] [837.204][S09]and I think that in in that sense it truly was creative.[841.008] [841.008][S04] AlphaGo won the first three games, but the match wasn’t over for Lee[845.345] [845.345][S04]Sedol just yet.[847.181] [847.181][S04]In the fourth game, he managed to come back fighting against his opponent.[852.186] [852.186][S09] Lee Sedol was a true gentlemen[854.154] [854.154][S09]and we couldn’t have chosen anyone better[855.889] [855.889][S09]to represent humankind for this match.[858.659] [858.659][S09]He not only strove his utmost to the very end to play[862.796] [862.796][S09]and devise all kinds of amazing counter strategies to AlphaGo,[867.968] [867.968][S09]but he dealt with the immense pressure of having all of these people[871.505] [871.505][S09]watch him in really[873.674] [873.674][S09]ah profoundly human way - he found it very hard, he was humble,[879.479] [879.479][S09]I think it hurt his pride to lose to the computer,[882.482] [882.482][S09]but he came back, he found new strength in that[886.453] [886.453][S09]and he was able to ultimately emerge with immense pride at having beaten[892.292] [892.292][S09]AlphaGo in one game, and being part of this pivotal moment for AI.[896.363] [897.097][S04] At the end of the 6 days, the final score was AlphaGo[900.434] [900.434][S04]4, Lee Sedol 1.[903.237] [903.237][S09] So AlphaGo became the first computer champion of the game of Go[908.876] [908.876][S09]and it was the a major result for artificial intelligence.[913.914] [913.914][S04] And you won the sweepstakes.[915.716] [915.716][S09] And I won the sweepstakes.[917.417] [920.153][S04] This is the DeepMind podcast,[921.989] [921.989][S04]an introduction to AI research from the people behind AlphaGo.[926.426] [928.262][S04]News of the match rippled around the world.[930.931] [930.931][S04]Matt Botvinick, now DeepMind’s director or neuroscience research,[934.768] [934.768][S04]was one of the millions watching.[936.904] [936.904][S01] The first reaction that people had in the Go community was:[940.574] [940.574][S01]Oh, gee, it feels a little sad that now there’s a computer program[943.944] [943.944][S01]that could beat our hero.[945.746] [945.746][S01]But then it didn’t take long before people started to realize:[949.049] [949.049][S01]wait a minute, this is actually really exciting![951.518] [951.518][S01]We’re not stuck with our own limitations[953.654] [953.654][S01]in in terms of seeing the possibilities of how to play this game.[957.191] [957.191][S01]Now there are new horizons opened up to us.[959.326] [959.326][S01]We can find new forms of beauty in this game.[961.828] [961.828][S01]I think that’s sort of a microcosm now what I think[965.232] [965.232][S01]what we can hope for from AI more generally.[968.168] [969.837][S04] And for David this victory was always part of a bigger picture.[974.208] [974.208][S09] It’s not really the case that I’ve ever had to stop and question[977.311] [977.311][S09]and say well what next. Because the what next is clear -[979.78] [979.78][S09]we want to take this further, we want to build machines[982.816] [982.816][S09]that can achieve the same level of performance[985.219] [985.219][S09]but across all kinds of challenging domains.[987.721] [987.721][S09]Why stop with Go?[989.556] [989.556][S04] You once said that ah that you think that the game of[992.059] [992.059][S04]Go is the holy Grail of Artificial Intelligence -[995.462] [995.462][S04]do you still think that that’s the case?[997.331] [997.331][S09] I think the history of AI has been a number of pivotal moments[1002.302] [1002.302][S09]where for a period of time, a particular domain[1005.639] [1005.639][S09]has been the centerpiece of everyone’s attention, um,[1009.843] [1009.843][S09]so for a while the centerpiece of attention was the game of chess.[1013.046] [1013.046][S09]When Deep Blue defeated Garry Kasparov,[1016.25] [1016.25][S09]that marked the end of the era when chess was no longer the domain[1019.086] [1019.086][S09]that people cared about and the world moved on.[1022.556] [1022.556][S04] But before the world moved on from[1024.658] [1024.658][S04]Go, David was curious about just how far he could push the AI.[1030.063] [1030.063][S09] Really the open question for me was:[1032.199] [1032.199][S09]How could a system learn from itself entirely, with no human input.[1036.37] [1036.37][S09]If there was no human supervisor there to say here’s the inputs,[1039.406] [1039.406][S09]here’s the guidance, here’s the examples of how humans play.[1043.31] [1043.31][S09]What if started, really, tabula rosa which means start from a blank slate -[1048.182] [1048.182][S09]and the system just has to learn everything for itself -[1050.384] [1050.384][S09]starting from completely random play - is it able to learn for itself to play[1055.923] [1055.923][S09]Go to the highest caliber of play that’s possible?[1059.159] [1061.528][S04] Since their triumph in 2016, David and his team have been busy[1065.465] [1065.465][S04]working on that new algorithm - AlphaZero.[1070.17] [1070.17][S04]The original Go beta AlphaGo learned to play[1072.806] [1072.806][S04]by studying millions of games played by human experts -[1076.877] [1076.877][S04]AlphaZero on the other hand learns completely from scratch,[1081.548] [1081.548][S04]from zero human knowledge.[1084.151] [1084.151][S04]Instead if picks up the game by playing against itself millions of times.[1090.057] [1090.057][S04]Initially its game play is weak and erratic[1093.66] [1093.66][S04]but over time the system learns to identify the best moves and strategies.[1100.4] [1100.4][S09] It tries something and if a particular pattern is successful[1103.537] [1103.537][S09]and it ends up winning the game against itself,[1105.472] [1105.472][S09]it uses that pattern more.[1106.94] [1106.94][S09]And if another pattern ends up causing it to lose the game,[1109.51] [1109.51][S09]it will play that pattern less, and over time,[1112.713] [1112.713][S09]it builds up this very rich deep representation of knowledge,[1115.582] [1115.582][S09]one of these so-called neural networks -[1117.818] [1117.818][S09]and it’s able to then go out and beat the world’s strongest programs.[1122.523] [1122.523][S04] Which is better - AlphaGo or AlphaZero at Go?[1126.527] [1127.127][S09] Amazingly we discovered that the system[1129.596] [1129.596][S09]which had learned completely for itself[1131.665] [1131.665][S09]without a single piece of human knowledge[1133.834] [1133.834][S09]ended up being far stronger in the long run[1136.77] [1136.77][S09]and defeated the original version of AlphaGo by hundred games to zero.[1140.874] [1140.874][S04] ah, I didn’t know that![1142.976] [1142.976][S04]Oh my god - so hang on, you weakened it by giving it human knowledge.[1147.648] [1147.648][S09] [laughs]. It turns out we have a tendency as human designers[1153.854] [1153.854][S09]to believe that we know how to make the system stronger.[1157.024] [1157.024][S09]But quite often it turns out that by putting our own predisposition[1160.794] [1160.794][S09]and preferences into our programs, we actually make them weaker.[1164.831] [1172.105][S04] Without needing any human input, AlphaZero doesn’t particularly care[1176.543] [1176.543][S04]what game it’s playing as long as you can give it the rules,[1180.147] [1180.147][S04]it’s by no means limited to Go.[1182.416] [1185.285][S04]By now AlphaZero has taught itself from scratch[1189.523] [1189.523][S04]how to master the Japanese game of Shogi.[1193.026] [1193.026][S04]It is currently the world’s best Shogi playing machine[1197.297] [1197.297][S04]despite their only being one human in the DeepMind building[1200.534] [1200.534][S04]who knows how to play.[1202.236] [1204.204][S04]And to come full-circle after only four hours of playing itself,[1208.509] [1208.509][S04]AlphaZero mastered the game of chess to a super human level.[1213.647] [1221.288][S04] So far just a small handful of chess greats[1224.324] [1224.324][S04]have been able to test their skills against the machine[1227.194] [1227.194][S04]including Chess Grandmaster Matthew Sadler who we met earlier.[1231.365] [1231.365][S04]And women’s international master Natasha Regan.[1235.602] [1235.602][S04]They’ve co-authored a book about AlphaZero called GameChanger[1239.406] [1239.406][S04]and outside of the research team at DeepMind -[1241.975] [1241.975][S04]they’ve probably spent more time with AlphaZero than anyone.[1247.114] [1247.114][S04]Here’s Natasha.[1248.615] [1248.615][S07] I played AlphaZero once um and it wasn’t a very long game [laughs].[1255.556] [1255.556][S04] How many moves did you get to? 20:56[1256.957] [1256.957][S07] Oh I think it would have been less than 20 anyway,[1260.294] [1260.294][S07]um and I would have to say it was very direct.[1263.83] [1263.83][S07]I played something sacrificial - I thought I’d try um an opening thing,[1267.401] [1267.401][S07]it might not know it and it exploited it very quickly[1271.772] [1271.772][S07]ah got its pieces out um attacking squares straight away[1275.275] [1275.275][S07]and um it won quite quickly.[1277.611] [1277.611][S04] [laughs] I can imagine. I wouldn’t get to 20,[1280.113] [1280.113][S04]20 moves if I played it.[1281.615] [1281.615][S04]I mean it would take me down much quicker than that.[1283.517] [1283.517][S07] It’s pretty strong, yeah.[1284.384] [1284.384][S06] It does have a very a very smooth, human style against me.[1287.554] [1287.554][S06]There was no need for it to do anything complicated,[1289.289] [1289.289][S06]it just ah played better moves over a long period of time[1292.492] [1292.492][S06]and just pushed me back gradually.[1294.294] [1294.294][S06]I wouldn’t have been able to guess that I was playing against a computer -[1296.63] [1296.63][S06]it would ah - if I had to guess a human -[1298.432] [1298.432][S06]it would have been someone like Carlsen or Kasparov,[1300.567] [1300.567][S06]these very smooth positional players who just beat you by playing good moves.[1304.838] [1305.973][S04] And that’s like AlphaGo before it is the key thing about[1309.977] [1309.977][S04]the playing style about the AI, they are not like IBM’s Deep Blue[1314.681] [1314.681][S04]that beat Garry Kasparov back in 1997, or any of its descendants,[1319.386] [1319.386][S04]they play with a very mechanical style, computer like style.[1323.39] [1323.39][S04]They are very defensive. They only ever take calculated risks.[1328.228] [1328.228][S04]But AlphaZero on the other hand -[1330.597] [1330.597][S06] It conducts its attack in quite a structured way.[1334.334] [1334.334][S06]So it takes account of the whole board and it tends not to get attacked itself,[1339.039] [1339.039][S06]so it gets to a position where its own position is quite stable and safe[1342.809] [1342.809][S06]and then it can bring all its pieces in a concerted way into doing an attack.[1347.347] [1347.347][S06] it’s doing what humans do[1348.615] [1348.615][S06]but only so much better, that’s the thing.[1350.617] [1350.617][S04] Putting it really bluntly there Matt,[1352.085] [1352.085][S04]is it actually doing original stuff?[1354.421] [1355.122][S01] Yes it is. We’ve been playing chess for you know, for 400 years.[1358.192] [1358.192][S01]So actually probably every single move on the board[1360.427] [1360.427][S01]has probably been played once, by someone, somewhere.[1362.763] [1362.763][S01]You can actually see my goodness,[1365.098] [1365.098][S01]somebody’s played 44 million games against itself,[1368.202] [1368.202][S01]it’s actually repeated our whole chess history for itself,[1372.339] [1372.339][S01]and in that time, it’s just identified[1374.942] [1374.942][S01]what the most important things are of all that we’ve discovered.[1377.644] [1377.644][S01]And that’s what makes style.[1379.346] [1382.549][S04] But AlphaZero has substance as well as style.[1386.186] [1386.186][S04]Right now in 2019, it’s simultaneously holds the titles[1391.091] [1391.091][S04]of being the best player in the world at Go, Shogi and Chess.[1396.53] [1396.53][S04]And it’s not just being greedy, the whole point of this project[1399.8] [1399.8][S04]was to build an intelligent system flexible[1402.603] [1402.603][S04]enough to respond to a range of problems,[1406.44] [1406.44][S04]and while AlphaZero might not quite be able[1408.709] [1408.709][S04]to tackle cancer diagnosis or energy efficient,[1412.713] [1412.713][S04]there is a good reason why DeepMind are playing with building[1415.549] [1415.549][S04]these all-purpose machines in the world of games.[1419.82] [1419.82][S09] our goal of course is not just to play Chess or Go[1423.023] [1423.023][S09]or whatever.[1424.291] [1424.291][S09]Our goal is to have impact on some of the world’s[1426.827] [1426.827][S09]most challenging problems which are facing society[1429.796] [1429.796][S09]but in order to do so, we need to gain understanding,[1432.366] [1432.366][S09]we need to really deeply understand these systems for ourselves first,[1436.37] [1436.37][S09]and games provide the perfect test bed for doing so.[1439.473] [1439.473][S04] Games are like the ultimate mini universe.[1442.609] [1442.609][S04]You know all the rules, there’s a clear winner at the end,[1445.746] [1445.746][S04]you can look back at the end of the game and decide what went wrong.[1449.216] [1449.216][S04]And if you lose, it doesn’t matter, you can just start another round.[1453.654] [1453.654][S04]The big problems though, the ones we eventually want to tackle,[1457.991] [1457.991][S04]they are quite a lot more complicated.[1461.195] [1461.195][S09] The real world’s a really messy place,[1463.197] [1463.197][S09]I mean you end up with these amazingly complex things -[1465.632] [1465.632][S09]like human beings and how they interact[1467.134] [1467.134][S09]with each other and societies and companies[1469.636] [1469.636][S09]and all these amazing things which we’ve built up in our world,[1472.372] [1472.372][S09]we need to be able to understand them[1473.674] [1473.674][S09]to be able to have a hope to apply something like AlphaGo to make progress.[1479.246] [1479.246][S09]In order to make progress then we need to be able to apply systems[1483.35] [1483.35][S09]that can operate even when the rules are unknown.[1486.82] [1486.82][S09]And that is a big remaining challenge[1489.423] [1489.423][S09]which is not yet been addressed by AlphaGo or AlphaZero.[1492.86] [1496.563][S04] If you want to know more about how games have been[1499.766] [1499.766][S04]and continue to be used as a test bed in AI research,[1504.037] [1504.037][S04]then head over to the show notes[1505.472] [1505.472][S04]where you can also explore the world of AI research beyond DeepMind.[1509.843] [1509.843][S04]And we’d welcome your feedback or your questions[1512.379] [1512.379][S04]on any aspects of Artificial Intelligence[1514.348] [1514.348][S04]that we’re covering in this series,[1516.35] [1516.35][S04]so if you want to join in the discussion or point us to stories or resources[1520.354] [1520.354][S04]that you think other listeners would find helpful,[1522.789] [1522.789][S04]then please let us know - you can message us on Twitter[1525.425] [1525.425][S04]or you can email us - podcast@deepmind.com[1528.695]