[0.0][MUSIC PLAYING][4.293] [6.423][S01] Welcome back to "Google DeepMind--[8.34] [8.34][S01]The Podcast."[9.0] [9.0][S01]I'm Professor Hannah Fry.[10.53] [10.53][S01]Now there is broad consensus across the tech industry,[14.22] [14.22][S01]governments, and society that as AI becomes[17.18] [17.18][S01]more and more embedded in every aspect our world,[20.73] [20.73][S01]regulation is essential.[23.31] [23.31][S01]But once you start to decipher what that regulation should[26.18] [26.18][S01]actually look like, well, then agreement[28.73] [28.73][S01]becomes much more elusive.[31.08] [31.08][S01]How do we protect against the harms of new technologies[34.19] [34.19][S01]without stifling innovation?[36.63] [36.63][S01]How can we give AI the autonomy that it[39.17] [39.17][S01]needs to be able to solve complex problems while retaining[43.37] [43.37][S01]human control?[44.79] [44.79][S01]And is it possible to keep development[47.21] [47.21][S01]transparent and accountable to the public,[49.47] [49.47][S01]even as companies fiercely guard their competitive secrets?[53.61] [53.61][S01]Well, in our episode with Demis Hassabis[55.61] [55.61][S01]at the start of the season, we talked about his general views[58.91] [58.91][S01]on the regulation of AI.[60.54] [60.54][S01]But in this episode, I want to dig a bit deeper[63.23] [63.23][S01]into the nuances of this topic.[64.86] [64.86][S01]I want to take time to explore the arguments[67.76] [67.76][S01]and the counter-arguments to the different forms of regulation[71.45] [71.45][S01]that are on the table.[72.62] [72.62][S01]Now, of course, to address the elephant in the room,[75.9] [75.9][S01]this is the "Google DeepMind" Podcast.[78.36] [78.36][S01]So it's important to be upfront about the fact[80.81] [80.81][S01]that my guest today is going to be offering us[83.24] [83.24][S01]one particular view of AI regulation.[86.16] [86.16][S01]However, I want to promise you that as far as I possibly can,[90.15] [90.15][S01]I am going to try and make this a robust discussion,[93.32] [93.32][S01]and leave it up to you, the audience,[95.31] [95.31][S01]to decide where you stand on these issues.[97.83] [97.83][S01]And so with that in mind, I am delighted to be joined today[101.21] [101.21][S01]by Nicklas Lundblad, Google DeepMind's[103.49] [103.49][S01]Head of Public Policy and Public Affairs.[105.66] [105.66][S01]Nicklas has spent years working at the intersection[108.14] [108.14][S01]of technology and policy at Google,[110.52] [110.52][S01]navigating the intricate landscape of AI's evolution[114.08] [114.08][S01]and regulation.[115.26] [115.26][S01]Nicklas, welcome to the podcast.[116.64] [116.64][S02] Well, thank you so much for having me.[117.74] [117.74][S01] Of course.[118.657] [118.657][S01]So, OK, in terms of your current role at the moment,[121.22] [121.22][S01]I know that you spend a lot of time talking to industry[124.09] [124.09][S01]leaders, talking to governments, to the public,[127.72] [127.72][S01]about perceptions of AI.[130.509] [130.509][S01]How would you describe the current public mood, as it were,[134.99] [134.99][S01]around the subject?[136.01] [136.01][S02] I think it's hesitant,[137.71] [137.71][S02]but also hesitantly optimistic.[140.44] [140.44][S02]And then you find the people who are hesitantly pessimistic.[142.94] [142.94][S02]So it's almost like evenly cut, I think.[145.18] [145.18][S02]To some degree, there are a lot of people[146.973] [146.973][S02]who hope that this technology can help us solve some[149.14] [149.14][S02]of the knotty and complex problems[150.61] [150.61][S02]that we have encountered, climate[152.26] [152.26][S02]change, pandemics, other kinds of things[154.66] [154.66][S02]that we really need to deal with.[156.17] [156.17][S02]And there are a lot of people who[157.545] [157.545][S02]say, well, yes, it might be able to do that, but at what price?[160.28] [160.28][S02]And so you end up with these two different groups.[163.075] [163.075][S01] And where do you sit between those two?[165.2] [165.2][S02] I am hopeful.[166.67] [166.67][S02]I also think that this has, artificial intelligence to me,[169.76] [169.76][S02]has a very natural place in progress[172.18] [172.18][S02]overall, because what we're doing[174.7] [174.7][S02]is what we've done for a long time,[176.51] [176.51][S02]is that we have bought progress and welfare[178.57] [178.57][S02]at the price of complexity.[180.47] [180.47][S02]And as social complexity increases[182.77] [182.77][S02]massively, we need new ways of dealing[185.35] [185.35][S02]with it, hence technological and social innovation.[188.78] [188.78][S02]And so the way that artificial intelligence[190.9] [190.9][S02]fits into this very, very simple picture[192.79] [192.79][S02]is that it is a way of dealing with increased complexity[196.39] [196.39][S02]and allows the unlocking of further progress.[199.49] [199.49][S02]And to me, that's absolutely essential.[202.48] [202.48][S02]There are two really interesting views[204.7] [204.7][S02]about artificial intelligence, two stories that are in tension.[207.86] [207.86][S02]One of them is that we do this because we can,[210.325] [210.325][S02]the sort of Frankenstein story.[212.09] [212.09][S02]We shall become as gods.[213.41] [213.41][S02]That's what we're doing.[214.84] [214.84][S02]And that story has had its time.[216.675] [216.675][S02]And there are a lot of people who[218.05] [218.05][S02]are worried about that story.[219.62] [219.62][S02]The other story, which is as extreme, but on the other side[222.85] [222.85][S02]is, no, we do this because we must.[225.02] [225.02][S02]The kinds of problems we have are not[227.26] [227.26][S02]going to be solvable without this kind of technology.[230.63] [230.63][S01] I mean, on both of those,[232.63] [232.63][S01]I guess this underlying idea that there is no choice, really.[236.55] [236.55][S01]That we're marching forwards regardless of almost intention.[241.87] [241.87][S01]I mean, it's like we have to.[243.727] [243.727][S02] Oh, they're both extremes.[245.56] [245.56][S02]And I would not say that I think any one of them is right.[248.92] [248.92][S02]But I think it's always helpful to think about society[251.76] [251.76][S02]and think about politics as sort of a reasonable disagreement[254.82] [254.82][S02]between two extreme views and then[256.529] [256.529][S02]see where you can get to the middle, what[258.57] [258.57][S02]the optimal way of thinking about this is.[261.67] [261.67][S02]I think the story where we do this[264.24] [264.24][S02]because it's going to unlock progress has been undertold.[267.61] [267.61][S02]So that's why I'm more interested in that, perhaps.[269.928] [269.928][S01] Do you think there's been a shift in public mood?[272.47] [272.47][S01]I mean, with the explosion of generative AI into the scenes,[275.238] [275.238][S01]and now it's really, I think, a topic[276.78] [276.78][S01]that everybody is at the forefront[278.745] [278.745][S01]of a lot of people's minds.[279.87] [279.87][S01]Have you noticed a mood shift as a result?[282.123] [282.123][S02] Oh, there's been many mood shifts.[284.29] [284.29][S02]I mean, one of the things that happened[285.915] [285.915][S02]with almost all technologies is that you first[287.94] [287.94][S02]encounter an enormous sort of euphoria, almost where you go[291.21] [291.21][S02]like, oh, this is fantastic.[292.48] [292.48][S02]It's going to solve all my problems.[294.01] [294.01][S02]And then you sink into deep, deep depression because it[296.85] [296.85][S02]didn't.[297.49] [297.49][S01] The Gartner Hype Cycle.[298.77] [298.77][S02] It's the Gartner Hype Cycle,[300.687] [300.687][S02]or the way that Jose Borges put it in "The Library of Babel,"[304.24] [304.24][S02]where he said, there are finally, in this short story,[308.26] [308.26][S02]which is beautiful, he writes about how they find a library[310.95] [310.95][S02]with all of the possible permutations of the alphabet,[313.69] [313.69][S02]all possible books.[315.1] [315.1][S02]And he says, and they were so joyful[317.61] [317.61][S02]because they had found all possible books.[319.78] [319.78][S02]And then a few sentences later he writes,[321.88] [321.88][S02]and then they became very depressed[323.79] [323.79][S02]because they had found all possible books.[326.07] [326.07][S02]And that thing characterizes almost all technology cycles[330.18] [330.18][S02]as well, I think.[331.157] [331.157][S01] That we get excited first.[332.74] [332.74][S02] We get excited,[333.61] [333.61][S02]and then we go to the trough of disillusionment,[335.925] [335.925][S02]and to use your Gartner Hype Cycle,[338.11] [338.11][S02]and then to the plateau of reasonable expectations.[340.93] [340.93][S02]And I think generative AI, to some degree,[344.61] [344.61][S02]is probably going through exactly that hype cycle.[347.38] [347.38][S02]And we now see a lot of articles about,[349.2] [349.2][S02]where's the productivity growth?[351.06] [351.06][S02]Where is the economic boost that we hoped for?[353.13] [353.13][S02]Is the investment worth it?[354.63] [354.63][S02]And that is something that we saw around the internet as well.[357.66] [357.66][S02]We've seen it around computers.[359.25] [359.25][S02]We've seen it around all kinds of technologies[361.43] [361.43][S02]for the last, at least two centuries.[363.49] [363.49][S01] And we should expect to see it going forwards as[365.99] [365.99][S01]well, I guess, with the new technologies that are to come?[368.58] [368.58][S02] Yeah, absolutely.[370.038] [370.038][S01] OK.[370.73] [370.73][S01]All right, I guess it might make sense for us[373.58] [373.58][S01]just at the beginning of this to really sketch out[375.722] [375.722][S01]where we are at the moment in terms[377.18] [377.18][S01]of the landscape of regulation on artificial intelligence.[380.19] [380.19][S01]Could you give us a brief overview of where we are?[382.8] [382.8][S02] Of course.[383.967] [383.967][S02]No, so we're currently in a land of many different kinds[387.17] [387.17][S02]of solutions being tried.[388.68] [388.68][S02]So in the US, for example, the Biden administration[391.91] [391.91][S02]decided to put forward what it's called an executive order, which[395.42] [395.42][S02]is sort of a presidential decree in which they set out[398.12] [398.12][S02]a lot of different rules for the different agencies.[400.92] [400.92][S02]They wanted guidelines.[402.33] [402.33][S02]They wanted testing.[403.77] [403.77][S02]They had a ton of different rules[405.47] [405.47][S02]that they set out around artificial intelligence,[407.7] [407.7][S02]but they were mostly sort of industry-led[410.45] [410.45][S02]and they were industry close.[412.02] [412.02][S02]And then you have the UK that has[413.54] [413.54][S02]said that we believe that the best way to approach[415.85] [415.85][S02]this is to look at sectoral legislation, which is, how is[419.81] [419.81][S02]this actually going to be used when it's used in healthcare,[422.965] [422.965][S02]when it's used in education, when[424.34] [424.34][S02]it's used in different kinds of sectors in the economy.[427.64] [427.64][S02]And then you have the European Union[429.14] [429.14][S02]that shows this broad horizontal regulatory approach in which[432.89] [432.89][S02]they said, we are going to look at the risk[435.32] [435.32][S02]that these systems present widely to society.[438.66] [438.66][S02]And then, of course, you have China who[441.47] [441.47][S02]has been regulating as well.[442.92] [442.92][S02]And the way they have been regulating[444.462] [444.462][S02]has been mostly around figuring out[446.12] [446.12][S02]how does this technology shift power in a society, which[449.63] [449.63][S02]is really interesting.[450.63] [450.63][S02]So if you look at the kind of regulation[452.297] [452.297][S02]that China has put in place, it has been around a lot[454.82] [454.82][S02]about information power.[456.6] [456.6][S02]So what do we do with recommendation algorithms,[459.36] [459.36][S02]for example.[459.9] [459.9][S02]That's been one of their first focuses, foci.[462.48] [462.48][S02]And so I think that's an interesting approach as well.[466.11] [466.11][S02]So you have these four different ongoing experiments[469.46] [469.46][S02]in the regulatory space.[470.88] [470.88][S02]And we're learning from each and every one of them.[473.01] [473.01][S02]And I think it's actually not a horrible approach[475.85] [475.85][S02]to a new technology because there's so much evidence that we[479.9] [479.9][S02]need to gather, not just about the technology,[482.25] [482.25][S02]but also about the regulatory fit of these different kinds[485.18] [485.18][S02]of rules.[486.06] [486.06][S01] So, OK, I'm going to ask quite a basic question,[490.04] [490.04][S01]a sort of a foundational one, as it were.[492.8] [492.8][S01]Do you think that AI, as a distinct technology,[496.91] [496.91][S01]needs regulating sort of independently[500.12] [500.12][S01]of other technologies?[502.25] [502.25][S02] I don't think that we ever[504.56] [504.56][S02]only regulate the technology.[507.15] [507.15][S02]And the view I have is quite sociotechnical.[509.07] [509.07][S02]I think we regulate the uses of technology,[511.26] [511.26][S02]the design of technology, the deployment of technology.[514.23] [514.23][S02]But I don't think that you can talk[515.69] [515.69][S02]about regulating a technology, purely the technology itself.[519.69] [519.69][S02]And that is as it should be, because[521.539] [521.539][S02]to some degree, what we are regulating[523.58] [523.58][S02]is also how a technology affects power.[527.05] [527.05][S02]The German philosopher Hans Jonas[528.61] [528.61][S02]had a beautiful way of phrasing this, where he said,[530.78] [530.78][S02]all use of technology is the exercise of power.[533.45] [533.45][S02]And so the use of technology is what[535.84] [535.84][S02]we're usually thinking about when we think about regulation.[538.78] [538.78][S02]Another thing that I think is quite important to bring[541.87] [541.87][S02]into our discussion here is that regulation is not the same[545.26] [545.26][S02]as legislation.[546.7] [546.7][S02]Those are different.[547.94] [547.94][S02]One of the best regulatory models that has come up[550.18] [550.18][S02]in the last couple of decades is one that was launched[553.18] [553.18][S02]by Professor Lawrence Lessig in a book that he wrote back[556.36] [556.36][S02]in 1999, I want to say, called "Code and Other Laws[560.11] [560.11][S02]of Cyberspace."[561.23] [561.23][S02]And in this book, he sort of outlines regulation[563.98] [563.98][S02]as consisting of four different components.[566.9] [566.9][S02]One is, of course, law, legislation.[569.24] [569.24][S02]We have laws and they regulate.[571.31] [571.31][S02]Another is architecture.[573.23] [573.23][S02]The way we actually build technology matters.[576.17] [576.17][S02]A third is markets because economic pressures also[579.73] [579.73][S02]regulate technology.[580.91] [580.91][S02]Some things are possible.[582.14] [582.14][S02]Some things are not possible.[583.43] [583.43][S02]And then lastly, the fourth is norms.[585.68] [585.68][S02]So these four forces of regulation impact the situation.[590.33] [590.33][S02]And this will become important later because when[592.54] [592.54][S02]we talk about regulating a technology, or regulating AI,[595.82] [595.82][S02]we can't just say we should regulate AI without also[599.14] [599.14][S02]discussing, OK, how does that regulation distribute[601.63] [601.63][S02]across these four different forces?[603.088] [603.088][S01] I want to pick up on something[604.838] [604.838][S01]you said a minute ago, actually, because you said that when[607.69] [607.69][S01]it comes to regulation, we regulate[609.455] [609.455][S01]the applications of technology rather than the technology[611.83] [611.83][S01]itself.[612.5] [612.5][S01]But I mean, that's not always true.[615.16] [615.16][S01]I'm thinking here about CRISPR, for instance, the gene editing[617.95] [617.95][S01]technology, which has regulations[620.44] [620.44][S01]on the technology itself, sort of independent[623.143] [623.143][S01]of its applications because of the recognition[625.06] [625.06][S01]of the potential harm to ecosystems,[627.76] [627.76][S01]or I don't know, to human health or human life[631.21] [631.21][S01]if you allow this kind of free use of the technology.[633.428] [633.428][S01]I mean, there are situations where the technology[635.47] [635.47][S01]itself is regulated, right?[636.73] [636.73][S02] But you're still regulating the use[638.23] [638.23][S02]because the reason you're regulating the technology,[640.27] [640.27][S02]or that you're going at the architecture[641.89] [641.89][S02]in this particular case, is because you're[643.64] [643.64][S02]worried about the ecosystems.[644.87] [644.87][S02]So you can imagine that you regulate that you say,[647.66] [647.66][S02]here's how this technology can be designed,[651.13] [651.13][S02]here's how this technology can be developed.[653.03] [653.03][S02]But it's usually always going to be in that sociotechnical frame[655.84] [655.84][S02]that we discussed.[656.87] [656.87][S01] But how about--[657.7] [657.7][S01]I mean, I'm trying to think of counterexamples, though,[659.992] [659.992][S01]because I think that there are some that, to me, it doesn't[662.8] [662.8][S01]seem completely black and white that it's always[665.02] [665.02][S01]the applications, not the technology.[666.65] [666.65][S01]I'm thinking here about, I don't know, like plutonium,[669.138] [669.138][S01]for instance.[669.68] [669.68][S01]I mean, there's very strict rules about plutonium transport,[673.76] [673.76][S01]about how it can be used, by whom.[677.198] [677.198][S02] How it can be used.[678.74] [678.74][S01] How it can be used.[680.032] [680.032][S01]True, very true.[680.699] [680.699][S02] And I don't think you're wrong.[682.74] [682.74][S02]I think you will find different kinds[684.94] [684.94][S02]of regulatory interventions across the entire ecosystem.[689.29] [689.29][S02]Let's rephrase it and say, OK, maybe it's[691.24] [691.24][S02]not about use or applications and technology.[694.13] [694.13][S02]Maybe what we're always looking for in regulation,[696.86] [696.86][S02]maybe what we always want to think about is harm.[699.25] [699.25][S02]So what is the potential harm?[701.0] [701.0][S02]And then we find a place in the social-technical context[705.02] [705.02][S02]where we believe that we can best prevent that harm.[708.33] [708.33][S02]That gives us another frame of thinking about this.[710.79] [710.79][S02]And I'm not wed to any frame.[712.483] [712.483][S02]I think that's a really good frame[713.9] [713.9][S02]to think into, because if you focus on harm instead,[716.398] [716.398][S02]and you can say, in the CRISPR case, for example,[718.44] [718.44][S02]we believe there can be harm to ecosystems.[720.42] [720.42][S02]The best way for us to prevent that[722.93] [722.93][S02]is to look at the CRISPR technologies design.[726.33] [726.33][S01] If we apply that back to AI,[728.4] [728.4][S01]how do you regulate the harms without regulating[732.74] [732.74][S01]the technology, if you see what I mean?[734.703] [734.703][S01]What does that actually look like in practice?[736.62] [736.62][S02] Well, one thing[737.51] [737.51][S02]you can do, let's take a very concrete example,[739.5] [739.5][S02]because it's always good to do this with examples.[741.583] [741.583][S02]So you say you want to regulate bias.[744.35] [744.35][S02]And you say, bias is a harm.[745.68] [745.68][S02]If a system is deeply biased, there's a harm here.[748.647] [748.647][S02]The first thing we then have to do, and this is quite helpful,[751.23] [751.23][S02]is we have to say, OK, what does that harm look like?[754.2] [754.2][S02]Well, we believe that somebody could have a decision made[757.34] [757.34][S02]against them that sort of negative for them,[759.93] [759.93][S02]because the system is biased in some way.[763.16] [763.16][S02]And so we want to prevent that harm of the individual[765.86] [765.86][S02]not being admitted to a school, not getting a welfare check,[769.64] [769.64][S02]or whatever we want to do.[770.91] [770.91][S02]So what we're looking at is that kind of harm.[773.48] [773.48][S02]Now, the next question we should ask in a regulatory context[776.57] [776.57][S02]is, where can we best prevent this?[779.01] [779.01][S02]Now we have choices.[780.3] [780.3][S02]Now we can say, I think the best way[782.0] [782.0][S02]to do this is to make sure that there is never[784.25] [784.25][S02]any bias in a data set.[786.36] [786.36][S02]And that's a technical choice, right?[788.19] [788.19][S02]We choose to go all the way back to the data set.[790.92] [790.92][S02]And we say, the data set can under no circumstances[793.67] [793.67][S02]contain bias.[795.06] [795.06][S02]Now that is very, very, very hard.[797.32] [797.32][S02]But it's a choice we can make.[798.57] [798.57][S02]We can say, that's what we want to do.[800.153] [800.153][S02]Another choice we can make is all decisions that[803.0] [803.0][S02]go through an algorithm of some kind,[805.65] [805.65][S02]or where a data set can be used, or where[807.47] [807.47][S02]there is a risk for bias, need to be reviewed by two people.[811.16] [811.16][S02]That's another regulatory solution[812.96] [812.96][S02]to that same kind of problem.[814.62] [814.62][S02]Now the question we are faced with,[816.22] [816.22][S02]and it's, OK, what's the most effective intervention we[818.89] [818.89][S02]can make here?[819.74] [819.74][S02]Let's take a very ineffective intervention.[822.02] [822.02][S02]We can say, yes, the data can be biased[824.24] [824.24][S02]and there's going to be no human review.[826.07] [826.07][S02]But the algorithm that uses the data[829.33] [829.33][S02]has to be built in such a way that it excludes[831.82] [831.82][S02]all possibilities of bias.[833.54] [833.54][S02]I don't even know how I would start doing that.[835.7] [835.7][S02]But you can say that kind of regulation[837.43] [837.43][S02]is what you could come up with because you have data,[839.84] [839.84][S02]you have algorithms, and you have humans in the system.[843.29] [843.29][S02]So as we try to prevent the harm,[845.355] [845.355][S02]we figure out where in the system[846.73] [846.73][S02]we most effectively can prevent the harm.[848.505] [848.505][S01] Does that not require[849.88] [849.88][S01]you knowing in advance what the potential harms will be?[852.59] [852.59][S02] Oh, that's such a good question[853.81] [853.81][S02]and such a good point, because that[855.52] [855.52][S02]is what's truly hard with new technology, right?[857.84] [857.84][S02]When we have emerging technology,[859.55] [859.55][S02]we don't exactly know what the harms are,[861.88] [861.88][S02]and that's why we end up trying to use certain blanket[864.46] [864.46][S02]principles for new technology.[866.54] [866.54][S02]One of them, for example, is the cost-benefit principle,[869.29] [869.29][S02]where we try to figure out what the possible costs are[871.54] [871.54][S02]with some approximation of the probability of unknown harm,[875.28] [875.28][S02]which is super hard.[876.33] [876.33][S02]And now there is a precautionary principle[878.09] [878.09][S02]where we say that you have to show that there's[880.048] [880.048][S02]no downside to this particular technology[882.26] [882.26][S02]under these particular conditions before you deploy it.[885.18] [885.18][S01] Which is what the UK Secretary of State for Science,[888.15] [888.15][S01]Innovation and Technology said.[889.52] [889.52][S01]They said that social media platforms, for instance,[891.69] [891.69][S01]must prove their products are safe before release, which is[894.832] [894.832][S01]exactly as you're describing.[896.04] [896.04][S02] Yes, which creates[896.93] [896.93][S02]another kind of evidentiary problem[898.52] [898.52][S02]because it's really hard to prove a negative.[902.09] [902.09][S02]So what you end up with then is that you[905.27] [905.27][S02]have to prove to some standard.[906.6] [906.6][S02]You have to prove to some probability[908.27] [908.27][S02]that this technology is not going to be harmful,[910.35] [910.35][S02]which may actually be quite a good standard in many cases,[913.44] [913.44][S02]but it can also be a highly restrictive standard,[916.41] [916.41][S02]depending on how high you put the bar for saying that there[919.82] [919.82][S02]will be no harm here.[921.568] [921.568][S01] Give me an example.[922.86] [922.86][S01]In what situation would it make it difficult?[924.9] [924.9][S02] Well, let's take a very simple example,[927.275] [927.275][S02]scanning X-rays for cancer.[929.45] [929.45][S02]And you say, OK, we're only going to actually release[933.15] [933.15][S02]this system if we can prove that it will never have or miss[937.26] [937.26][S02]any kind of cancer.[939.76] [939.76][S02]That's a super simple precautionary principle[942.42] [942.42][S02]example for you.[943.48] [943.48][S02]And that standard is too high.[946.08] [946.08][S02]And the standard we should set is it should probably[949.11] [949.11][S02]outperform human doctors.[951.36] [951.36][S02]That's a better standard.[952.42] [952.42][S01] We should be absolutely clear about this.[953.47] [953.47][S01]Why is that standard too high?[955.39] [955.39][S02] Oh, I think it's too high because then you[956.88] [956.88][S02]deploy no system.[957.9] [957.9][S02]Because there's no system that can do that.[960.31] [960.31][S02]You can never prove that a system[962.04] [962.04][S02]will be 100% accurate in any real life situation.[965.56] [965.56][S02]So that's too high a bar.[966.685] [966.685][S02]And then you can go down and you can say, OK,[968.56] [968.56][S02]you should have these different kinds of--[971.52] [971.52][S02]it should prove these different kinds of accuracy.[973.865] [973.865][S02]One you can say is that it should[975.24] [975.24][S02]be more accurate on average than human doctors are on average.[978.617] [978.617][S02]And the other is that you can say, OK,[980.2] [980.2][S02]if human doctors are also reviewing,[981.858] [981.858][S02]you can actually go a little bit lower and you can say,[984.15] [984.15][S02]it just needs to have a signal value that[985.858] [985.858][S02]helps the human doctor to interpret the X-ray.[988.287] [988.287][S02]And so you can say, it doesn't have to be accurate[990.37] [990.37][S02]100% of the time.[991.28] [991.28][S02]It can be accurate 40% of the time,[992.93] [992.93][S02]even if the doctors are accurate 80% of the time,[996.11] [996.11][S02]because the extra signal value might boost your 80 to 82,[999.29] [999.29][S02]in the hybrid system.[1000.43] [1000.43][S02]So you end up with these sort of interesting questions[1003.57] [1003.57][S02]about where do you put the bar.[1005.097] [1005.097][S01] That's a bar on benefit though, right?[1007.18] [1007.18][S01]Because there's also the bar on the other side, on cost.[1011.86] [1011.86][S01]And deciding that in advance is--[1015.09] [1015.09][S01]well, how do you decide where the bar is for potential costs[1018.72] [1018.72][S01]that you're willing to accept?[1020.35] [1020.35][S02] I mean, you can take the cancer example[1022.08] [1022.08][S02]again, the X-ray example.[1023.157] [1023.157][S02]You can say, it should absolutely never have[1024.99] [1024.99][S02]a false positive, because that creates harm in the sense[1027.96] [1027.96][S02]that it uses the healthcare system unnecessarily.[1030.7] [1030.7][S02]And for the individual who gets the diagnose,[1032.829] [1032.829][S02]it's like a personal catastrophe, and so on.[1034.98] [1034.98][S02]That's a challenge.[1036.48] [1036.48][S02]Because we know there's no system[1037.89] [1037.89][S02]that can fulfill that standard.[1039.31] [1039.31][S02]And then we scale back from there[1040.71] [1040.71][S02]to find the standard that we think is reasonable.[1042.994] [1042.994][S01] If we expand this out[1044.369] [1044.369][S01]to a slightly broader use of technology,[1047.38] [1047.38][S01]I mean, the example of sensitivity and specificity,[1050.02] [1050.02][S01]and false positive and false negatives[1051.76] [1051.76][S01]is a very clear example of where the potential harms are.[1054.68] [1054.68][S01]But I'm thinking about more broadly about where[1057.1] [1057.1][S01]you would get unexpected harms.[1058.76] [1058.76][S01]For instance, I'm thinking about the 2008 financial crisis here[1062.17] [1062.17][S01]and how a lot of that problem was[1064.66] [1064.66][S01]caused by the same models being used over and over again.[1069.592] [1069.592][S01]And no one quite noticing that that[1071.05] [1071.05][S01]meant that there was this common vulnerability across this kind[1073.99] [1073.99][S01]of common weakness.[1076.21] [1076.21][S01]And then only after the big catastrophe[1079.433] [1079.433][S01]happened that it's like, you can go back[1081.1] [1081.1][S01]and pick through the rubble and decide what the regulation[1083.32] [1083.32][S01]should have been.[1084.05] [1084.05][S01]How can you prevent those kind of big global events[1088.69] [1088.69][S01]from happening in technology, where you don't necessarily[1091.96] [1091.96][S01]understand all of the harms in advance?[1093.925] [1093.925][S02] It's very hard,[1095.3] [1095.3][S02]is the answer to the question.[1096.55] [1096.55][S02]What you can do, though, is that you[1098.29] [1098.29][S02]can try as far as you can go to do scenario planning,[1101.69] [1101.69][S02]to do red teaming, to go through what the technology can[1104.38] [1104.38][S02]do and figure out how it fails.[1106.25] [1106.25][S02]One of the most important questions about any technology[1108.82] [1108.82][S02]is actually how it fails.[1110.23] [1110.23][S02]And one of the things that we can do,[1112.46] [1112.46][S02]and that we consistently try to do in technology,[1115.04] [1115.04][S02]is to design graceful failure.[1116.96] [1116.96][S02]If you think about an airplane, for example,[1119.45] [1119.45][S02]it fails gracefully to the extent[1121.15] [1121.15][S02]that it can even glide if all of its systems go down.[1124.58] [1124.58][S02]And one of the things that I think[1126.003] [1126.003][S02]people are thinking actively about[1127.42] [1127.42][S02]is how can you replicate graceful failure[1129.67] [1129.67][S02]in other kinds of systems.[1131.06] [1131.06][S02]And it doesn't just have to be AI.[1132.74] [1132.74][S02]It can be any kind of system really, that we rely on.[1136.03] [1136.03][S02]So we're faced with as soon as we move outside,[1139.43] [1139.43][S02]and you're quite right, when we move outside[1141.67] [1141.67][S02]of the small clinical example I gave you,[1144.23] [1144.23][S02]the uncertainty increases.[1146.0] [1146.0][S02]And then at some point, we have to decide as a society,[1149.15] [1149.15][S02]how much uncertainty are we willing to tolerate[1152.92] [1152.92][S02]for what progress?[1154.383] [1154.383][S01] Underlying what you're[1155.8] [1155.8][S01]saying there is almost this implication[1157.78] [1157.78][S01]that we have to accept that there[1161.05] [1161.05][S01]is some probability of catastrophic failure,[1163.27] [1163.27][S01]that we can't mitigate against everything.[1165.5] [1165.5][S01]I mean, even the looking for graceful failures, as you say.[1169.51] [1169.51][S01]I mean, there's no way that we can sit here[1171.99] [1171.99][S01]and say, we are sure that we are going[1175.23] [1175.23][S01]to build technology that does not[1177.12] [1177.12][S01]risk existential crisis even.[1179.725] [1179.725][S02] No.[1180.6] [1180.6][S02]We can never be sure.[1181.87] [1181.87][S02]And that's part of the human condition, I think.[1184.5] [1184.5][S02]Uncertainty is a part of the human condition,[1186.79] [1186.79][S02]at least if we want progress.[1188.62] [1188.62][S02]And I think progress actually exacerbate that uncertainty.[1191.47] [1191.47][S02]I think Friedrich von Hayek, the Austrian economist,[1195.67] [1195.67][S02]at some point wrote, "man has never been the ruler of her own[1199.29] [1199.29][S02]fate, and that is the reason we have progress."[1202.36] [1202.36][S02]And so there's a difficulty here, of course.[1206.98] [1206.98][S02]And I think I'm not saying that so we should with[1210.51] [1210.51][S02]abandon throw ourselves out into the darkness.[1213.49] [1213.49][S02]But what I am saying is that I think the notion of progress,[1216.73] [1216.73][S02]the notion of welfare, the notion of my kids[1218.64] [1218.64][S02]having it better than I do, does come with a bit of uncertainty.[1222.9] [1222.9][S02]And if I want that, then I also need[1224.94] [1224.94][S02]to tolerate managed uncertainty.[1227.32] [1227.32][S01] But I sort of wonder, I[1228.87] [1228.87][S01]wonder whether there's a question here about[1231.27] [1231.27][S01]must we do it at all.[1232.81] [1232.81][S01]If doing so means that there's unavoidable uncertainty, then[1238.2] [1238.2][S01]what about just not doing it?[1239.572] [1239.572][S02] Yeah, absolutely.[1241.03] [1241.03][S02]We can do that.[1241.78] [1241.78][S02]What we do then, as a society, or as a polity, is to say,[1244.86] [1244.86][S02]we choose not to progress.[1246.84] [1246.84][S02]From this point on, we will conserve what we have,[1249.18] [1249.18][S02]we will do nothing new, and take on no new risk.[1252.64] [1252.64][S02]Although risk is what drives societies forward.[1256.06] [1256.06][S02]And I think that's a political choice.[1258.37] [1258.37][S02]I mean, a lot of--[1259.77] [1259.77][S02]it's interesting, right?[1260.89] [1260.89][S02]Because a lot of what we do in policy,[1262.93] [1262.93][S02]and when we discuss regulation, is[1264.81] [1264.81][S02]that we're constantly exploring what John Rawls calls[1268.89] [1268.89][S02]reasonable disagreements.[1270.22] [1270.22][S02]So what are the two positions that[1272.52] [1272.52][S02]are warring with each other on this particular point?[1275.29] [1275.29][S02]And on this point, there's one position[1277.02] [1277.02][S02]that says, no, we should go for de-growth.[1278.86] [1278.86][S02]We should rescale our societies.[1281.14] [1281.14][S02]There's a lot to be had in an agricultural economy.[1284.93] [1284.93][S02]That's the extreme version of that.[1286.74] [1286.74][S02]And there's one that says, no, we need to go to the stars,[1289.157] [1289.157][S02]and we need to figure out how the universe works,[1291.198] [1291.198][S02]and we need to really make sure that we[1292.85] [1292.85][S02]continue to progress as human beings and build the future.[1295.56] [1295.56][S02]Those two are both defensible from a foundational ethical[1299.84] [1299.84][S02]perspective.[1300.69] [1300.69][S02]And at some point, we just have to choose.[1302.84] [1302.84][S02]And not choose binarily, but choose[1305.18] [1305.18][S02]where on the spectrum between the two we are.[1307.41] [1307.41][S01] But then we choose.[1308.88] [1308.88][S01]I mean, that's quite a big, like,[1310.85] [1310.85][S01]all encapsulating thing because there's no sort of opt-out here.[1315.35] [1315.35][S01]I mean, this is we, as all of humanity[1317.81] [1317.81][S01]collectively, we're either developing these technologies[1320.39] [1320.39][S01]or we're not.[1321.208] [1321.208][S02] Yeah, we are.[1322.5] [1322.5][S02]And I mean, I think that's an unavoidable truth[1325.76] [1325.76][S02]to some degree.[1326.52] [1326.52][S02]And that's why it's so, I think, important[1328.7] [1328.7][S02]to also invest in democracy.[1330.54] [1330.54][S02]And this is sort of a side point,[1332.04] [1332.04][S02]but it's the way we make regulation,[1334.44] [1334.44][S02]the methods whereby which we regulate technology actually[1337.4] [1337.4][S02]matter.[1338.04] [1338.04][S02]I do think that democracies have a better[1340.73] [1340.73][S02]way of discovering people's real views on this[1343.25] [1343.25][S02]and respecting and representing us[1345.44] [1345.44][S02]collectively than, for example, authoritarian systems have.[1349.15] [1349.15][S01] I do also want to talk about the role of private[1351.65] [1351.65][S01]companies in all of this, because I mean, of course,[1354.33] [1354.33][S01]we're having this conversation within the four walls of "Google[1357.98] [1357.98][S01]DeepMind."[1358.53] [1358.53][S01]What do you think the role should[1360.32] [1360.32][S01]be in having private companies to shape this regulation?[1364.35] [1364.35][S02] I think ideally, if we do it right,[1366.63] [1366.63][S02]the way that all of these policy discussions work,[1369.77] [1369.77][S02]or what's sometimes called lobbying works,[1372.81] [1372.81][S02]if lobbying really works for society's best, it's a knowledge[1376.4] [1376.4][S02]exchange equation.[1378.0] [1378.0][S02]So we give knowledge around how this technology works.[1381.18] [1381.18][S02]And in exchange for that, we get influence over how[1384.17] [1384.17][S02]the regulation is shaped.[1385.7] [1385.7][S02]And that is the role I think private companies should[1388.13] [1388.13][S02]take to a large degree in all emerging technologies.[1391.8] [1391.8][S02]And that's not just true for AI.[1393.15] [1393.15][S02]It's true for life sciences, for example.[1396.15] [1396.15][S02]There is this knowledge asymmetry[1397.76] [1397.76][S02]where a lot of the knowledge sits in the private sector,[1400.52] [1400.52][S02]and we need to even it out.[1401.96] [1401.96][S02]And I don't mean to say, sometimes hear people say,[1404.59] [1404.59][S02]we need to educate policymakers.[1406.22] [1406.22][S02]I think that's wrong.[1407.21] [1407.21][S02]I think that we have to have some kind of mutual dialogue[1410.2] [1410.2][S02]here where we educate about the technology,[1412.55] [1412.55][S02]but policymakers educate us about democracy, institutions,[1415.67] [1415.67][S02]and values in ways that actually can help us think about, OK,[1418.97] [1418.97][S02]how do we shape this technology?[1421.01] [1421.01][S02]A good example is, we understood early on[1423.49] [1423.49][S02]that there was going to be requirements to figure out[1426.88] [1426.88][S02]how to deal with content that was synthetically produced.[1430.06] [1430.06][S02]And so those values, those political sort[1433.21] [1433.21][S02]of interests in some way also informed the work[1437.2] [1437.2][S02]that we have done on a very technical thing called SynthID.[1440.2] [1440.2][S02]So figuring out how watermarking works,[1442.52] [1442.52][S02]how we can make it possible to distinguish[1445.54] [1445.54][S02]real content from content that's been synthetically generated,[1449.36] [1449.36][S02]is something that you learn if you have this ongoing dialogue[1453.13] [1453.13][S02]between the political and the technical,[1454.88] [1454.88][S02]between the public and the private.[1456.44] [1456.44][S01] But I mean, is that a universal belief[1459.76] [1459.76][S01]among the tech industry?[1461.39] [1461.39][S01]Some people, I mean, there is sort[1463.0] [1463.0][S01]of a push for local expertise, internal expertise being[1468.31] [1468.31][S01]the ultimate thing to be considered.[1470.493] [1470.493][S02] I think the more[1471.91] [1471.91][S02]you can build, not just expertise,[1474.185] [1474.185][S02]but also understanding of this technology in the public sector,[1476.81] [1476.81][S02]the better it is.[1477.518] [1477.518][S02]So I'm not opposed to that, but I mean,[1480.007] [1480.007][S02]you put your finger on something that's important.[1482.09] [1482.09][S02]Of course, not everyone in technology sector[1484.0] [1484.0][S02]thinks the same way.[1485.05] [1485.05][S02]We have lots of different perspectives.[1487.01] [1487.01][S02]And it also depends a little bit on where your ideological sort[1490.51] [1490.51][S02]of home is.[1491.95] [1491.95][S02]An interesting example of a difference[1494.32] [1494.32][S02]that we've seen in the last 10 years or so[1496.36] [1496.36][S02]is the difference between internet policy and AI policy.[1500.03] [1500.03][S02]Because if you think back to the early days of the internet,[1502.69] [1502.69][S02]to 1996, the internet ethos amongst[1507.12] [1507.12][S02]people who discussed policy, and politics,[1508.87] [1508.87][S02]and the internet was highly libertarian,[1511.82] [1511.82][S02]extremely libertarian.[1513.17] [1513.17][S02]In 1996, John Perry Barlow publishes the "Declaration[1517.35] [1517.35][S02]of Independence of Cyberspace," which is a fantastically[1520.74] [1520.74][S02]bombastic document.[1522.01] [1522.01][S02]It sort of says, you weary giants of steel and flesh,[1525.13] [1525.13][S02]leave us alone.[1526.51] [1526.51][S02]That's the basic tonality in the early internet policy debates.[1531.79] [1531.79][S02]In the AI policy debates, it's really different[1534.12] [1534.12][S02]because if you speak to young engineers[1535.8] [1535.8][S02]today, if you speak to anyone working on this, if you speak[1538.35] [1538.35][S02]to researchers, they say, no, this technology[1540.6] [1540.6][S02]is powerful enough that it needs to be regulated in some way.[1544.69] [1544.69][S02]And we have AI companies persistently saying[1548.04] [1548.04][S02]that this technology needs to be regulated.[1549.97] [1549.97][S02]Now, that begs another question.[1551.62] [1551.62][S02]When you say that, you should be able to answer how.[1554.28] [1554.28][S02]And I think that's a challenge, and we'll probably get to that.[1557.35] [1557.35][S02]But I do think that one of the things that you see[1559.77] [1559.77][S02]is that the ethos is different, and that's really important.[1562.57] [1562.57][S01] But there are some who[1563.987] [1563.987][S01]think that self-regulation should be the main lever that[1567.63] [1567.63][S01]gets used as we go forwards.[1568.797] [1568.797][S02] Self-regulation[1570.172] [1570.172][S02]can be good for two reasons.[1571.36] [1571.36][S02]One is because you want to keep sort of your ability[1574.41] [1574.41][S02]to do whatever you want.[1575.53] [1575.53][S02]So self-regulation becomes almost a defensive move.[1578.495] [1578.495][S02]You say, yes, yes, we'll self-regulate.[1580.12] [1580.12][S02]Another reason for self-regulating[1582.6] [1582.6][S02]can be that there is so much we don't[1584.73] [1584.73][S02]know that we need to make sure that we constantly[1587.91] [1587.91][S02]change the regulation.[1589.48] [1589.48][S02]Self-regulation is easier to change than legislation is.[1592.47] [1592.47][S02]But I don't think that the two are mutually exclusive.[1595.81] [1595.81][S02]Self-regulation with transparency[1597.69] [1597.69][S02]allows for legal review and auditing, for example.[1600.58] [1600.58][S02]So you can imagine different ways in which we as a company[1603.91] [1603.91][S02]could test our models and then review openly[1606.33] [1606.33][S02]how the models have been tested, like the AI Act says.[1609.61] [1609.61][S02]And then there is a possibility for legal and auditing[1613.68] [1613.68][S02]and review of that from outside of the company.[1616.18] [1616.18][S02]So you combine them in different ways.[1617.92] [1617.92][S01] I guess one of the other big counterarguments[1620.61] [1620.61][S01]is that this whole picture is really further colored[1624.96] [1624.96][S01]by the pursuit of profit, in the sense that companies who[1628.8] [1628.8][S01]have a hand in deciding regulations,[1631.26] [1631.26][S01]can you trust that they're doing so for public good,[1636.26] [1636.26][S01]or for competitive advantage?[1638.295] [1638.295][S02] And the profit motive[1639.92] [1639.92][S02]is always going to be something that we[1641.75] [1641.75][S02]should take into account.[1642.9] [1642.9][S02]But it shouldn't disqualify companies[1645.14] [1645.14][S02]from actually having some moral authority.[1647.97] [1647.97][S02]And I think the problem is that often the profit motive is used[1650.78] [1650.78][S02]as this blanket argument saying, you only[1652.52] [1652.52][S02]think about your profit.[1653.78] [1653.78][S02]And you can't do that, because if you only[1655.85] [1655.85][S02]think about your profit and mercilessly drive profit,[1658.62] [1658.62][S02]you're not going to be able to recruit the right talent.[1661.02] [1661.02][S02]People are not going to want to work on a company like that.[1663.52] [1663.52][S02]You're not going to be able to do deals[1665.532] [1665.532][S02]with others in the industry who want to be seen[1667.49] [1667.49][S02]as responsible and respectable.[1669.42] [1669.42][S02]There's a huge difference between a company that's[1671.75] [1671.75][S02]trying to maximize its profit quarter to quarter,[1674.67] [1674.67][S02]and a company that's trying to maximize its profits over 100[1677.45] [1677.45][S02]years.[1678.03] [1678.03][S02]And in order to do that, you actually[1679.79] [1679.79][S02]have to be a good player.[1680.91] [1680.91][S02]It's like any prisoner's dilemma that you reiterate.[1684.26] [1684.26][S02]The tit for tat rule.[1685.56] [1685.56][S02]What turns out to be most sustainable in a repeated game[1689.24] [1689.24][S02]is to be a trusted and somewhat good player.[1692.407] [1692.407][S02]I'm more worried about something else,[1693.99] [1693.99][S02]actually, if I can put on my worry hat.[1696.26] [1696.26][S02]And that is that I think currently,[1698.6] [1698.6][S02]investments in science from the private sector,[1701.42] [1701.42][S02]and this is true for a lot of different sectors,[1704.25] [1704.25][S02]are much, much higher than investments[1706.28] [1706.28][S02]from the public sector.[1707.66] [1707.66][S02]And I think that over the last couple of decades,[1710.79] [1710.79][S02]the prioritization of science as a public venture[1714.35] [1714.35][S02]has sadly fallen down a lot.[1717.39] [1717.39][S02]I actually would like to see more public investment[1719.96] [1719.96][S02]in science, more public work.[1721.9] [1721.9][S02]We've been, for example, deeply supportive[1723.65] [1723.65][S02]of the notion of a national AI resource in different shapes[1728.63] [1728.63][S02]or forms around the world, where we believe[1730.73] [1730.73][S02]that public investment in this technology[1732.74] [1732.74][S02]will actually give more public knowledge,[1735.45] [1735.45][S02]will give more insight into how the technology works,[1738.27] [1738.27][S02]and that will be a good thing.[1739.52] [1739.52][S01] And do you think then[1740.895] [1740.895][S01]with that as the background, there[1742.55] [1742.55][S01]is a risk that we'll end up in a future[1744.59] [1744.59][S01]where AI is in the hands of a very small number of companies?[1749.758] [1749.758][S02] There's always that risk.[1751.55] [1751.55][S02]And I think one of the things that we[1753.092] [1753.092][S02]see is that for many of the other markets or sectors that we[1756.19] [1756.19][S02]look at, we have this structure where[1758.26] [1758.26][S02]you have a few companies that are in some way or shape very[1762.91] [1762.91][S02]important in that sector.[1764.033] [1764.033][S02]You see it in pharma, you see it in telco, you see it in energy,[1766.7] [1766.7][S02]you see it in all of these different sectors.[1768.575] [1768.575][S02]And the question then is, if that is the case,[1772.16] [1772.16][S02]is that then harmful or is that OK?[1774.98] [1774.98][S02]Is it OK that that's sort of an effect[1776.8] [1776.8][S02]of industrial organization and economic pressures?[1779.8] [1779.8][S02]Is it something that holds back innovation?[1782.3] [1782.3][S02]Is it something that sort of reduces consumer welfare?[1785.39] [1785.39][S02]That's the question you have to ask.[1787.34] [1787.34][S02]I think I'm less worried about the market structure, where[1790.24] [1790.24][S02]you could say you have four or five really large pharma[1792.91] [1792.91][S02]companies or energy companies than I am about[1795.64] [1795.64][S02]how that impacts the overall change[1797.5] [1797.5][S02]and progress in those sectors.[1799.7] [1799.7][S01] OK, I want to get on to some really concrete examples[1802.54] [1802.54][S01]of regulation.[1803.27] [1803.27][S01]Those four different approaches exactly as you described.[1805.91] [1805.91][S01]Of course, the EU's AI Act came into force earlier this year.[1809.57] [1809.57][S01]And they've really gone for this risk-based approach.[1812.87] [1812.87][S01]So at the one end, the kind of unacceptable risks[1815.89] [1815.89][S01]are things like social credit scores.[1818.57] [1818.57][S01]And then down at the other end, the very low risk[1821.118] [1821.118][S01]stuff for things like chatbots.[1822.41] [1822.41][S01]Do you think this is broadly a good approach?[1825.045] [1825.045][S02] So I think it's[1826.42] [1826.42][S02]a good one to start with that.[1827.57] [1827.57][S02]I think it's actually when the commission originally[1829.737] [1829.737][S02]published its proposal, I think it was a really thoughtful way[1833.65] [1833.65][S02]to approach a problem that is very[1836.02] [1836.02][S02]much like the one we discussed.[1837.35] [1837.35][S02]What do you do with the technology[1838.767] [1838.767][S02]when it's hard to predict the harms?[1841.947] [1841.947][S02]And so in some way, I think it's a really good way[1844.03] [1844.03][S02]to think about things.[1844.947] [1844.947][S02]Where is the risk?[1846.77] [1846.77][S02]And then tune the regulation to the risk.[1850.63] [1850.63][S02]I think the downsides, of course,[1852.86] [1852.86][S02]as to how you assess risk and who assesses it.[1857.21] [1857.21][S02]It's been pointed out by people that if companies[1859.51] [1859.51][S02]get to assess their own risk entirely,[1861.62] [1861.62][S02]then that's going not to work, to your point[1863.8] [1863.8][S02]about self-regulation.[1864.792] [1864.792][S02]Now, that's not what the regulation says.[1866.5] [1866.5][S02]The regulation has other views about this.[1868.71] [1868.71][S02]Another argument against the risk-based regulation,[1872.62] [1872.62][S02]of course, is that you are not looking at all[1875.73] [1875.73][S02]at the upside or the benefits.[1878.02] [1878.02][S02]So you say, this is a high risk, but what if it's high reward?[1881.85] [1881.85][S02]Say we have a high risk application that[1884.58] [1884.58][S02]can be used to, we go back to the medical case, cure a disease[1888.72] [1888.72][S02]or sort of massively improve a situation.[1891.04] [1891.04][S02]Shouldn't we then at some point factor in the fact[1894.51] [1894.51][S02]that it's so high reward and say, OK, we're[1898.02] [1898.02][S02]going to regulate this, but we're also[1899.88] [1899.88][S02]going to give you a little bit of leeway[1901.83] [1901.83][S02]because we think that risk alone isn't the best metric here?[1905.16] [1905.16][S02]That criticism could be leveled against the system.[1908.85] [1908.85][S02]But I think that it's probably even harder[1912.21] [1912.21][S02]to assess reward than risk.[1914.1] [1914.1][S02]So it becomes even messier to do that.[1916.71] [1916.71][S02]And that is what you're seeing at work here,[1919.9] [1919.9][S02]if you scratch the surface, is a tension[1923.04] [1923.04][S02]between the American and the European approach,[1925.66] [1925.66][S02]because the American approach is the cost-benefit principle,[1928.947] [1928.947][S02]whereas you have the precautionary principle[1930.78] [1930.78][S02]in Europe.[1931.28] [1931.28][S02]And the precautionary principle says, let's look at the risk.[1934.02] [1934.02][S02]And the cost-benefit principle says,[1935.83] [1935.83][S02]let's look at the reward and the risk and see how they compare.[1939.3] [1939.3][S01] I do wonder a little bit[1940.8] [1940.8][S01]about that difference between the US and the EU approach,[1943.972] [1943.972][S01]because I guess now there's a reasonable chance[1945.93] [1945.93][S01]that those executive orders will be overturned.[1948.19] [1948.19][S01]Is there a chance that we could end up in a situation where[1951.18] [1951.18][S01]there is this big gap between the types of regulation that you[1954.177] [1954.177][S01]get in the US and the types of regulation that you have within[1956.76] [1956.76][S01]the EU, which then exacerbates the "winner takes all"[1960.3] [1960.3][S01]thing where really get this huge advantage in innovation[1963.54] [1963.54][S01]for American companies?[1964.907] [1964.907][S02] I think there is a definite risk[1966.99] [1966.99][S02]that regulation becomes determinative[1969.33] [1969.33][S02]in what kind of economic growth you get, what kind of welfare[1973.49] [1973.49][S02]you get.[1973.99] [1973.99][S02]And I do think that that's one of the things[1976.2] [1976.2][S02]that the European model will test.[1980.11] [1980.11][S02]And we will see if that kind of regulatory model[1982.76] [1982.76][S02]can carry the kind of innovation that Europe wants.[1986.22] [1986.22][S02]The hypothesis that the European regulator has[1989.75] [1989.75][S02]is that a clear playing field, sort of clear rules,[1994.25] [1994.25][S02]and a way to access the entire European market that's[1997.76] [1997.76][S02]been designed for the European AI Act[1999.77] [1999.77][S02]is going to boost innovation, and it's[2002.56] [2002.56][S02]going to boost investments, it's going[2004.21] [2004.21][S02]to boost the use of this technology[2005.74] [2005.74][S02]because people now know what to do.[2007.52] [2007.52][S02]And their hypothesis then, is that the US model will[2011.74] [2011.74][S02]create so much uncertainty.[2013.43] [2013.43][S02]And with US litigiousness and liability,[2016.85] [2016.85][S02]it's actually going to make it much slower[2019.27] [2019.27][S02]than if you had a comprehensive piece of legislation.[2021.83] [2021.83][S02]Those hypotheses will be tested out in real time.[2024.015] [2024.015][S01] Because I mean, there[2025.39] [2025.39][S01]are some situations where actually regulation has[2027.67] [2027.67][S01]accelerated innovation.[2029.03] [2029.03][S01]I mean, I'm thinking here about emissions on vehicles ending up[2033.52] [2033.52][S01]causing innovation in the electric vehicle market.[2035.98] [2035.98][S01]I mean, as a Formula One fan, the years that[2038.933] [2038.933][S01]are most exciting in terms of innovation[2040.6] [2040.6][S01]are the ones where new regulations come in.[2042.71] [2042.71][S01]And as you say, people have this framework[2044.89] [2044.89][S01]within which they know that they have to innovate.[2047.03] [2047.03][S02] Yeah, I think that can be absolutely true.[2049.53] [2049.53][S02]That depends on that regulation being very specific,[2052.04] [2052.04][S02]setting out a space that is viable.[2053.929] [2053.929][S02]And there are examples of this in technology regulation too.[2056.69] [2056.69][S02]I think that the Digital Millennium Copyright[2058.75] [2058.75][S02]Act, for example, which was put in place very early on,[2061.73] [2061.73][S02]had certain rules for what you had to do in order[2065.469] [2065.469][S02]to escape liability.[2067.07] [2067.07][S02]And one of the things was that you[2068.62] [2068.62][S02]had to have some way for people to claim their content[2071.08] [2071.08][S02]on the platform, which led to content ID, which[2074.193] [2074.193][S02]is one of the greatest innovations[2075.61] [2075.61][S02]that YouTube brought to the table.[2077.48] [2077.48][S02]And so you can see how that piece of innovation[2080.38] [2080.38][S02]was encouraged by the way that legislation was set up.[2083.21] [2083.21][S02]That's hard to do, but it's absolutely possible to do.[2086.042] [2086.042][S01] Do you think then, that we will see[2088.0] [2088.0][S01]similar types of innovation?[2089.54] [2089.54][S01]I mean, I'm thinking here about some of the generative AI stuff,[2093.76] [2093.76][S01]SynthID you mentioned earlier, which[2095.739] [2095.739][S01]has come about as a way to identify AI-generated content.[2101.52] [2101.52][S01]Do you think that the EU regulations[2104.79] [2104.79][S01]will force sort of wider adoption of those kind of ideas[2107.88] [2107.88][S01]and maybe even a wider agreement on standardized systems[2112.05] [2112.05][S01]for them?[2113.533] [2113.533][S02] I hope so.[2114.7] [2114.7][S02]I'm not entirely sure that we will see that,[2116.795] [2116.795][S02]because one of the things that you require in order[2118.92] [2118.92][S02]to get that innovative effect is that you[2121.8] [2121.8][S02]require a certain precision.[2123.37] [2123.37][S02]And currently, the European AI Act[2125.31] [2125.31][S02]is still being negotiated in a code of practice.[2128.26] [2128.26][S02]It's going to be implemented in all of the member states.[2130.69] [2130.69][S02]So we don't know yet if it will be clear enough[2133.5] [2133.5][S02]to drive that kind of innovation.[2135.1] [2135.1][S02]I think the innovation on SynthID and watermarking[2137.82] [2137.82][S02]comes from a global political dialogue, where you in the US,[2141.55] [2141.55][S02]in some of the United States, and in Europe, of course,[2144.04] [2144.04][S02]have sought for a means to figure out[2146.28] [2146.28][S02]how you can bring back some way of assessing the authority[2151.26] [2151.26][S02]or authoritativeness of content.[2153.4] [2153.4][S02]And I think that's a good example of how[2156.54] [2156.54][S02]regulation, or regulatory concerns, or policy concerns[2159.87] [2159.87][S02]can drive innovation.[2161.23] [2161.23][S01] I mean, I guess there's also the necessary,[2164.032] [2164.032][S01]but not sufficient argument here as well, which[2165.99] [2165.99][S01]is that it's all very well, sort of labeling, just as an example,[2169.12] [2169.12][S01]AI-generated content.[2170.56] [2170.56][S01]But if you're not doing it in real time, then[2173.04] [2173.04][S01]actually the harms can be caused--[2174.76] [2174.76][S01]you can't just retrospectively label things.[2177.2] [2177.2][S02] And the entire approach of labeling things[2179.7] [2179.7][S02]is interesting too, because we focus on the misinformation,[2182.19] [2182.19][S02]on the fake information.[2183.52] [2183.52][S02]And one of the things you can say[2184.983] [2184.983][S02]is that that's going to be really,[2186.4] [2186.4][S02]really, really hard because of the delay you mentioned[2188.7] [2188.7][S02]or because of the amount of information.[2190.51] [2190.51][S02]What we may want to do instead is to figure out,[2192.52] [2192.52][S02]is there another way for us to assign authority to content?[2195.16] [2195.16][S02]Can we actually say, not that this is misinformation[2198.0] [2198.0][S02]or this is synthetic, but this is really authoritative[2201.66] [2201.66][S02]because we have this flat information surface[2204.09] [2204.09][S02]that we're struggling with, and finding out what's actually what[2206.965] [2206.965][S02]or what's true in this flat information[2208.59] [2208.59][S02]surface is really hard.[2209.662] [2209.662][S02]Now, if you could build some peaks and valleys,[2211.62] [2211.62][S02]and if you could make this into an authoritative landscape,[2216.0] [2216.0][S02]then you could actually start to build in a way to sift out,[2220.43] [2220.43][S02]not just the fake, but also the highly valuable.[2223.41] [2223.41][S02]And if you look at the early days of newspapers,[2226.48] [2226.48][S02]that's actually what happened.[2227.73] [2227.73][S02]You started with a plethora of different newspapers[2231.02] [2231.02][S02]that were more or less libelous every single one of them.[2233.7] [2233.7][S02]Infamous scribblers, I think George Washington called them[2236.75] [2236.75][S02]in the US.[2237.51] [2237.51][S02]They would just make stuff up.[2240.36] [2240.36][S02]And then after a while, you saw the evolution[2243.65] [2243.65][S02]of publishing standards.[2245.22] [2245.22][S02]You saw some of the companies say,[2247.13] [2247.13][S02]this is printed in our newspaper,[2248.82] [2248.82][S02]so it was really relevant.[2250.23] [2250.23][S02]And you saw people building authority around content.[2253.59] [2253.59][S02]They weren't trying to attack the bad stuff.[2256.2] [2256.2][S02]They were trying to show you ways to the good stuff.[2259.14] [2259.14][S01] Yeah, I really like that idea.[2261.357] [2261.357][S01]Although, I also think that this is something that takes time.[2263.94] [2263.94][S01]And newspapers, as a really lovely example,[2268.08] [2268.08][S01]is something that developed over the course of decades rather[2271.19] [2271.19][S01]than the situation that we're in now where things are changing[2274.85] [2274.85][S01]in a lightning fast way.[2276.24] [2276.24][S01]Trust takes time to build up.[2279.05] [2279.05][S01]The EU actually also has this unacceptable risk category,[2283.34] [2283.34][S01]which I guess includes things like social credit scores[2286.37] [2286.37][S01]and lots of real time biometric identification.[2290.548] [2290.548][S02] And those are interesting[2292.34] [2292.34][S02]and we should discuss them.[2293.465] [2293.465][S02]I actually think it's quite useful for a society to say,[2296.31] [2296.31][S02]here are some uses that we're absolutely not going to condone.[2299.67] [2299.67][S02]Social scoring is one, for example.[2302.34] [2302.34][S02]I think that's good.[2303.39] [2303.39][S02]I think that is actually both helpful and healthy.[2307.53] [2307.53][S02]And I agree with that being a prohibited category.[2310.38] [2310.38][S02]And I think that part of the AI Act[2313.13] [2313.13][S02]is actually very reasonable because it points[2315.44] [2315.44][S02]to stuff that we do not want.[2317.07] [2317.07][S02]And it's interesting because the European Union has done[2319.88] [2319.88][S02]this in many different cases.[2321.27] [2321.27][S02]It actually has selected not to have[2324.53] [2324.53][S02]certain kinds of technology.[2326.01] [2326.01][S02]It did with the prohibited categories of AI,[2328.35] [2328.35][S02]but also historically, famously, it has done the same with GMOs.[2331.55] [2331.55][S01] How about within "Google DeepMind, then?[2333.82] [2333.82][S01]Do you have certain applications that you consider off[2336.07] [2336.07][S01]limits here?[2336.65] [2336.65][S02] We have our AI principles.[2338.21] [2338.21][S02]So one example is that there are certain applications of weapons,[2340.63] [2340.63][S02]et cetera, that we wouldn't consider.[2342.35] [2342.35][S02]But generally go into the AI principles[2344.47] [2344.47][S02]and consulting them gives you a sense of how we try to balance[2347.08] [2347.08][S02]the different equities here.[2348.35] [2348.35][S01] I do wonder about in terms of particular projects,[2351.17] [2351.17][S01]how do you decide at "Google DeepMind"[2352.99] [2352.99][S01]which projects you will and won't get involved in?[2355.28] [2355.28][S01]How does that decision process happen?[2357.043] [2357.043][S02] There is a set of principles[2358.96] [2358.96][S02]against which we test ethically the different kinds of things[2362.23] [2362.23][S02]we do.[2362.9] [2362.9][S02]That is then taken to two different councils,[2365.77] [2365.77][S02]actually, that look, review, interview the engineers,[2369.47] [2369.47][S02]set out different conditions, and then give[2371.713] [2371.713][S02]a green light or a yellow light or a red light[2373.63] [2373.63][S02]and say generally, here's how you should think about this[2376.6] [2376.6][S02]from our ethical perspective.[2378.71] [2378.71][S02]So these ethical review boards is[2380.612] [2380.612][S02]something that we're not the only ones who have them,[2382.82] [2382.82][S02]there are others who have them too.[2384.278] [2384.278][S02]They are essentially a way of looking at how different[2386.77] [2386.77][S02]projects can be assessed, what we should do,[2389.137] [2389.137][S02]and if we think they should be, if we should proceed with them[2391.72] [2391.72][S02]or not.[2392.39] [2392.39][S01] If that's the frameworks that[2393.49] [2393.49][S01]exist at the moment, I think it's probably also[2395.448] [2395.448][S01]important to talk about frontier models.[2397.97] [2397.97][S01]So this idea of the most powerful AI models,[2402.115] [2402.115][S01]kind of at the very cutting-edge and the risk[2403.99] [2403.99][S01]that they might pose.[2404.99] [2404.99][S01]And so we got to talk to Demis to kick off this season.[2409.25] [2409.25][S01]And he said that he would recommend[2411.97] [2411.97][S01]kind of beefing up existing regulations at the moment[2414.31] [2414.31][S01]in the domains where we have AI, but also making sure[2420.28] [2420.28][S01]that you understand and test the frontier model simultaneously.[2423.26] [2423.26][S01]And then he said, start regulating around that maybe[2426.13] [2426.13][S01]in a couple of years time.[2427.293] [2427.293][S01]I mean, it's slightly unfair of me[2428.71] [2428.71][S01]to give you a quote from Demis and then[2430.465] [2430.465][S01]ask you to explain what he meant.[2431.84] [2431.84][S02] Yeah, but there you go.[2433.49] [2433.49][S02]Happy to.[2433.9] [2433.9][S02]Yes.[2434.23] [2434.23][S01] But there you go.[2435.438] [2435.438][S01]Why in a couple of years?[2436.73] [2436.73][S01]Why not now?[2437.72] [2437.72][S02] I think that the evidence base[2439.72] [2439.72][S02]we have is simply too slim.[2441.14] [2441.14][S02]I think what he is sort of hinting[2442.72] [2442.72][S02]at is that if we get into the habit of testing[2444.82] [2444.82][S02]these models now, when the capabilities are impressive,[2447.72] [2447.72][S02]but not dangerous, that will be a really good way for us to have[2451.32] [2451.32][S02]built habits, to have built regulation, to have built[2454.02] [2454.02][S02]institutions that can do this.[2456.42] [2456.42][S02]In the UK, for example, the AI Safety Institute[2458.88] [2458.88][S02]was put in place as a government initiative in the UK first,[2463.03] [2463.03][S02]but now there are AI safety institutes in many places,[2465.54] [2465.54][S02]to look at and understand more about foundation models[2469.47] [2469.47][S02]or really powerful frontier models.[2471.37] [2471.37][S02]And they've recruited scientists, recruited policy[2474.27] [2474.27][S02]people to really figure out how the government can learn more[2478.71] [2478.71][S02]about these models.[2480.04] [2480.04][S02]And then you might see different kinds of other capabilities[2482.79] [2482.79][S02]evolve through this particular testing work[2485.31] [2485.31][S02]or through this sort of analysis that we're doing.[2488.23] [2488.23][S02]And that, in turn, will help us to figure out what[2490.95] [2490.95][S02]does good regulation look like.[2492.68] [2492.68][S02]The challenge is the pace of evolution of these models.[2496.03] [2496.03][S02]And you can see that if you do a retrospective and say, OK, let's[2500.52] [2500.52][S02]assume that you were a regulator or a legislator,[2503.13] [2503.13][S02]and you're asked to regulate AI in 2015.[2507.27] [2507.27][S02]That's nine years ago.[2508.98] [2508.98][S02]What do you think that you would produce?[2511.3] [2511.3][S02]And I think that's--[2513.17] [2513.17][S02]if you do that, you'll see easily why he says, wait[2516.36] [2516.36][S02]a couple of years, because the knowledge we have,[2519.13] [2519.13][S02]the evidence we're getting, all of that[2521.55] [2521.55][S02]is going to be different in a couple of years, even,[2524.55] [2524.55][S02]I think to some degree, the unit of analysis.[2528.34] [2528.34][S02]One of the things that we have seen[2530.1] [2530.1][S02]is a huge focus on this notion of the model[2533.07] [2533.07][S02]as the object of regulation.[2535.44] [2535.44][S02]And that is, while it's a sort of a helpful first approximation[2539.61] [2539.61][S02]of where regulation should sit, one[2541.47] [2541.47][S02]of the things we're seeing now is[2543.33] [2543.33][S02]that as the effectiveness of the scaling laws[2546.0] [2546.0][S02]seem to be diminishing, people are building[2548.82] [2548.82][S02]several different models and then[2550.41] [2550.41][S02]having them work together in different ways.[2552.58] [2552.58][S02]So you get architectures that don't have a single model that[2557.16] [2557.16][S02]is feasible to regulate, but where[2559.74] [2559.74][S02]it's a collection, or a set, or a portfolio of models together[2562.79] [2562.79][S02]that produces the capability.[2564.66] [2564.66][S02]It's easy to see that those capabilities that[2566.78] [2566.78][S02]are produced by two models will be different.[2569.01] [2569.01][S02]It's a little bit like thinking about how would you[2571.61] [2571.61][S02]say that you want to regulate a symphony orchestra.[2574.7] [2574.7][S02]Do you regulate the guy with a triangle?[2576.99] [2576.99][S02]Do you regulate the conductor?[2579.12] [2579.12][S02]That seems to be pretty fair, but at some point,[2582.18] [2582.18][S02]the conductor also needs to make sure that everything else is[2585.08] [2585.08][S02]regulated.[2585.98] [2585.98][S02]And so I think to some degree, we're[2587.48] [2587.48][S02]moving into an architectural paradigm that may even[2590.81] [2590.81][S02]challenge the notion that these frontier models in the singular[2596.12] [2596.12][S02]are the right thing to regulate.[2598.05] [2598.05][S01] I mean, you said there a moment ago that we're[2601.01] [2601.01][S01]at the point now where these models are[2604.1] [2604.1][S01]impressive, but not harmful.[2606.12] [2606.12][S01]But then I think that there are some situations[2609.68] [2609.68][S01]in which large language models can be harmful already, right?[2612.66] [2612.66][S01]I mean, I'm thinking here in terms of amplification[2615.14] [2615.14][S01]of misinformation.[2616.14] [2616.14][S01]I'm thinking in terms of bias and disinformation.[2620.37] [2620.37][S01]I mean, is sitting back and waiting[2623.96] [2623.96][S01]until we have further information definitely[2626.45] [2626.45][S01]the right approach, or could we not move now[2630.2] [2630.2][S01]and be willing to be flexible as we go along?[2632.46] [2632.46][S02] Well, you're right and I'm wrong.[2633.63] [2633.63][S02]They can be harmful now.[2634.68] [2634.68][S02]But I think one thing that's important[2636.29] [2636.29][S02]is that I'm not recommending we sit back and wait.[2638.49] [2638.49][S02]One of the things that is also often stresses[2641.18] [2641.18][S02]is that we don't have the right benchmarks or the right tests[2643.82] [2643.82][S02]to figure out how we assess what the models can actually do.[2648.44] [2648.44][S02]And that means that we don't exactly[2649.94] [2649.94][S02]know what the possible harms are.[2652.11] [2652.11][S02]But this we can solve, and we can solve this[2654.92] [2654.92][S02]by attacking the problem now.[2656.7] [2656.7][S02]And that's why the scientific exploration and evaluation[2659.81] [2659.81][S02]of models is going to be incredibly important to invest[2662.6] [2662.6][S02]in and work with over the coming couple of years,[2665.43] [2665.43][S02]together with the public sector and third parties,[2668.81] [2668.81][S02]and together with other companies as well.[2671.97] [2671.97][S02]And that's why we set up the Frontier Model Forum, which[2674.72] [2674.72][S02]is an organization of different kinds of AI companies[2677.78] [2677.78][S02]coming together to try to build these evaluations[2680.65] [2680.65][S02]on a scientific basis.[2682.36] [2682.36][S01] So am I getting the picture here then,[2684.89] [2684.89][S01]that with these frontier models, with the real cutting-edge[2688.24] [2688.24][S01]of artificial intelligence, it's having[2690.403] [2690.403][S01]the discussion about the safety before they're[2692.32] [2692.32][S01]built, while they're built, and after they're built.[2695.15] [2695.15][S01]It's kind of every step of the process?[2696.923] [2696.923][S02] Yes.[2697.84] [2697.84][S02]And I do think that after is more important than it usually[2701.11] [2701.11][S02]is, because there's only so much pre-testing you[2704.26] [2704.26][S02]can do before you've actually built the artifact.[2706.34] [2706.34][S02]If you think about in regulation, you often say that,[2708.548] [2708.548][S02]should we have something be safe by design,[2711.02] [2711.02][S02]make sure that it's absolutely safe from the outset,[2713.66] [2713.66][S02]or should we have it be safe by testing?[2715.91] [2715.91][S02]So we test it afterwards and see if it's safe.[2718.22] [2718.22][S02]You do it differently for cars.[2719.66] [2719.66][S02]Cars you test and you build them safe.[2721.435] [2721.435][S02]You hope that the design is safe,[2722.81] [2722.81][S02]and then you test it a lot.[2723.95] [2723.95][S02]This is even more so.[2725.777] [2725.777][S02]What you want to do is you want to build it,[2727.61] [2727.61][S02]then you want to understand it, you want to test it,[2729.777] [2729.777][S02]you want to explore how it works.[2731.33] [2731.33][S02]You also want to explore how it works[2733.69] [2733.69][S02]in a sociotechnical context.[2735.17] [2735.17][S01] Well, I think, that's[2736.545] [2736.545][S01]the important point, the difference between the example[2739.208] [2739.208][S01]of cars, or I'm thinking pharmaceuticals as well,[2741.25] [2741.25][S01]COVID vaccine, for instance, where[2743.74] [2743.74][S01]you have the opportunity to test it before it[2746.08] [2746.08][S01]gets embedded in society.[2747.89] [2747.89][S01]You have a very particular controlled environment.[2751.48] [2751.48][S01]With some of these frontier models,[2753.04] [2753.04][S01]that might not necessarily always be possible.[2756.64] [2756.64][S02] Well, I think you can not--[2758.563] [2758.563][S02]so that's an interesting question.[2759.98] [2759.98][S02]Can you test it before you release it?[2762.1] [2762.1][S02]We do a ton of testing, and all AI companies do,[2765.1] [2765.1][S02]before we put anything out to the public[2767.59] [2767.59][S02]or make it generally available.[2769.495] [2769.495][S02]And that's one part of the processes[2770.995] [2770.995][S02]that you said Demis talks about, we[2772.75] [2772.75][S02]build the institutional habits of making sure[2775.42] [2775.42][S02]that we test, we red team with selected third parties,[2778.49] [2778.49][S02]we work with the AI safety institutes,[2780.44] [2780.44][S02]and we have this sort of period where we test it.[2783.2] [2783.2][S02]It's not as if we test it by throwing it out to market.[2785.812] [2785.812][S01] On that note, though, I mean,[2787.52] [2787.52][S01]there are people who have said that actually we[2789.73] [2789.73][S01]should be building kill switches into these things.[2791.87] [2791.87][S01]And that actually perhaps that should even[2793.62] [2793.62][S01]be part of regulatory control, is that an insistence on that.[2796.3] [2796.3][S01]What's your take on that idea?[2797.555] [2797.555][S02] I don't know what that would look like.[2799.93] [2799.93][S02]I would have to look at some kind of computer science[2802.41] [2802.41][S02]proof that shows me that a kill switch can be really effective.[2805.78] [2805.78][S02]Now, if we're talking about a system that's[2808.08] [2808.08][S02]sort of vastly more intelligent than us,[2810.33] [2810.33][S02]then I think it's probably hard to build a kill[2812.61] [2812.61][S02]switch into that system.[2814.722] [2814.722][S02]I can build a kill switch into my lamp.[2817.0] [2817.0][S02]It's the on/off button.[2818.12] [2818.12][S02]That's fine.[2818.62] [2818.62][S02]I can do that.[2819.48] [2819.48][S02]My lamp is likely to not outsmart me often,[2822.52] [2822.52][S02]and so I can do that.[2823.63] [2823.63][S02]But what do I do with the system that is not just highly complex,[2827.08] [2827.08][S02]but also distributed?[2828.588] [2828.588][S02]The notion of a kill switch goes back[2830.13] [2830.13][S02]to what we talked about before, that there's[2831.963] [2831.963][S02]a single model we can regulate.[2833.65] [2833.65][S02]Now, what if the model actually uses, say, 10, 20, 200,[2839.26] [2839.26][S02]200,000 different kinds of nodes in a network?[2842.89] [2842.89][S02]Where do you put the kill switch?[2845.25] [2845.25][S02]I mean, the way that people now try[2847.47] [2847.47][S02]to shut off the internet is by shutting it off at the border[2850.71] [2850.71][S02]and essentially saying, we will have no internet whatsoever,[2854.13] [2854.13][S02]because it's highly distributed.[2856.51] [2856.51][S02]A system like this, an AGI that is basically[2859.71] [2859.71][S02]a network of different kinds of models,[2861.7] [2861.7][S02]will be even more distributed and will have a natural tendency[2865.83] [2865.83][S02]to allocate or reallocate its tasks[2869.55] [2869.55][S02]or its capabilities across this network if any piece of it[2872.28] [2872.28][S02]is shut down.[2873.45] [2873.45][S01] If we do watch and wait,[2875.68] [2875.68][S01]I mean, are there any emerging capabilities[2878.22] [2878.22][S01]that you are particularly concerned about?[2880.83] [2880.83][S02] I think it's worth[2882.33] [2882.33][S02]looking more closely at persuasion and deception,[2885.76] [2885.76][S02]not because I think that they in themselves[2888.09] [2888.09][S02]are extremely dangerous, but because we need to understand[2890.65] [2890.65][S02]so much more about both persuasion and deception[2893.85] [2893.85][S02]and how they work if there is technology or AI in the mix.[2898.62] [2898.62][S01] But this does sound one area[2900.84] [2900.84][S01]in which there are some very clear sort of blanket rules that[2905.46] [2905.46][S01]might actually be useful.[2907.0] [2907.0][S01]A model should always let you know that it's a model.[2910.11] [2910.11][S01]You should never pretend to be a human.[2911.86] [2911.86][S01]Deception should never happen.[2913.11] [2913.11][S01]I mean, these sound sort of beginning[2916.01] [2916.01][S01]to get towards sort of regulatory ideas.[2919.898] [2919.898][S02] And the declaration[2921.44] [2921.44][S02]that you're interacting with an AI[2922.64] [2922.64][S02]is actually a part of many regulations[2924.29] [2924.29][S02]today, which I think is not a bad thing.[2926.593] [2926.593][S01] We started off by talking about the four[2928.76] [2928.76][S01]different approaches that you see globally towards regulation,[2931.62] [2931.62][S01]and the experiment that they are effectively[2934.04] [2934.04][S01]running in real time.[2936.162] [2936.162][S01]Do you think that we're going to get to a point[2938.12] [2938.12][S01]where international bodies will collaborate on AI regulation,[2941.7] [2941.7][S01]or do you think that we'll stay in this situation[2944.888] [2944.888][S01]where different regions of the world approach it differently?[2947.43] [2947.43][S02] There are different attempts[2949.347] [2949.347][S02]at international collaboration through the OECD,[2951.95] [2951.95][S02]which is not all of the countries in the world,[2954.32] [2954.32][S02]through the UN, for the G7.[2956.25] [2956.25][S02]So there are these nascent attempts[2958.16] [2958.16][S02]to try to figure out what are the overarching rules[2961.082] [2961.082][S02]that we want to make sure are true.[2962.54] [2962.54][S02]Things like, AI should benefit humanity.[2964.76] [2964.76][S02]Very, very general, but still some kind[2967.58] [2967.58][S02]of at least indication to what we think the policy tonality[2971.0] [2971.0][S02]should be.[2972.44] [2972.44][S02]I think that there will be challenges here,[2974.88] [2974.88][S02]and I think those challenges are exacerbated[2976.76] [2976.76][S02]by geopolitical pressures and tensions.[2978.95] [2978.95][S02]They're also exacerbated by the fact[2980.45] [2980.45][S02]that there is a lot of national competitiveness involved.[2983.7] [2983.7][S02]And so we're not in a period of human history[2987.86] [2987.86][S02]where multilateral institutions are super strong.[2990.48] [2990.48][S02]So while I'm very happy that we have[2993.553] [2993.553][S02]these international collaborations[2994.97] [2994.97][S02]and the early attempts, I'm not entirely sure[2997.91] [2997.91][S02]how we progress from the very general agreements[3001.3] [3001.3][S02]to, for example, limits on the use of AI in simple situations.[3005.75] [3005.75][S02]Say that you wanted to take the European AI Act[3008.02] [3008.02][S02]and say we should universally outlaw social scoring with AI.[3013.42] [3013.42][S02]Would that work or are there countries[3015.67] [3015.67][S02]that actually believe that social scoring is a net good?[3018.32] [3018.32][S02]Well, there are, and so we should think through,[3020.89] [3020.89][S02]where are the areas where international collaboration can[3023.56] [3023.56][S02]happen?[3024.38] [3024.38][S01] I mean, one place that has not yet[3027.972] [3027.972][S01]planted their flag about regulation at all is the UK.[3031.403] [3031.403][S01]Do you think that when it does, it[3032.82] [3032.82][S01]will end up being closer to the EU or to the US?[3037.71] [3037.71][S02] The sort of approach in the UK has been[3040.603] [3040.603][S02]advertised by the new Labour government as something where[3043.02] [3043.02][S02]they're going to put a UK bill out before the end of the year[3046.05] [3046.05][S02]for consultation.[3047.41] [3047.41][S02]And they're looking at a set of narrow regulations[3051.48] [3051.48][S02]that will put them closer to the US, I think,[3053.95] [3053.95][S02]than to the EU, which is interesting in many different[3057.2] [3057.2][S02]ways.[3057.7] [3057.7][S02]And then I think they'll continue to push.[3059.88] [3059.88][S02]And this I really think is important.[3061.542] [3061.542][S02]They'll continue to push for the sectoral application[3063.75] [3063.75][S02]of the rules, because I think there[3065.37] [3065.37][S02]is a lot of value in how a sector or a set of professionals[3069.09] [3069.09][S02]work with this.[3070.436] [3070.436][S02]I think it would be very, very valuable[3072.99] [3072.99][S02]to have a set of nurses or teachers[3074.52] [3074.52][S02]talk about what would make most sense to them[3076.83] [3076.83][S02]in their professional day to day when[3079.17] [3079.17][S02]it comes to the use of these technologies.[3081.13] [3081.13][S02]That kind of normative regulation[3083.94] [3083.94][S02]is actually really important.[3085.66] [3085.66][S02]And if you can encourage that on top[3087.57] [3087.57][S02]of whatever narrow regulation you believe is needed,[3090.25] [3090.25][S02]I think you can actually progress in[3092.22] [3092.22][S02]what is another objective of regulation,[3094.51] [3094.51][S02]and that is the diffusion of the technology in the economy.[3097.18] [3097.18][S02]One way to measure the effectiveness and the value[3099.9] [3099.9][S02]of regulation is actually to look[3101.85] [3101.85][S02]at how fast is this technology diffusing through the economy.[3105.395] [3105.395][S02]And that's something that I think[3106.77] [3106.77][S02]we don't discuss often enough, because technology diffusion[3109.74] [3109.74][S02]also unlocks benefits and also unlocks welfare.[3112.3] [3112.3][S02]And so as we look at different kinds of regulation,[3116.02] [3116.02][S02]that's actually one, not the only one,[3117.87] [3117.87][S02]but one interesting metric to think through.[3120.113] [3120.113][S01] I mean, I think what's[3121.53] [3121.53][S01]been really interesting about this conversation is[3123.613] [3123.613][S01]that I don't feel as though there are[3125.37] [3125.37][S01]any aspects of regulation where you're like, no,[3127.72] [3127.72][S01]that's definitely a bad idea.[3129.357] [3129.357][S01]I mean, it's still sort of feels like you're[3131.19] [3131.19][S01]open to all of the potential options on the table.[3133.45] [3133.45][S01]I do wonder, though, in terms of-- is that fair?[3135.485] [3135.485][S02] I have no real views.[3137.11] [3137.11][S02]Yes.[3137.61] [3137.61][S01] No, no, no, no, no.[3138.96] [3138.96][S02] No, no.[3139.36] [3139.36][S02]But I do think there are bad things that you can do.[3141.28] [3141.28][S02]And one of them is banning technology.[3142.9] [3142.9][S02]I think that's always wrong.[3144.19] [3144.19][S02]So there's a hard line.[3145.653] [3145.653][S02]You shouldn't ban technology because you[3147.32] [3147.32][S02]don't know how it works.[3148.77] [3148.77][S02]I also think it's probably wrong to try[3151.19] [3151.19][S02]to regulate too much at the data level[3153.11] [3153.11][S02]and say, these data sets must be absolutely perfect,[3156.228] [3156.228][S02]because I think that will be really, really hard to do.[3158.52] [3158.52][S02]There are some aspects of that in the EU AI Act[3161.317] [3161.317][S02]that I think will be really difficult,[3162.9] [3162.9][S02]but can be cleared up through government practice[3165.65] [3165.65][S02]and implementation in different ways.[3167.49] [3167.49][S02]But it is also, to be honest, a wide open field.[3171.57] [3171.57][S02]And we need to not foreclose any of the models we have,[3176.34] [3176.34][S02]but explore them equally and figure this out as we go along.[3179.793] [3179.793][S01] But then on the flip side of that,[3181.71] [3181.71][S01]what do you think is the most urgent thing that[3183.668] [3183.668][S01]requires attention?[3184.47] [3184.47][S01]I mean, the wait and see slowly, slowly, nice[3188.12] [3188.12][S01]and carefully is all good.[3189.93] [3189.93][S01]But where do you see that there's the most urgency?[3192.503] [3192.503][S02] I think building the institutions,[3194.67] [3194.67][S02]making sure we have the science that makes it possible[3196.97] [3196.97][S02]for us to understand this technology,[3198.63] [3198.63][S02]whether it's a single model or a network of models,[3201.03] [3201.03][S02]making sure that there are ways of testing and benchmarking[3203.66] [3203.66][S02]the system, not sitting and waiting,[3205.47] [3205.47][S02]but constantly building up the public knowledge about these[3209.79] [3209.79][S02]together with the private sector,[3211.38] [3211.38][S02]I think is super important.[3213.03] [3213.03][S02]I want a regulatory curiosity, and I[3214.94] [3214.94][S02]think that's actually what we have[3217.52] [3217.52][S02]in a lot of the different regions we described.[3219.59] [3219.59][S02]Regulatory curiosity, understanding the technology,[3222.012] [3222.012][S02]figuring out where are the levers that we could[3223.97] [3223.97][S02]pull if we needed to pull them.[3225.54] [3225.54][S02]What lever is better than another lever?[3227.28] [3227.28][S02]Is it better to regulate the data, the algorithm, the system,[3230.28] [3230.28][S02]the user, the deployment?[3231.81] [3231.81][S02]Where in the value chain should we be?[3233.75] [3233.75][S02]That regulatory curiosity is absolutely[3236.06] [3236.06][S02]essential and urgent because the technology[3238.25] [3238.25][S02]is developing so fast.[3239.727] [3239.727][S01] What a fascinating conversation.[3241.56] [3241.56][S01]Thank you so much.[3242.55] [3242.55][S01]That was really, really wonderful.[3244.06] [3244.06][S01]Thank you.[3244.56] [3244.56][S02] Thanks for having me.[3245.31] [3245.31][S02]Thank you.[3246.205] [3246.205][S01] That was a discussion[3247.58] [3247.58][S01]that probably left us with more questions than answers.[3250.47] [3250.47][S01]But I do think that one takeaway for me,[3253.41] [3253.41][S01]anyway, is the extraordinary complexity that[3256.97] [3256.97][S01]is involved in regulating this technology,[3259.58] [3259.58][S01]to allow for benefits while mitigating against harms.[3263.16] [3263.16][S01]And there really aren't simple solutions here.[3266.37] [3266.37][S01]There aren't any quick wins, no silver bullets,[3268.77] [3268.77][S01]no ideas that are even guaranteed[3270.92] [3270.92][S01]to continue to work across time.[3273.9] [3273.9][S01]Everything is a balance here in every direction.[3277.35] [3277.35][S01]But as hard as this will be to get right,[3280.89] [3280.89][S01]the one thing we can be sure of, as Nicklas[3283.19] [3283.19][S01]said, the only way to proceed without any risk of harms[3288.05] [3288.05][S01]is to not proceed at all.[3290.775] [3290.775][S01]You have been listening to "Google DeepMind--[3292.65] [3292.65][S01]The Podcast" with me, Professor Hannah Fry.[3295.1] [3295.1][S01]If you enjoyed that episode, then[3296.99] [3296.99][S01]do subscribe to our YouTube channel.[3299.1] [3299.1][S01]You can also find us on your favorite podcast platform.[3302.82] [3302.82][S01]And we have got plenty more episodes[3305.6] [3305.6][S01]on a whole range of topics to come, so do check those out too.[3309.63] [3309.63][S01]See you next time![3310.64] [3310.64][MUSIC PLAYING][3314.59]