[7.674][S02] Welcome back to the 6th episode of DeepMind, the podcast.[11.812] [12.312][S02]My name is Hannah Fry, I am a mathematician[14.515] [14.515][S02]who’s worked with data and algorithms for the last decade or so.[18.318] [18.318][S02]And I spent the last year at DeepMind -[21.054] [21.054][S02]an organisation that is trying to solve intelligence[24.091] [24.091][S02]and then use it to solve some of society’s problems.[27.594] [30.898][S02]There are an awful lot of people[32.533] [32.533][S02]working in the field of artificial intelligence,[35.469] [35.469][S02]moving forward our understanding of the whole area[39.239] [39.239][S02]and for many of them it is a terrifically exciting place to be.[43.11] [43.11][S02]We’re breaking new frontiers of problem solving and seeing great leaps ahead.[48.882] [48.882][S02]But before any of that hits the outside world,[51.118] [51.118][S02]the first inklings of new breakthroughs here at DeepMind[55.122] [55.122][S02]come in the form of regular poster sessions.[58.358] [60.127][S01] Our goal here was to um have a model[62.629] [62.629][S01]that can carry out the following simple tasks[64.932] [64.932][S01]where I’m going to give you a number and a secret symbol[67.601] [67.601][S01]and what the sum between those two are,[69.703] [69.703][S01]and you have to infer from that what the value of the symbol is,[72.84] [72.84][S01]but it requires our agents to have some properties[75.175] [75.175][S01]that we think are desirable like learning to learn,[78.579] [78.579][S01]like having a memory and processing those memories ourselves.[81.048] [81.048][S04] We’re studying analogical reasoning -[84.184] [84.184][S04]analogical reasoning is very important[85.452] [85.452][S04]because it’s a key to scientific discovery and also human reasoning -[90.457] [90.457][S04]the main question we ask is how can we design neural networks[93.393] [93.393][S04]that are able to do analogical reasoning.[94.962] [94.962][S01] My poster is about verification of neural networks.[98.298] [98.298][S01]In this day and age, when we deploy neural networks[100.734] [100.734][S01]into the real world applications,[102.369] [102.369][S01]we want to make sure that these neural networks are safe,[105.239] [105.239][S01]for example if you have an image classifier,[107.674] [107.674][S01]we don’t ever want to predict a cat to be like a car or something like that.[111.144] [111.144][S01] I always say DeepMind is a bit like academia on steroids,[113.614] [113.614][S01]like it is still academia, but we have a lot of compute,[116.149] [116.149][S01]a lot of great people clustered together,[118.185] [118.185][S01]a lot of help to manage ourselves, so yeah.[121.154] [124.892][S02] while there is obvious excitement about AI research,[128.061] [128.061][S02]this new era of artificial intelligence also comes with concerns.[132.833] [132.833][S02]There is an ease about the way it might be implemented, used and abused.[138.338] [138.338][S02]For the rest of this episode,[139.706] [139.706][S02]we are looking at the more human side of technology,[143.043] [143.043][S02]and the fight to find a future of AI that works for everyone.[148.182] [148.182][S02]In 2017, DeepMind set up dedicated teams[151.885] [151.885][S02]working on how AI impacts ethics and society.[155.889] [155.889][S02]With the aim of making sure that the algorithms designed in this building[159.826] [159.826][S02]are a positive force for good.[162.229] [162.93][S02]But hang on - I know what you are thinking -[164.765] [164.765][S02]surely algorithms aren’t ever good or bad in and of themselves,[169.102] [169.102][S02]it’s how they’re used that matters.[171.638] [171.638][S02]After all, GPS was invented to launch nuclear missiles,[175.609] [175.609][S02]and now helps to deliver pizzas.[178.111] [178.111][S02]And speakers playing pop music on repeat have been deployed as a torture device.[184.251] [184.251][S02]Isn’t the technology itself just neutral?[188.055] [188.055][S06] Good question![188.956] [188.956][S06]And um I think something a lot of people say and believe[195.495] [195.495][S06]and I can see why they say that.[198.432] [198.432][S02] This is Verity Harding, co-lead of DeepMind Ethics and Society.[203.003] [203.003][S06] I think there’s a famous saying about as long as there’s been fire,[207.241] [207.241][S06]there’s been arson.[208.542] [208.542][S06]You can use something that’s for good you can use it for bad.[211.144] [211.144][S06]But It think actually is we’re developing increasingly[213.714] [213.714][S06]sophisticated technologies that have real impact on people’s lives.[218.385] [218.385][S06]It’s not really an acceptable thing to say.[220.888] [220.888][S06]You can’t be building something that’s going to have[223.257] [223.257][S06]this kind of monumental impact - or potentially transformative impact[227.895] [227.895][S06]and not care about how it’s going to be used.[232.666] [232.666][S02] Is that part of the concern then?[234.034] [234.034][S02]That technology that might have been built for one purpose[237.304] [237.304][S02]ends up being used in a different way?[240.574] [240.574][S06] I think definitely that’s some of it.[242.242] [242.242][S06]I think definitely that’s some of it. Because you could force a situation[246.38] [246.38][S06]where you’re building facial recognition tool[249.917] [249.917][S06]because you want to allow somebody to quickly find pictures[253.453] [253.453][S06]of their husband or wife or mom or dad and that’s a great thing[258.892] [258.892][S06]but that facial recognition technology, once developed,[262.229] [262.229][S06]could of course be used to target political dissidents[265.265] [265.265][S06]and pick them out of a crowd and you know, so I think,[267.634] [267.634][S06]that’s definitely one of the concerns[269.236] [269.236][S06]that you might create something for one purpose and it be used for another.[272.973] [275.742][S02] On the topic of facial recognition, Brad Smith,[278.779] [278.779][S02]the President of Microsoft,[280.647] [280.647][S02]recently refused a request by a US police department[284.218] [284.218][S02]to install their algorithm in cop cars and body cameras,[288.155] [288.155][S02]and he's publicly called for more careful thought and societal dialogue[292.559] [292.559][S02]about potentially regulating the use of the technology.[296.496] [296.496][S02]And here at DeepMind more generally there is a strong sense[299.533] [299.533][S02]that the people behind the science have a duty to investigate the wider[304.605] [304.605][S02]and perhaps less predictable impacts of their work.[308.008] [308.008][S06] I don’t think it’s okay to build something -[311.078] [311.078][S06]whether that be a product or a service and put it out there[314.581] [314.581][S06]and and just hope that you make the world a better place.[318.218] [318.218][S06]It think it’s important that you are deliberate and intentional[321.989] [321.989][S06]about why you’re building this, who are you building it for,[325.459] [325.459][S06]what are you hoping to do?[327.027] [327.027][S06]What is your intention with this technology?[329.83] [329.83][S06]And if you start from that premise[331.064] [331.064][S06]then I think you are more likely to get to a better outcome[334.001] [334.001][S06]where you do the good that you hoped you were going to do.[336.703] [336.703][S02] The problem is that without these steps,[339.239] [339.239][S02]it’s very easy for unintentional consequences to creep up on you.[343.877] [343.877][S02]You only need to look at the new stories about social media[346.78] [346.78][S02]from the past few years to see just how much algorithms[350.984] [350.984][S02]have changed our society in unexpected ways.[355.155] [355.155][S02]Lila Ibrahim is DeepMind COO[357.591] [357.591][S02]and has over 20 years’ experience working in the tech sector.[361.028] [361.028][S02]She has seen first hand how hard a booming industry[364.498] [364.498][S02]has found it to keep up with being responsible.[368.302] [368.302][S03] In 2006 I went into the middle of the Amazon[371.305] [371.305][S03]and we built a computer lab and health care,[374.842] [374.842][S03]so we put in internet and computers, etc.[377.678] [377.678][S03]and we knew we had a responsibility not to just leave it there[380.347] [380.347][S03]but to train folks to take care of it, to think about the sustainability.[385.385] [385.385][S03]But I think that’s kind of where things tend to end.[388.088] [388.088][S03]Ethics means something very different now,[390.224] [390.224][S03]and responsibility means something very different now[392.459] [392.459][S03]because technology is in everybody’s hands.[394.695] [394.695][S03]It’s no longer limited to a few people for a specific application.[400.234] [400.234][S03]Um it’s a lot easier to get into the tech sector[402.669] [402.669][S03]and to make technology that can have value to people,[406.106] [406.106][S03]and at the same time that comes a lot of responsibility[408.742] [408.742][S03]that I don't think in general the tech sector has taken into account.[412.579] [414.014][S02] But the last few years have shown[415.716] [415.716][S02]how hugely transformative and disruptive AI can be[420.921] [420.921][S02]and brought sharply into focus the very possible negative[424.491] [424.491][S02]outcomes of ill-thought through technology.[428.295] [428.295][S02]But as Verity told me, the tide is slowly beginning to turn.[433.066] [433.066][S02]Much of the drive for a conversation about Ethics[436.336] [436.336][S02]is coming from within the technology community itself.[440.007] [440.007][S06] 3 years ago in 2016 some of the scientists from different[445.412] [445.412][S06]ah labs at different companies met at a conference[448.549] [448.549][S06]and were talking about how excited they were about the potential for AI[452.819] [452.819][S06]to do a lot of good, ah but acknowledging that -[456.156] [456.156][S06]a technology that’s so powerful that it has the potential[459.726] [459.726][S06]to be transformative in a very good way must also have the potential[463.397] [463.397][S06]to be very transformative in in in not so good a way,[467.267] [467.267][S06]and so they came together to say well what can we do about it?[470.804] [470.804][S02] And so the partnership on AI was born.[474.675] [474.675][S02]It includes members from Amnesty International,[477.11] [477.11][S02]Electronic Frontier Foundation,[478.779] [478.779][S02]the BBC and Princeton University amongst many, many others.[483.584] [483.584][S02]And together they are hoping to come up with best practices in AI[487.754] [487.754][S02]making sure that society stays firmly at the forefront of engineers’ minds.[493.493] [493.493][S06] So the partnership of AI interestingly was founded[496.997] [496.997][S06]by the biggest tech companies so it was founded by DeepMind[501.201] [501.201][S06]but also Google, Facebook, Amazon, IBM and Apple.[506.306] [506.306][S06]One thing that’s really interesting about the partnership and AI[508.475] [508.475][S06]is that the board membership is made up of independent board members[512.846] [512.846][S06]and representatives of the company[514.915] [514.915][S06]and so it’s creating a space where those different groups[519.62] [519.62][S06]aren’t siloed from each other, having debates in different rooms[522.99] [522.99][S06]and not listening but somewhere[524.958] [524.958][S06]where honest people with the best of intentions[528.228] [528.228][S06]can come together and challenge each other[530.297] [530.297][S06]and scrutinise each other and hold each other accountable[532.666] [532.666][S06]but also have frank, open honest debate[536.27] [536.27][S06]about issues where reasonable people can disagree.[539.439] [539.439][S06]I really believe that the outcome of that will be better decision[542.242] [542.242][S06]making both in companies but elsewhere as well.[545.579] [545.579][S02] Does it sometimes get quite heated in those conversations?[549.316] [549.316][S06] You know my experience of it is that it doesn’t get heated[552.719] [552.719][S06]but it’s passionate, so people aren’t angry with each other[557.157] [557.157][S06]and and there’s not aggressive argument,[560.694] [560.694][S06]but people are very honest, and very open and very challenging.[564.198] [564.198][S06]But that’s been received really well in all cases.[568.101] [568.101][S02] How do you protect against rogue companies[572.472] [572.472][S02]just who are not part of these groups, just doing whatever they want.[577.411] [577.411][S06] If enough companies and enough groups sign up to something[581.048] [581.048][S06]and it becomes the norm,[582.382] [582.382][S06]it’s then really obvious when people aren’t doing it.[584.885] [584.885][S06]And I do think people are kind of being called out for that -[588.555] [588.555][S06]it will no longer be tenable to not operate[591.892] [591.892][S06]in the way that everybody else is operating.[594.294] [596.363][S02] It’s not just theoretical concerns about runaway[599.433] [599.433][S02]applications of AI that’s prompting these conversations,[602.836] [602.836][S02]but real examples of algorithms[605.105] [605.105][S02]that have already been let loose on the world with real question marks[609.543] [609.543][S02]about whether their benefits outweigh their harm.[612.813] [612.813][S02]A notorious example is the use of AI in the criminal justice system -[618.185] [618.185][S02]now you may have heard of these algorithms already.[620.32] [620.32][S02]When a defendant appears in court, the AI can assess a defendant’s chances[624.958] [624.958][S02]of going on to commit another crime in future,[628.695] [628.695][S02]and that risk score is then used by a judge to help decide[632.432] [632.432][S02]whether the defendant should be awarded bail,[635.002] [635.002][S02]and in some cases, how long someone’s sentence should be.[639.006] [639.873][S02]There is good justification for something like this[642.643] [642.643][S02]because there is an enormous amount of luck[645.078] [645.078][S02]involved in the human judicial system.[648.415] [649.349][S02]Studies have shown that if you take the same case to a different judge,[653.187] [653.187][S02]you will often get a different response.[656.056] [656.056][S02]If you take the same case to the same judge on a different day,[660.127] [660.127][S02]you’ll often get a different response.[662.296] [662.296][S02]Judges don't like giving the same response too many times in a row,[665.666] [665.666][S02]and so if a series of successful cases of bail hearings have gone before you,[670.404] [670.404][S02]your chances of being successful fall[673.574] [673.574][S02]and there is even evidence to suggest that judges tend to be a lot stricter[678.111] [678.111][S02]in towns where the local sports team has lost recently.[682.282] [682.282][S02]Using AI to help make these decisions can help to eliminate[686.553] [686.553][S02]a lot of that inconsistency, but you have to tread pretty carefully.[691.658] [691.658][S06] if you without thought and care[695.262] [695.262][S06]and due attention to the history of racial prejudice[698.832] [698.832][S06]in the criminal justice system, build something that claims to be able[702.97] [702.97][S06]to predict somebody’s likelihood of reform and rehabilitation,[708.408] [708.408][S06]and reoffending, then it is likely at least in my view[713.38] [713.38][S06]that that’s going to fail.[715.782] [715.782][S06]If you build something with the intention of addressing those biases,[719.052] [719.052][S06]and you work to include the community in some way,[721.788] [721.788][S06]there could then potentially be a beneficial outcome maybe,[724.725] [724.725][S06]but I haven’t seen it yet.[726.46] [726.46][S02] And by fail you’re really talking about[729.263] [729.263][S02]treating black defendants differently to white defendants.[732.132] [732.132][S06] Absolutely! And once you tend to look at the algorithms[735.836] [735.836][S06]and the data that they’ve been, they’ve been built on,[738.739] [738.739][S06]um oftentimes you can see where they were built on data[740.607] [740.607][S06]that was already biased of course this was the outcome.[743.377] [745.746][S02] The issue came to public attention in 2016[749.183] [749.183][S02]after a group of US investigative journalists from Pro Public published[753.153] [753.153][S02]a damning report of one particular company’s criminal risk scores.[758.525] [758.525][S02]Their study showed that the algorithm was twice[761.361] [761.361][S02]as likely to wrongly categorise black defendants[765.566] [765.566][S02]as being likely to re-offend than white defendants.[769.903] [769.903][S02]Now I should just point out that DeepMind does not build these systems,[773.607] [773.607][S02]but the whole industry alongside the partnership of AI[776.476] [776.476][S02]has been part of the conversation about how to address them.[780.447] [781.014][S02]One of those people is William Issac, a social scientist at DeepMind.[785.185] [785.185][S02]He says that the 2016 ProPublica investigation[788.555] [788.555][S02]made people realise that switching over to algorithms[791.758] [791.758][S02]doesn’t make decisions any more objective.[794.928] [794.928][S07] Even with AI and ML tools[798.098] [798.098][S07]you are getting into the social environment[801.802] [801.802][S07]where you actually have the same norms,[804.438] [804.438][S07]the same kind of like systematic biases, they’re still all present.[808.408] [808.408][S07]So it’s really hard to say that somehow this will replace[812.412] [812.412][S07]all of the kind of subjective,[814.548] [814.548][S07]preconceived notions about certain groups,[816.016] [816.016][S07]or historical biases against them,[818.218] [818.218][S07]and that you can start all over again and so I think that was the wakeup call[822.456] [822.456][S07]was that it’s not as objective as it seems and that as a result,[827.461] [827.461][S07]we still have to grapple with those questions.[830.364] [831.064][S02] The problem is that the data[832.366] [832.366][S02]which gives the algorithm predictive abilities[835.335] [835.335][S02]are questions like how many times were you arrested as a juvenile,[840.04] [840.04][S02]but if you are say a young, black man in America,[843.844] [843.844][S02]it doesn’t matter how law-abiding you are,[846.28] [846.28][S02]the chances are that you will have had many more negative interactions[849.65] [849.65][S02]with the police than someone exactly like you,[853.086] [853.086][S02]who happens to be white.[855.255] [855.255][S02]And if you’re using that data to dictate who deserves to be given bail or not,[860.394] [860.394][S02]then you are in serious risk of perpetuating[863.397] [863.397][S02]societal imbalances going forwards.[867.0] [867.0][S02]This is Silvia Ciappia, a staff research scientist at DeepMind.[870.871] [870.871][S05] Researchers don’t fully understand what this fairness[874.675] [874.675][S05]is about they also look like messier in the sense that it involves,[878.812] [878.812][S05]it is not purely technical problem[881.815] [881.815][S05]and it’s very difficult to understand what how to define fairness,[886.486] [886.486][S05]and it’s difficult to separate the technical part[888.789] [888.789][S05]from the ethical one.[891.425] [894.094][S02] This is an important point,[896.096] [896.096][S02]because defining exactly what you mean by “fair” is surprisingly tricky.[901.902] [901.902][S02]Of course you’d want an algorithm[903.303] [903.303][S02]that makes equally accurate predictions for black and white defendants,[907.407] [907.407][S02]the algorithm should also be equally good[909.276] [909.276][S02]at picking out the defendants[911.011] [911.011][S02]who are likely to reoffend whatever racial group they belong to.[915.415] [915.415][S02]And as Pro Publica pointed out,[917.284] [917.284][S02]the algorithm should make the same kind of mistakes[920.587] [920.587][S02]at the same rate for everyone regardless of race.[925.125] [925.125][S02]Ethically you’d want all of those things to be true,[928.562] [928.562][S02]but technically that’s not always going to be possible.[932.165] [932.165][S02]If you’re data set has bias in it, there are some kinds of fairness[936.436] [936.436][S02]that are mathematically incompatible with others.[939.806] [939.806][S02]And even if you could guarantee all of these things,[942.776] [942.776][S02]there are still a number of ethical issues to contend with.[945.846] [945.846][S02]How do you measure fairness, who’s excluded from your definition,[949.65] [949.65][S02]how do you make those decisions transparent,[952.186] [952.186][S02]and ultimately how do people contest[955.088] [955.088][S02]the decisions made by those algorithms?[958.492] [958.492][S02]See, I told you it was tricky.[961.228] [961.228][S02]Coming at this from two very different perspectives,[964.164] [964.164][S02]William and Silvia started looking into the bigger issue[967.334] [967.334][S02]of fairness in algorithms.[969.336] [969.336][S07] Even though we had kind of different frameworks,[971.405] [971.405][S07]me as a social scientist and Silvia as a machine learning researcher,[974.942] [974.942][S07]the actual overlap between how we would approach this[979.112] [979.112][S07]and basically the assumptions that are embedded within it[981.648] [981.648][S07]were remarkably similar and actually part of what we’re saying[985.419] [985.419][S07]is like oh look at these papers in social science[987.554] [987.554][S07]that are kind of making this same point,[989.623] [989.623][S07]they just hadn’t actually had a way to actually communicate that formally.[994.161] [995.596][S02] You’re listening to a podcast from the people at DeepMind.[999.032] [1000.434][S02]In April 2019, William and Silvia co-published a paper on fairness in AI[1005.372] [1005.372][S02]entitled a causal Bayesian network’s viewpoint on fairness.[1009.843] [1009.843][S02]In it they show that no matter how fair algorithms might be,[1013.68] [1013.68][S02]if the data they’re learning from is biased,[1016.45] [1016.45][S02]we still can’t trust their results.[1019.353] [1019.353][S05] I don’t think it’s possible to find technical solutions[1023.824] [1023.824][S05]that are completely satisfactory.[1026.293] [1026.293][S05]At some point we need to take decisions[1030.063] [1030.063][S05]whether the kind of fairness is acceptable or not,[1033.4] [1033.4][S05]but we can advance a lot[1035.169] [1035.169][S05]and that’s why we need more researchers involved[1037.905] [1037.905][S05]- and not just machine learning researchers[1039.606] [1039.606][S05]but researchers from different communities[1042.442] [1042.442][S05]to be raising awareness about this this problem, find solutions,[1048.148] [1048.148][S05]but as we would never be able to find completely satisfactory solutions[1052.853] [1052.853][S05]from a technical viewpoint,[1054.154] [1054.154][S05]and at that point we need to take decisions[1055.923] [1055.923][S05]- is it important to talk about these such that, that we are -[1060.46] [1060.46][S05]something that is missing at the moment.[1063.163] [1063.163][S07] I do think this is fundamentally[1065.732] [1065.732][S07]like a societal, ethical question and challenge.[1069.236] [1069.236][S07]And it will require lots of stakeholders to address.[1072.773] [1072.773][S07]If you have let’s say a data set of facial recognition tool[1075.809] [1075.809][S07]that’s designed to find missing children -[1077.945] [1077.945][S07]what threshold do you say as a society well[1079.913] [1079.913][S07]you say okay this is acceptable, if we maybe are less successful[1083.817] [1083.817][S07]at identifying children with darker faces,[1086.887] [1086.887][S07]what threshold do we say that’s acceptable,[1089.256] [1089.256][S07]because that’s not a technical question,[1091.291] [1091.291][S07]that’s a social and political question, a normative question.[1095.195] [1095.195][S07]Even if you do have a classifier or a facial recognition software[1098.665] [1098.665][S07]that’s fair, the application of it may be in unfair ways.[1102.769] [1102.769][S07]And so that might present a second question[1105.072] [1105.072][S07]that is separate from the actual kind of like[1107.908] [1107.908][S07]if you decide on a threshold[1109.51] [1109.51][S07]that if you’re just using it in a neighbourhood[1111.545] [1111.545][S07]in a predominantly one group or one ethnicity,[1114.581] [1114.581][S07]that presents a whole other set of challenges[1117.117] [1117.117][S07]for whether or not that’s an ethical use of a particular technology.[1121.021] [1121.021][S02] You can’t assess whether these algorithms are good or bad in isolation,[1125.692] [1125.692][S02]they don’t exist on their own.[1128.028] [1128.028][S02]You have to place them in the context of the worlds that they are being used,[1132.399] [1132.399][S02]like the criminal justice system or in healthcare.[1135.836] [1135.836][S02]Here’s Verity Harding again.[1137.371] [1137.371][S06] This is what I mean by it being a kind of much bigger discussion[1141.341] [1141.341][S06]that that potentially the use of algorithms is highlighting.[1144.711] [1144.711][S06]My fear is that people kind of get a checkmark that says:[1148.081] [1148.081][S06]we tested and this algorithm isn’t biased[1151.418] [1151.418][S06]and therefore you should feel free to use it.[1153.854] [1153.854][S06]And that to me isn’t going far enough.[1156.623] [1156.623][S06]I think there needs to be a further discussion then about um[1160.294] [1160.294][S06]but is this making those decisions that were already bad worse, or more quickly,[1165.265] [1165.265][S06]and therefore more of them and you know that that kind of thing.[1167.768] [1167.768][S02] But things are changing.[1169.77] [1169.77][S02]Here’s William on what has happened since that Pro Publica story broke.[1173.54] [1173.54][S07] They’re going back and reconsidering[1175.342] [1175.342][S07]what measures they collect rather than going back[1177.744] [1177.744][S07]and trying to create more robust data sets,[1179.947] [1179.947][S07]thinking about who is collecting the actual data itself.[1183.317] [1183.317][S07]Will it be ever perfect?[1184.484] [1184.484][S07]Will we have bias-free, purely pure data,[1187.254] [1187.254][S07]no I don’t think that’s--I don’t think that’s ever going to happen.[1189.957] [1189.957][S07]But I do think that people will be skeptical[1193.894] [1193.894][S07]when people ask about what data sets you use[1196.396] [1196.396][S07]and they don’t get a satisfactory answer,[1198.432] [1198.432][S07]but I do think people will ask - is this data set representative?[1201.835] [1201.835][S07]Does it have balance across different groups?[1203.737] [1203.737][S07]So people will start asking questions and interrogating data[1206.406] [1206.406][S07]sets and models more aggressively[1208.976] [1208.976][S07]and I think that will lead to better outcomes.[1211.645] [1211.645][S02] And crucially, more people are now being included[1214.815] [1214.815][S02]as part of the conversation.[1216.583] [1216.583][S07] In the aftermath of some of my work[1218.986] [1218.986][S07]and on many others on predictive policing,[1221.622] [1221.622][S07]many cities in California actually started implementing citizen boards.[1225.859] [1225.859][S07]So when police departments wanted to acquire a new police technology,[1230.531] [1230.531][S07]that included uses of machine learning or artificial intelligence,[1234.268] [1234.268][S07]that they had to go in front of a citizen board[1237.971] [1237.971][S07]and actually have the local community evaluate[1241.508] [1241.508][S07]the tool for different metrics including fairness and bias.[1245.345] [1248.682][S02] Getting different voices involved in the conversation[1250.851] [1250.851][S02]is essential to making sure that we build a future[1253.554] [1253.554][S02]that belongs to all of us[1255.556] [1255.556][S02]because what seems obvious to just one person[1257.758] [1257.758][S02]just wouldn’t occur to another.[1260.093] [1260.093][S02]Your perspective is hard coded into the work that you create.[1265.365] [1265.365][S02]There are clear examples of this everywhere outside of AI -[1269.102] [1269.102][S02]able-bodied people designing buildings[1271.305] [1271.305][S02]that disabled people can’t use or new types of plasters[1274.975] [1274.975][S02]that only work if your skin is one particular colour -[1279.079] [1279.079][S02]presumably the same as the designer’s.[1281.715] [1281.715][S02]And the algorithms that we’ve created, they’re really highlighting this issue.[1286.086] [1286.086][S02]Like the ones used to automatically screen[1288.455] [1288.455][S02]CVs and predict which candidates will fit best in a company.[1294.027] [1294.027][S02]Here’s Verity Harding again.[1295.562] [1295.562][S06] If it’s based on historically discriminatory hiring decisions[1300.267] [1300.267][S06]by either intentionally or unintentionally by us humans,[1304.137] [1304.137][S06]um then it’s going to kind of recreate those patterns.[1307.374] [1307.374][S02] Like if you’ve got a company where white men have succeeded[1310.31] [1310.344][S06] Yes[1311.245] [1311.245][S02] and you’re looking for candidates who will succeed,[1313.68] [1313.68][S02]it’s going to pick out white male CVs.[1316.35] [1316.35][S06] Yes, exactly.[1317.451] [1317.451][S06]And if the people building the technology are white males as well,[1322.322] [1322.322][S06]then the likelihood of paying attention to that potential bias[1326.46] [1326.46][S06]and being aware of it, I mean we all have our blind spots,[1329.329] [1330.097][S06]then then the likelihood increases.[1332.633] [1333.667][S02] We’ve seen driverless cars[1335.202] [1335.202][S02]that don’t spot pedestrians with darker skin tones.[1338.305] [1338.305][S02]Tumour screening algorithms that aren’t as effective for patients[1341.341] [1341.341][S02]with ethnicities other than white European,[1344.912] [1344.912][S02]and lots and lots and lots of issues around gender.[1350.551] [1350.551][S02]All of this is kind of inevitable[1352.886] [1352.886][S02]unless you have a range of different viewpoints in your design process.[1357.124] [1357.124][S06] The most important thing in my point of view for ensuring[1361.762] [1361.762][S06]that these things are um if not biased,[1365.866] [1365.866][S06]but that you are being intentional about what you’re building,[1368.602] [1368.602][S06]and aware of the potential bias is that your team is a diverse team[1373.373] [1373.373][S06]is that you have a broad set of voices involved,[1377.244] [1377.244][S06]and it’s actually much simpler to do that than it’s suggested.[1382.416] [1382.416][S02] And the issue of gender diversity[1384.251] [1384.251][S02]has been a particular focus of late.[1386.587] [1386.587][S06] I think there’s plenty of young women and girls[1389.756] [1389.756][S06]who are really excited by science and stem subjects[1393.594] [1393.594][S06]and it’s an easy get out to say that there aren’t enough women in STEM[1397.631] [1397.631][S06]and that’s why work forces aren’t diverse,[1399.967] [1399.967][S06]but actually it’s much more about making sure[1403.437] [1403.437][S06]that it’s a safe space for women and girls to work,[1406.306] [1406.306][S06]that they’re not discriminated against once they’re there.[1409.51] [1409.51][S06]That you’re able to not just attack and hire them[1412.98] [1412.98][S06]but that you’re able to keep them and[1414.882] [1414.882][S06]and make sure that it’s a place where they feel comfortable working.[1417.684] [1417.684][S06]And so I think it’s much more important that we look[1420.153] [1420.153][S06]at how women are treated in science than just dismiss it[1422.723] [1422.723][S06]as something that girls aren’t interested in at a young age.[1426.059] [1426.059][S02] Lila Ibrahim, DeepMind CCO is very conscious that diversity[1430.097] [1430.097][S02]is still a problem in the tech sector as a whole.[1433.3] [1433.3][S03] Talk about things that keep me up at night.[1435.502] [1435.502][S03]Right, so, here I am, a professional of ah 25+ years[1441.608] [1441.608][S03]with an engineering background.[1443.744] [1443.744][S03]A mom also raising 9 year old twin daughters.[1448.215] [1448.215][S03]I would have hoped by now we would have solved the problem[1450.384] [1450.384][S03]and yet we haven’t[1451.451] [1451.451][S03]- we’re like at the same, the numbers are flat.[1454.188] [1454.188][S02] But there are steps being taken to address it.[1457.191] [1457.191][S03] There’s the short-term stuff you can do[1458.559] [1458.559][S03]which are things like you diversify your candidate pools,[1462.596] [1462.596][S03]you, if you’re doing university recruiting,[1465.098] [1465.098][S03]you look at a broader range of universities[1468.335] [1468.335][S03]and ones that have um that have a broader student representation[1472.272] [1472.272][S03]and have support structures often in place[1474.608] [1474.608][S03]to help the students through their academic and communities.[1478.245] [1478.245][S03]Um you look at job descriptions[1480.314] [1480.314][S03]and ensure that you don’t have unconscious bias[1482.416] [1482.416][S03]reflected in your job descriptions.[1484.551] [1484.551][S03]You - so once you’re in the recruiting pipeline,[1486.887] [1486.887][S03]then you need to make sure candidates have the right experience.[1490.324] [1490.324][S03]We are being very deliberate about how we invest back in education.[1495.562] [1495.562][S03]AI is something that will change future generations -[1499.9] [1499.9][S03]so how do we make this a field that is more accessible -[1504.004] [1504.004][S03]so for example, um whether it’s funding diversity scholars at universities,[1509.476] [1509.476][S03]or funding AI chairs in universities to try and increase the pipeline,[1514.481] [1514.481][S03]and I think that helps fuel some of the academic aspects[1518.285] [1518.285][S03]as well as support are like long-term recruiting.[1522.122] [1522.122][S02] This isn’t just tokenism that we’re talking about here,[1525.125] [1525.125][S02]this is about making better technology.[1528.195] [1528.195][S03] Diversity and diverse perspectives will create a drive faster and safer[1536.537] [1536.537][S03]and with just a better a better result because one of the things I worry about[1542.009] [1542.009][S03]is how do we avoid our own internal bias?[1544.811] [1544.811][S03]A lot of the work around deep reinforcement[1546.547] [1546.547][S03]learning started from specific pockets and many people grew up in those labs[1550.717] [1550.717][S03]or those universities and you know they bought their former colleagues[1555.088] [1555.088][S03]and so we have our pretty strong network of people[1557.925] [1557.925][S03]who have known each other for a long time which is fantastic[1561.395] [1561.395][S03]and they can really advance certain aspects of our of our work[1564.831] [1564.831][S03]and yet there are other areas that are emerging um how do you teach curiosity,[1570.337] [1570.337][S03]how do you um how do we ensure that we minimise bias[1575.175] [1575.175][S03]and the code that we’re writing? Who’s to say ah what intelligence is[1580.781] [1580.781][S03]and isn’t unless you have a better representation from society?[1585.819] [1585.819][S03]And that’s just on the research side.[1587.487] [1587.487][S03]On the operations side too you think about things like okay,[1592.192] [1592.192][S03]think about public policy, like if you’re asking governments[1595.963] [1595.963][S03]to think about how they’re going to treat artificial intelligence[1600.534] [1600.534][S03]then you want people that are representative[1602.369] [1602.369][S03]of the of the constituents of the population.[1605.038] [1605.038][S03]If we want to ah be focused on education,[1608.375] [1608.375][S03]making sure that we’re not just focused on the specific schools[1611.078] [1611.078][S03]but our broader range so we’re bringing more people into the space.[1614.314] [1614.314][S03]I just think it’s going to be imperative for us to truly solve intelligence[1620.153] [1620.153][S03]that we’re just going to need to have more diversity.[1623.457] [1623.457][S02] So is part of the well solutions,[1625.826] [1625.826][S02]probably a bit grand a word to suggest[1628.462] [1628.462][S02]but it’s part of the way forward making ethics a kind of keystone[1634.201] [1634.201][S02]at every stage of the process rather than having it as an afterthought.[1637.638] [1637.638][S03] Oh, absolutely. It we have to be thinking about our responsibility[1641.074] [1641.074][S03]for the technology we develop[1642.609] [1642.609][S03]and to candidly to society as a whole - every step along the way.[1649.349] [1649.349][S03]And I think that there’s something quite special[1651.885] [1651.885][S03]about being headquarters out of London versus being[1655.355] [1655.355][S03]based out of the Silicon Valley - I love Silicon Valley,[1659.393] [1659.393][S03]it’s where my career has really developed[1663.297] [1663.297][S03]and yet you’re surrounded by technologists,[1665.966] [1665.966][S03]you know from the billboard signs to the marketing and promotion.[1670.804] [1670.804][S03]Here in London it’s so multicultural[1673.941] [1673.941][S03]and I feel like it’s part of your daily life.[1677.01] [1677.01][S03]You need to be thinking about the work that you’re doing[1679.68] [1679.68][S03]and how it’s going to impact all the people around you.[1682.816] [1684.785][S02] But of course we can’t just leave the solutions[1687.254] [1687.254][S02]solely in the hands of the people who are designing these things.[1691.992] [1691.992][S02]It’s our future, too.[1693.894] [1693.894][S02]The public and government should also have a hand in this.[1697.865] [1697.865][S06] My impression is that people want to understand what they’re using[1701.034] [1701.034][S06]and want to understand what makes it work and how it works.[1705.072] [1705.072][S06]But they more importantly want their representatives[1708.208] [1708.208][S06]and the people tasked with keeping them safe[1710.41] [1710.41][S06]and secure to understand it too[1712.279] [1712.279][S06]and I think that’s where we’ve seen a bit of a breakdown in recent times.[1715.983] [1717.518][S02] If you want to know more about ethics, diversity and fairness,[1721.722] [1721.722][S02]then head over to the show notes where you can also explore[1724.458] [1724.458][S02]the world of AI research beyond DeepMind.[1727.528] [1727.528][S02]And we’d welcome your feedback and questions[1730.063] [1730.063][S02]on any aspects of artificial intelligence[1732.032] [1732.032][S02]that we’re covering in this series.[1734.034] [1734.034][S02]So if you want to join in the discussion or point us to stories or resources[1738.038] [1738.038][S02]that you think other listeners would find helpful, then please let us know.[1741.375] [1741.375][S02]You can message us on Twitter, or you can email us -[1744.278] [1744.278][S02]podcast@deepmind.com[1746.38]