Add files using upload-large-folder tool

15. Transfer Learning Build a Flower & Monkey Breed Classifier/3. Build a Monkey Breed Classifier with MobileNet using Transfer Learning.srt

@@ -0,0 +1,769 @@
+1
+00:00:00,490 --> 00:00:06,120
+Hi and welcome to Chapter 15 point to where we're going to build a monkey bridge justifier and basically
+
+2
+00:00:06,120 --> 00:00:11,780
+use the concept of transfer learning to get very high accuracy very quickly.
+
+3
+00:00:11,780 --> 00:00:13,550
+So let's take a look at this dataset.
+
+4
+00:00:13,560 --> 00:00:19,380
+This is where it was taken a stick and from a Kaggle project and basically it has about 80 images I
+
+5
+00:00:19,380 --> 00:00:22,240
+think of about 10 different types of monkeys each.
+
+6
+00:00:22,270 --> 00:00:24,290
+Is it a species of monkeys here.
+
+7
+00:00:25,140 --> 00:00:30,120
+And actually not 80 18:00 into the 152 images of each class.
+
+8
+00:00:30,120 --> 00:00:35,420
+And these are some sample images here and you'll notice that some are quite small and differently.
+
+9
+00:00:35,670 --> 00:00:40,740
+Different aspect ratios images of various sizes and quality as well.
+
+10
+00:00:40,770 --> 00:00:45,900
+So it's pretty much like what you might build as your own data sets effectively.
+
+11
+00:00:46,050 --> 00:00:53,310
+It's not well standardized not super neat not super high quality images just random images taken from
+
+12
+00:00:53,310 --> 00:00:54,080
+the Internet.
+
+13
+00:00:54,360 --> 00:01:01,280
+So now let's move onto Pitre notebook and begin creating this toxify OK.
+
+14
+00:01:01,290 --> 00:01:07,830
+So before we begin I hope you downloaded your resource file monkey Breede datasets and have placed it
+
+15
+00:01:07,950 --> 00:01:09,560
+inside of the territory here.
+
+16
+00:01:09,810 --> 00:01:15,660
+This is the translating directory so you have monkey abused monkey read our victory here with our training
+
+17
+00:01:15,690 --> 00:01:16,300
+images.
+
+18
+00:01:16,320 --> 00:01:17,960
+And each one is in full to here.
+
+19
+00:01:18,300 --> 00:01:22,980
+And let's go back and now hopefully that's set up correctly for you.
+
+20
+00:01:23,250 --> 00:01:28,070
+So now we can go back so we went back to full and open up here.
+
+21
+00:01:28,200 --> 00:01:33,630
+I already have it open right now so I'm going to go through this step by step so you understand exactly
+
+22
+00:01:33,630 --> 00:01:36,210
+how we can apply transfer linning.
+
+23
+00:01:36,210 --> 00:01:37,140
+All right.
+
+24
+00:01:37,140 --> 00:01:39,180
+So we're doing this with more on that.
+
+25
+00:01:39,210 --> 00:01:44,090
+And the reason I have to move on that for is because it actually trains quite quickly on C-p use.
+
+26
+00:01:44,430 --> 00:01:46,430
+So let's import ballots here.
+
+27
+00:01:47,010 --> 00:01:53,880
+And then let's define image rows and columns so we're going to use uniform square images of 2:24 by
+
+28
+00:01:53,880 --> 00:01:55,650
+
+29
+2:24 in size.
+
+30
+00:01:55,740 --> 00:01:58,620
+And this is how we basically define that.
+
+31
+00:01:58,650 --> 00:02:01,280
+When we loaded in we wanted to we had to be Image nets.
+
+32
+00:02:01,290 --> 00:02:03,970
+We've seen this before in our pretreated Model S..
+
+33
+00:02:04,020 --> 00:02:06,240
+However we haven't seen these parameters here.
+
+34
+00:02:06,240 --> 00:02:11,610
+I will quickly discuss this with you what we're going to do is that we're going to include the top and
+
+35
+00:02:11,610 --> 00:02:13,120
+said this at Falls.
+
+36
+00:02:13,380 --> 00:02:18,150
+What this means is that the fully connectedly as the last layers on the top of the model are basically
+
+37
+00:02:18,150 --> 00:02:19,820
+not included in the model.
+
+38
+00:02:20,130 --> 00:02:24,650
+So I'm going to show you what it looks pretty soon and in what shape is of center thing.
+
+39
+00:02:24,690 --> 00:02:30,420
+We just defined in what shape of this model to be this as why we define these parameters up here and
+
+40
+00:02:30,420 --> 00:02:33,050
+tree means color depth of tree RGV.
+
+41
+00:02:33,450 --> 00:02:36,990
+So this is a cool thing we can do with terrorist models upload.
+
+42
+00:02:37,320 --> 00:02:39,790
+So we have a model here called Mobile that.
+
+43
+00:02:39,890 --> 00:02:44,310
+So by addressing the layers within that Dudley as districting an array.
+
+44
+00:02:44,490 --> 00:02:50,450
+And we can basically loop through these areas here and actually turn it off manually.
+
+45
+00:02:50,690 --> 00:02:57,360
+The treatable parameter a flag that controls what it is should be trainable or not.
+
+46
+00:02:57,360 --> 00:03:02,980
+So what we do in this two lines of code here is that we basically setting all the layers in Mobile and
+
+47
+00:03:02,990 --> 00:03:06,450
+that's to be non-tradable basically fixed.
+
+48
+00:03:06,450 --> 00:03:09,160
+This is how we freeze DeWitt's right here.
+
+49
+00:03:09,690 --> 00:03:11,830
+So now we could actually print these layers here.
+
+50
+00:03:12,120 --> 00:03:16,250
+And basically what we are printing is the name of Leo number.
+
+51
+00:03:16,320 --> 00:03:22,740
+I go to the loop and we're going to print the flag Liautaud trainable what it's treatable.
+
+52
+00:03:22,770 --> 00:03:23,940
+True or false.
+
+53
+00:03:23,970 --> 00:03:29,970
+So you get to see all the layers now which is quite a bit a mobile that are set to false.
+
+54
+00:03:29,970 --> 00:03:31,840
+So this is pretty awesome already.
+
+55
+00:03:32,100 --> 00:03:35,290
+So I hope you're following simple code so far.
+
+56
+00:03:36,000 --> 00:03:42,090
+So now we're going to do is we're going to create a simple function here that basically adds to fully
+
+57
+00:03:42,090 --> 00:03:47,730
+connected head back onto the model we loaded here because remember we loaded it.
+
+58
+00:03:47,880 --> 00:03:49,180
+But we didn't get to the top.
+
+59
+00:03:49,200 --> 00:03:52,950
+So now we have a model without any top.
+
+60
+00:03:52,950 --> 00:03:55,070
+So no actually I want to show you something quickly.
+
+61
+00:03:55,290 --> 00:03:57,590
+What if we said this to true.
+
+62
+00:03:57,620 --> 00:03:58,090
+All right.
+
+63
+00:03:58,110 --> 00:03:59,220
+How would this model look.
+
+64
+00:03:59,250 --> 00:04:04,330
+So we saw we had 86 differently as the last one being removed.
+
+65
+00:04:04,650 --> 00:04:09,000
+So let's now print this and see what it looks
+
+66
+00:04:14,460 --> 00:04:16,170
+takes about five to 10 seconds to run.
+
+67
+00:04:16,170 --> 00:04:17,500
+There we go.
+
+68
+00:04:18,330 --> 00:04:18,990
+Oh good.
+
+69
+00:04:18,990 --> 00:04:24,770
+So before we head up to 86 now we see we have basically this is the top fully connected head.
+
+70
+00:04:25,200 --> 00:04:28,240
+This is what we left out before previously.
+
+71
+00:04:28,320 --> 00:04:29,510
+So now let's put it back in.
+
+72
+00:04:29,670 --> 00:04:30,690
+OK.
+
+73
+00:04:32,350 --> 00:04:35,090
+Because what we're going to do we're going to add a head here.
+
+74
+00:04:35,340 --> 00:04:38,660
+These are really as we we are going to add onto the model now.
+
+75
+00:04:38,710 --> 00:04:40,660
+So how do we use this function.
+
+76
+00:04:40,660 --> 00:04:43,720
+This function takes a number of classes.
+
+77
+00:04:43,790 --> 00:04:46,120
+I do our data sets.
+
+78
+00:04:46,420 --> 00:04:48,220
+We specify how many classes we want.
+
+79
+00:04:48,220 --> 00:04:54,370
+So for a monkey breed they is that it's going to be 10 and the bottom bottom model is basically this
+
+80
+00:04:54,420 --> 00:04:55,040
+model here.
+
+81
+00:04:55,080 --> 00:04:57,250
+Well not at all it's for us and it's.
+
+82
+00:04:57,580 --> 00:05:00,040
+So let's quickly see what this function does.
+
+83
+00:05:00,100 --> 00:05:02,200
+It takes a lot of muddled model here.
+
+84
+00:05:02,420 --> 00:05:07,310
+Guess's gets the output part of it here and we create basically the top model now.
+
+85
+00:05:07,660 --> 00:05:13,990
+So what we do now we have to find a top model like this here and now the top model we just simply basically
+
+86
+00:05:14,080 --> 00:05:15,450
+add these layers here.
+
+87
+00:05:15,670 --> 00:05:18,240
+It's a different way of Ardingly as no one cares.
+
+88
+00:05:18,580 --> 00:05:21,010
+So we added an adjusted to the top model here.
+
+89
+00:05:21,280 --> 00:05:28,600
+So for us we do global pooling Tuti we do a densely with a thousand and 28 nodes that again another
+
+90
+00:05:28,600 --> 00:05:33,590
+densely a here and then we do a final densely with soft Macksville attend classes we want.
+
+91
+00:05:33,790 --> 00:05:38,490
+And then what this does retune the model Top Model back.
+
+92
+00:05:38,600 --> 00:05:45,640
+OK so now what we do below is obviously we just load all Olias of need and defined number of classes
+
+93
+00:05:45,670 --> 00:05:51,640
+but now we can actually use our function here where we actually enter a number of classes.
+
+94
+00:05:51,730 --> 00:05:57,480
+We enter the mobile in that model that we created we loaded before and we add a top.
+
+95
+00:05:57,580 --> 00:06:02,000
+That's actually a we defined here to this model and that's why we call it the C.
+
+96
+00:06:02,360 --> 00:06:08,840
+And what we do know is that we use this cross model function so we use it now to get inputs here which
+
+97
+00:06:08,840 --> 00:06:13,680
+are defined as a mobile at model output speed the other possible are we going to train.
+
+98
+00:06:13,840 --> 00:06:18,970
+And basically this combines it into one model now one model where it looks like this when print printed
+
+99
+00:06:18,970 --> 00:06:19,920
+out.
+
+100
+00:06:20,520 --> 00:06:26,300
+So a lot of layers I just saw before 86 Lia's But now we have four sort six Malis is now these are three
+
+101
+00:06:26,320 --> 00:06:27,360
+to find here.
+
+102
+00:06:27,790 --> 00:06:29,250
+And that's going to show up right here.
+
+103
+00:06:29,320 --> 00:06:30,590
+So this is pretty cool.
+
+104
+00:06:30,820 --> 00:06:31,950
+And look at this here.
+
+105
+00:06:31,960 --> 00:06:37,710
+So we have five million parameters five point equally actually and trainable parameters.
+
+106
+00:06:37,750 --> 00:06:38,870
+Only 2.6.
+
+107
+00:06:38,890 --> 00:06:43,430
+And the non-tradable parameters which are the width of we froze our trillion.
+
+108
+00:06:43,720 --> 00:06:48,850
+So effectively we've taken a model of at MIT How was pretty complex not super complex like a Viji and
+
+109
+00:06:48,880 --> 00:06:54,850
+couple of others but complex enough and we've made it into a much simpler model to train.
+
+110
+00:06:55,030 --> 00:06:57,390
+So let's get to training of monkey breed.
+
+111
+00:06:57,400 --> 00:07:00,600
+They just had no training on Lockerby to classify.
+
+112
+00:07:00,910 --> 00:07:05,720
+So we loaded data sets using imaged digit data generators that you've seen before.
+
+113
+00:07:06,460 --> 00:07:12,410
+We do a standard thing here which you of which you should be pretty familiar with by now and then we
+
+114
+00:07:12,400 --> 00:07:14,530
+define some checkpoints and Colback sorry.
+
+115
+00:07:14,650 --> 00:07:20,470
+So we use stopping and checkpointing here and then we train for only five bucks for now.
+
+116
+00:07:20,680 --> 00:07:23,440
+That's because we don't want it to be like take too long.
+
+117
+00:07:23,830 --> 00:07:26,380
+And actually treating it separately in this window here.
+
+118
+00:07:26,830 --> 00:07:30,980
+So I've actually already trained almost five e-books and realize so much time.
+
+119
+00:07:31,390 --> 00:07:38,590
+So look at this here you can see after this epoch which took just under five minutes our validation
+
+120
+00:07:38,590 --> 00:07:41,180
+accuracy was 88 percent already.
+
+121
+00:07:41,470 --> 00:07:44,750
+That is actually pretty damn good for such a short space of time.
+
+122
+00:07:45,070 --> 00:07:49,580
+Now and a second night duration because it's such a early start to the trading hour.
+
+123
+00:07:49,630 --> 00:07:55,690
+Even though the trading loss is much lower the and accuracy is a little bit less 84 percent.
+
+124
+00:07:55,780 --> 00:07:56,320
+That's OK.
+
+125
+00:07:56,350 --> 00:07:58,060
+We can sort of live it out.
+
+126
+00:07:58,120 --> 00:08:03,280
+We'll let a train from what ebox and see how it evolves because training these pre-treat models when
+
+127
+00:08:03,280 --> 00:08:09,120
+it's something which is frozen is a little bit different than how we turn to CNN's they basically do
+
+128
+00:08:09,180 --> 00:08:12,890
+they do effectively converge and get a very high value.
+
+129
+00:08:12,940 --> 00:08:15,920
+However you do sometimes see some odd fluctuations like this.
+
+130
+00:08:16,270 --> 00:08:20,320
+And look we have it back up to 91 percent 90 percent.
+
+131
+00:08:20,320 --> 00:08:26,110
+If you wait a few minutes sorry about 20 seconds at least here we can actually see what evaluation accuracy
+
+132
+00:08:26,110 --> 00:08:28,540
+is at the end of the fifth book.
+
+133
+00:08:28,540 --> 00:08:39,640
+So let's wait and see what it looks.
+
+134
+00:08:39,650 --> 00:08:46,360
+One thing to note is that you can actually see our callback stopping callback actually telling us how
+
+135
+00:08:46,370 --> 00:08:52,340
+validation loss did not improve did not improve if we left this for 20 bucks and we had actually it
+
+136
+00:08:52,340 --> 00:08:57,060
+was here as well so basically no matter what this is going to be the last epoch because I'm pretty sure
+
+137
+00:08:57,060 --> 00:09:00,070
+I said my patience to Tree Hill.
+
+138
+00:09:00,320 --> 00:09:00,770
+Yep.
+
+139
+00:09:00,770 --> 00:09:02,360
+I usually always do.
+
+140
+00:09:02,840 --> 00:09:07,250
+So right now what it's doing had a reason why I stuck it two seconds even did two seconds would have
+
+141
+00:09:07,250 --> 00:09:13,220
+passed by the time I started the sentence is that it's predicting on treating all validation data set.
+
+142
+00:09:13,220 --> 00:09:16,540
+Now that's something that a lot of beginners don't know.
+
+143
+00:09:16,720 --> 00:09:19,620
+Take the seat pause at the end of it like a note stuck.
+
+144
+00:09:19,760 --> 00:09:20,770
+It isn't actually stuck.
+
+145
+00:09:20,780 --> 00:09:24,270
+It's just waiting to run on the validation data set now.
+
+146
+00:09:24,380 --> 00:09:29,480
+So it takes a little while to honestly because sometimes validation data sets are quite big.
+
+147
+00:09:29,800 --> 00:09:31,110
+Ah there we go.
+
+148
+00:09:31,140 --> 00:09:32,240
+So look at this.
+
+149
+00:09:32,410 --> 00:09:37,120
+We got 93 percent accuracy in such a short space of time.
+
+150
+00:09:37,190 --> 00:09:38,300
+So this is quite good.
+
+151
+00:09:38,360 --> 00:09:41,050
+So no it's actually go back to this main page here.
+
+152
+00:09:41,450 --> 00:09:44,290
+Let's look at our model which takes me about 10 seconds.
+
+153
+00:09:47,460 --> 00:09:52,570
+And what are we going to do once this model is loaded We're going to basically use open C-v because
+
+154
+00:09:52,810 --> 00:09:53,310
+messy.
+
+155
+00:09:53,340 --> 00:09:59,970
+But of course I wrote quickly that loads the images here and it runs into predictive that we just loaded
+
+156
+00:10:00,180 --> 00:10:07,980
+here and so on already and we're actually going to see the monkey class see how accurate a real ossify
+
+157
+00:10:07,980 --> 00:10:10,410
+really is 90 percent accurate.
+
+158
+00:10:10,410 --> 00:10:12,280
+So let's find out.
+
+159
+00:10:12,480 --> 00:10:13,390
+There we go.
+
+160
+00:10:13,800 --> 00:10:14,560
+So this is the truth.
+
+161
+00:10:14,560 --> 00:10:16,720
+US battled.
+
+162
+00:10:17,080 --> 00:10:18,710
+Yes that's like a Japanese monkey.
+
+163
+00:10:20,080 --> 00:10:22,850
+OK so fiercely it got this one wrong.
+
+164
+00:10:23,120 --> 00:10:28,310
+This is what Elmo predicted Whitehead had a cabbage in and no it was not a white hat.
+
+165
+00:10:28,810 --> 00:10:30,520
+Let's see if he gets it right.
+
+166
+00:10:30,520 --> 00:10:31,300
+Yeah it did.
+
+167
+00:10:31,330 --> 00:10:32,630
+Got this one right.
+
+168
+00:10:32,710 --> 00:10:33,110
+Pick me.
+
+169
+00:10:33,110 --> 00:10:34,020
+I'm almost at.
+
+170
+00:10:34,270 --> 00:10:37,010
+Let's see what the other is.
+
+171
+00:10:37,070 --> 00:10:39,000
+Gary langar definitely.
+
+172
+00:10:39,020 --> 00:10:40,020
+Right.
+
+173
+00:10:40,280 --> 00:10:41,590
+Pygmy marmosets again.
+
+174
+00:10:41,660 --> 00:10:42,710
+Got it right.
+
+175
+00:10:42,740 --> 00:10:44,090
+Got it right.
+
+176
+00:10:44,090 --> 00:10:44,990
+Got that right.
+
+177
+00:10:44,990 --> 00:10:46,210
+Got it right.
+
+178
+00:10:46,550 --> 00:10:48,010
+Got it right again.
+
+179
+00:10:48,560 --> 00:10:49,930
+Got it right.
+
+180
+00:10:50,000 --> 00:10:51,350
+So seems pretty good.
+
+181
+00:10:51,530 --> 00:10:55,230
+So aside from the first one model got basically nine out of 10 right.
+
+182
+00:10:55,250 --> 00:10:58,550
+Which kind of corresponds to 90 percent accuracy.
+
+183
+00:10:58,550 --> 00:10:59,560
+We got here.
+
+184
+00:10:59,930 --> 00:11:07,100
+So you've just learnt to create a model a basically a train model using transfer learning and you see
+
+185
+00:11:07,100 --> 00:11:07,850
+how simple it is.
+
+186
+00:11:07,850 --> 00:11:15,770
+You just basically linnets load it with the weight speed frozen and the top being not included.
+
+187
+00:11:15,770 --> 00:11:18,800
+Then you build the function to add the top whatever top you want to add.
+
+188
+00:11:18,860 --> 00:11:24,770
+I didn't hear all these make sure the Lasley is number of classes you have in your dataset.
+
+189
+00:11:24,860 --> 00:11:28,540
+Then you basically compare concatenates and compile the bottles here.
+
+190
+00:11:29,690 --> 00:11:32,890
+Well combinable I should say you do it your image under it.
+
+191
+00:11:32,900 --> 00:11:38,980
+It does define your check points and callbacks compile and we go and train.
+
+192
+00:11:39,400 --> 00:11:42,880
+So it's really very simple and I hope you find a disruptive quite useful.
+
+193
+00:11:43,060 --> 00:11:43,340
+Thank you.

15. Transfer Learning Build a Flower & Monkey Breed Classifier/3.1 Download the Monkey Breed Dataset.html

@@ -0,0 +1 @@
+<script type="text/javascript">window.location = "https://drive.google.com/file/d/1l-7wsAaDi89TpaFPjFW-oS8pdgSn_ekw/view?usp=sharing";</script>

15. Transfer Learning Build a Flower & Monkey Breed Classifier/4. Build a Flower Classifier with VGG16 using Transfer Learning.srt

@@ -0,0 +1,475 @@
+1
+00:00:00,750 --> 00:00:06,480
+Hi and welcome to Chapter 15 point tree where we're about to build a flow of classify and we're going
+
+2
+00:00:06,480 --> 00:00:08,580
+to use transfer learning to do this.
+
+3
+00:00:08,580 --> 00:00:10,280
+So let's take a look at how we actually.
+
+4
+00:00:10,370 --> 00:00:13,230
+What is Flora ossify our flower data set.
+
+5
+00:00:13,230 --> 00:00:14,920
+I should say so.
+
+6
+00:00:15,060 --> 00:00:20,150
+It comes from the Oxford University's visual geometry group as called 17.
+
+7
+00:00:20,310 --> 00:00:26,270
+And that's because there are 17 categories of flowers and their images in each class say the sets and
+
+8
+00:00:26,270 --> 00:00:27,200
+not that much.
+
+9
+00:00:28,110 --> 00:00:33,900
+So this is some sample images from the flow Josephite the flowers 17 there is that this is the web page
+
+10
+00:00:33,900 --> 00:00:34,940
+from Oxford University.
+
+11
+00:00:34,950 --> 00:00:40,170
+And this is the link you can go to if you want to download it from day itself or you can use that link
+
+12
+00:00:40,230 --> 00:00:44,580
+I have on the left here on the demi site panel.
+
+13
+00:00:44,640 --> 00:00:49,440
+Please use that link to actually download it because I've already preprocess the data into a format
+
+14
+00:00:49,470 --> 00:00:54,330
+that is easily imported into Karris if you downloaded from Oxford University site you're going to have
+
+15
+00:00:54,330 --> 00:00:55,760
+to do a pre-processing itself.
+
+16
+00:00:55,770 --> 00:01:00,630
+And I don't think if you're a beginner you're not going to find that fun at all although it's a good
+
+17
+00:01:00,630 --> 00:01:02,740
+exercise to do sometimes.
+
+18
+00:01:03,600 --> 00:01:08,790
+So anyway our approach to this problem is that we're going to actually use a pre-trained Fiji A16 model
+
+19
+00:01:09,540 --> 00:01:14,490
+with all of its way it's frozen except the top layer and we're only going to train the top ahead of
+
+20
+00:01:14,490 --> 00:01:17,490
+the model with a final output of 17 classes.
+
+21
+00:01:17,490 --> 00:01:21,370
+So let's go back to our I and notebook and get this done.
+
+22
+00:01:21,710 --> 00:01:22,100
+OK.
+
+23
+00:01:22,140 --> 00:01:24,750
+So welcome back to our virtual machine.
+
+24
+00:01:24,780 --> 00:01:28,820
+I hope you downloaded the flowers dataset and extracted it to this folder here.
+
+25
+00:01:29,040 --> 00:01:34,170
+That's this folder called transfer linning and financing and Plaisted right here so we can quickly just
+
+26
+00:01:34,170 --> 00:01:37,910
+inspect it taking a look at some of those pictures.
+
+27
+00:01:38,330 --> 00:01:42,120
+Let's put it on toenail view and it looks quite nice.
+
+28
+00:01:42,120 --> 00:01:46,280
+So as you can see we don't have that many images in this data set.
+
+29
+00:01:46,380 --> 00:01:51,380
+So let's see what kind of accuracy we can get without transfer learning on the Viji model.
+
+30
+00:01:51,390 --> 00:01:53,380
+So let's go to it here.
+
+31
+00:01:53,790 --> 00:02:02,170
+So no let me just close some of these windows open and let's quickly go back to this one here so you
+
+32
+00:02:02,170 --> 00:02:03,350
+can actually see how I do it.
+
+33
+00:02:03,360 --> 00:02:05,080
+It's 15.
+
+34
+00:02:05,080 --> 00:02:07,090
+And we go to making a flower classifier.
+
+35
+00:02:07,210 --> 00:02:08,440
+That's this file here.
+
+36
+00:02:08,830 --> 00:02:10,260
+So now that we're in the file.
+
+37
+00:02:10,300 --> 00:02:11,800
+Let's take a look at what's going on.
+
+38
+00:02:11,800 --> 00:02:15,770
+So we import the BTG model that's easily done here.
+
+39
+00:02:16,120 --> 00:02:23,470
+Viji was designed to work open 24 or 224 by 224 pixel image input's Isiah's.
+
+40
+00:02:23,500 --> 00:02:26,450
+So let's keep the standard size and go forward.
+
+41
+00:02:26,530 --> 00:02:32,200
+So let's load the model with out his weights or with the weights of image's nuts without the top layer.
+
+42
+00:02:32,410 --> 00:02:34,360
+I should say so we do that.
+
+43
+00:02:34,420 --> 00:02:36,960
+And let's just print out the layers in this model.
+
+44
+00:02:37,060 --> 00:02:37,560
+OK.
+
+45
+00:02:37,930 --> 00:02:44,740
+So as you can see default actually is loaded here and by default all the layers are trainable.
+
+46
+00:02:44,740 --> 00:02:52,370
+True that means the default in of when you load EGD all the weights are trainable.
+
+47
+00:02:52,630 --> 00:02:55,090
+So we now have to set this true to false.
+
+48
+00:02:55,090 --> 00:02:56,490
+So that's what we do here.
+
+49
+00:02:56,860 --> 00:03:03,010
+So we loaded with our top head with Image net weights and we set all the treatable as we said this flag
+
+50
+00:03:03,090 --> 00:03:04,210
+to false.
+
+51
+00:03:04,270 --> 00:03:08,030
+So let's do this quickly and that's done there.
+
+52
+00:03:08,520 --> 00:03:13,450
+And now let's create the function where we add a fully connected head.
+
+53
+00:03:13,510 --> 00:03:17,960
+This is where we delay as we add now back to the top of our Viji that network.
+
+54
+00:03:18,190 --> 00:03:24,340
+Notice this is different to the layers we added in the mobile network and that's because PDG has a different
+
+55
+00:03:24,340 --> 00:03:26,000
+design to mobile and that.
+
+56
+00:03:26,020 --> 00:03:30,190
+So you're going to have to look at the final design BTG and replace easily as here.
+
+57
+00:03:30,340 --> 00:03:35,700
+And this here this densely a number of densely as dense units here.
+
+58
+00:03:36,190 --> 00:03:38,440
+By default we are going to use 256.
+
+59
+00:03:38,440 --> 00:03:47,550
+However this function allows us to specify it in here we can add 128 and it would be 128 units here.
+
+60
+00:03:47,890 --> 00:03:50,480
+So let's leave the default right.
+
+61
+00:03:50,500 --> 00:03:57,220
+And then you said drop out who said these things we input a number of classes which is 17 from the flow
+
+62
+00:03:57,220 --> 00:04:01,450
+was data set 17 17 Sivam should make sense you know.
+
+63
+00:04:01,780 --> 00:04:04,730
+And we just concatenated models here.
+
+64
+00:04:05,110 --> 00:04:08,800
+Well the parts of the model to get the full model and then printed out.
+
+65
+00:04:08,800 --> 00:04:13,690
+So let's take a look at and we see there 14 million parameters.
+
+66
+00:04:13,880 --> 00:04:18,150
+It's less than between 19 and 16 sorry BTD 19.
+
+67
+00:04:18,440 --> 00:04:23,180
+And with treatable parameters only 135 tells him that's quite good.
+
+68
+00:04:23,720 --> 00:04:25,060
+So let me just run this.
+
+69
+00:04:25,130 --> 00:04:33,150
+So we have fresh and no we just do it data generators here to deflower validation and Floetry unfold
+
+70
+00:04:33,250 --> 00:04:35,290
+as we said our size.
+
+71
+00:04:35,320 --> 00:04:38,210
+We can go actually just keep it at 16.
+
+72
+00:04:38,490 --> 00:04:38,910
+All right.
+
+73
+00:04:38,950 --> 00:04:43,140
+And keep going here.
+
+74
+00:04:43,260 --> 00:04:49,500
+So now we declare all callbacks right here and we just create we create a callback array which we pass
+
+75
+00:04:49,500 --> 00:04:51,740
+in here and let's run this now.
+
+76
+00:04:51,850 --> 00:04:55,430
+So I to leave you to run this over and run this already.
+
+77
+00:04:55,450 --> 00:04:56,800
+And it takes quite some time.
+
+78
+00:04:57,040 --> 00:05:01,540
+But what I want you to observe is look at the validation accuracy in 25 books.
+
+79
+00:05:01,540 --> 00:05:06,230
+The highest we get was actually 95 percent which is quite good.
+
+80
+00:05:06,820 --> 00:05:11,500
+So you keep going see did it ever pass 95 tree at one time.
+
+81
+00:05:11,560 --> 00:05:12,990
+So this is quite good.
+
+82
+00:05:13,240 --> 00:05:19,370
+So we've got 95 percent accuracy using transfer linning using Viji 16 in translating.
+
+83
+00:05:19,630 --> 00:05:22,710
+So let's keep going let's see what else we can do.
+
+84
+00:05:22,750 --> 00:05:24,080
+OK.
+
+85
+00:05:24,430 --> 00:05:26,020
+So this section here.
+
+86
+00:05:26,020 --> 00:05:27,620
+Can we speed this up.
+
+87
+00:05:27,730 --> 00:05:31,060
+So let's try resizing the images to 64 by 64.
+
+88
+00:05:31,200 --> 00:05:34,820
+You remember it was assigned to a can 224 224.
+
+89
+00:05:34,910 --> 00:05:37,660
+Now let's do this to 64.
+
+90
+00:05:37,930 --> 00:05:44,100
+So let's use this comment to setting the input size.
+
+91
+00:05:44,100 --> 00:05:49,660
+Now to.
+
+92
+00:05:49,780 --> 00:05:55,670
+All right and do the standard thing where we load with image that way it's we don't include the top
+
+93
+00:05:55,780 --> 00:06:01,810
+specified in U shape and we make the last train with three syllables.
+
+94
+00:06:02,190 --> 00:06:04,040
+So that's good.
+
+95
+00:06:04,050 --> 00:06:07,050
+And now let's move on to this.
+
+96
+00:06:07,460 --> 00:06:13,330
+Let us actually start treating the small so as we can see this model has a different input sites.
+
+97
+00:06:14,180 --> 00:06:16,010
+And let's see what we get.
+
+98
+00:06:16,010 --> 00:06:18,940
+So I've trained this before so you don't have to do it.
+
+99
+00:06:18,950 --> 00:06:26,180
+So what I want you to see though is that what what's happened here previously before actually did not
+
+100
+00:06:26,180 --> 00:06:30,130
+used the callbacks or that's it in view but I should have thought it and I.
+
+101
+00:06:30,410 --> 00:06:32,490
+But what I've done now is a discipline we do.
+
+102
+00:06:32,540 --> 00:06:41,660
+So we see some callbacks feedback from stopping so we see it's not increasing monitoring patients is
+
+103
+00:06:41,660 --> 00:06:42,310
+good.
+
+104
+00:06:42,320 --> 00:06:45,740
+So at the end Epopt 12 is what we use.
+
+105
+00:06:45,770 --> 00:06:49,530
+So let's go back to Iraq 12 pastorate ago.
+
+106
+00:06:49,920 --> 00:06:53,210
+That's this one 82 percent.
+
+107
+00:06:53,230 --> 00:06:58,340
+So 82 percent was our best loess validation loss and our best accuracy.
+
+108
+00:06:58,340 --> 00:07:06,500
+So you can see by resizing the images a 64 by 64 which is a substantial decrease in size 2 to 24 by
+
+109
+00:07:06,500 --> 00:07:09,580
+224 we got it into possessory.
+
+110
+00:07:09,860 --> 00:07:10,860
+How much was it again.
+
+111
+00:07:11,520 --> 00:07:11,850
+Sorry.
+
+112
+00:07:11,950 --> 00:07:13,930
+82 percent accuracy.
+
+113
+00:07:14,060 --> 00:07:20,570
+So that's not too bad to be fair actually sorry 86 percent accuracy we got that was fifteen point five
+
+114
+00:07:20,570 --> 00:07:22,150
+six five two.
+
+115
+00:07:22,370 --> 00:07:22,730
+Right.
+
+116
+00:07:22,730 --> 00:07:24,540
+So that is actually this one.
+
+117
+00:07:25,010 --> 00:07:26,140
+So yep.
+
+118
+00:07:26,150 --> 00:07:27,620
+So this is good.
+
+119
+00:07:27,710 --> 00:07:29,960
+It's not great but is pretty good.

15. Transfer Learning Build a Flower & Monkey Breed Classifier/4.1 Download the 17-Flowers Dataset.html

@@ -0,0 +1 @@
+<script type="text/javascript">window.location = "https://drive.google.com/file/d/16KBCSvjMSCJSdcrvcws3g-bk9Ov9JrFS/view?usp=sharing";</script>

16. Design Your Own CNN - LittleVGG A Simpsons Classifier/1. Chapter Introduction.srt

@@ -0,0 +1,27 @@
+1
+00:00:00,780 --> 00:00:08,490
+Hi and welcome to Chapter 16 where we get to design or customize CNN one week we're going to call little
+
+2
+00:00:08,490 --> 00:00:12,360
+Viji.
+
+3
+00:00:12,420 --> 00:00:17,270
+So in this section we've introduced the concepts of how we developed the tool VDB.
+
+4
+00:00:17,520 --> 00:00:21,630
+And then in sixteen point two we're actually going to use little Viji to do some Simpsons character
+
+5
+00:00:21,630 --> 00:00:22,720
+recognition.
+
+6
+00:00:22,740 --> 00:00:24,630
+So I hope you're looking forward to getting started.
+
+7
+00:00:24,750 --> 00:00:25,850
+Let's get into it.

16. Design Your Own CNN - LittleVGG A Simpsons Classifier/2. Introducing LittleVGG.srt

@@ -0,0 +1,87 @@
+1
+00:00:00,830 --> 00:00:08,220
+And welcome to chapter sixteen point one where I introduce little BTG and customize Fiji network.
+
+2
+00:00:08,460 --> 00:00:15,220
+So BTG little BTG is basically a dumb size version of EGD 16 or 19.
+
+3
+00:00:15,510 --> 00:00:21,320
+And remember Viji inspired networks all use a series of Treffry convolutional Lia's where a number of
+
+4
+00:00:21,320 --> 00:00:25,160
+filters just increase as you go for it and fit it into the network.
+
+5
+00:00:25,160 --> 00:00:29,770
+So let's take a look at a little BTG network dagga on network architecture.
+
+6
+00:00:30,080 --> 00:00:33,550
+So this was Viji 19 and 16 here.
+
+7
+00:00:33,860 --> 00:00:36,490
+So what I've done Viji BTD.
+
+8
+00:00:36,620 --> 00:00:38,640
+You can call them has 9 Wheatly.
+
+9
+00:00:38,690 --> 00:00:41,900
+And basically this is how it lines up compared to this one here.
+
+10
+00:00:41,900 --> 00:00:42,480
+All right.
+
+11
+00:00:42,650 --> 00:00:50,460
+So we have it first compositionally is here with 64 filter's Max spieling then our Ottilia here is convolutional
+
+12
+00:00:50,460 --> 00:00:52,700
+Lia's with 128 filters.
+
+13
+00:00:53,000 --> 00:00:54,070
+And then this one here.
+
+14
+00:00:54,080 --> 00:00:56,400
+However we don't have as much we just have to.
+
+15
+00:00:56,840 --> 00:01:00,070
+And then we have Max beling again and then our fully connectedly.
+
+16
+00:01:00,080 --> 00:01:01,180
+So we stop here.
+
+17
+00:01:01,430 --> 00:01:06,760
+We don't go on to do as deeply as here like even Viji 11 does.
+
+18
+00:01:06,830 --> 00:01:14,090
+So we just stop at it 56 count one count of this last convolutional filter and then we get straight
+
+19
+00:01:14,090 --> 00:01:16,200
+into the FC leads.
+
+20
+00:01:16,490 --> 00:01:18,570
+So this is a number of parameters here.
+
+21
+00:01:18,740 --> 00:01:25,810
+So let's build this in Chris and then we get to use this on the Simpsons joining on the Simpsons character
+
+22
+00:01:26,070 --> 00:01:26,400
+set.

16. Design Your Own CNN - LittleVGG A Simpsons Classifier/3. Simpsons Character Recognition using LittleVGG.srt

@@ -0,0 +1,583 @@
+1
+00:00:00,630 --> 00:00:06,260
+So in Section sixteen point two we're going to use little Viji on our Simpsons character.
+
+2
+00:00:06,260 --> 00:00:09,180
+They are set to do some Simpsons character recognition.
+
+3
+00:00:09,180 --> 00:00:10,560
+This is going to be pretty cool.
+
+4
+00:00:10,560 --> 00:00:15,720
+So this latest ad was sourced from Kaggle Here's a link to it and it basically it actually came with
+
+5
+00:00:15,720 --> 00:00:21,060
+more classes and this however limited it to 20 classes of the most popular characters and there are
+
+6
+00:00:21,060 --> 00:00:24,990
+about 200 to 400 R.G. be pitches in it each.
+
+7
+00:00:24,990 --> 00:00:31,530
+All of which are different sizes and orientations and aspect ratios and images of yes as I just said
+
+8
+00:00:31,530 --> 00:00:31,770
+that.
+
+9
+00:00:31,780 --> 00:00:38,250
+So that's fine but it's good to know that even though some images have multiple characters some of the
+
+10
+00:00:38,250 --> 00:00:41,420
+images are basically the main characters focus of it.
+
+11
+00:00:41,610 --> 00:00:45,030
+So let's take a look at some of the images here.
+
+12
+00:00:45,360 --> 00:00:47,100
+So this is what I mean by the way.
+
+13
+00:00:47,100 --> 00:00:49,440
+Like Lisa is definitely the main character here.
+
+14
+00:00:49,440 --> 00:00:51,150
+However they are the characters here.
+
+15
+00:00:51,360 --> 00:00:53,600
+So that may confuse overclassify.
+
+16
+00:00:53,730 --> 00:00:55,140
+But let's see how it performs.
+
+17
+00:00:57,450 --> 00:00:57,860
+OK.
+
+18
+00:00:57,880 --> 00:01:03,090
+So we're in a virtual machine with Python Simpson's notebook open.
+
+19
+00:01:03,100 --> 00:01:08,420
+Let me just direct you to how we got into here in case you're confused says Chapter 16 here.
+
+20
+00:01:08,530 --> 00:01:15,280
+I knew it when CNN called it Viji and I hope you downloaded the data set in the resources section of
+
+21
+00:01:15,280 --> 00:01:18,650
+this file of the section of this chapter and place it right here.
+
+22
+00:01:18,700 --> 00:01:21,770
+And again just always check to make sure the images are here.
+
+23
+00:01:21,880 --> 00:01:27,610
+So we have characters or classes and basically let's go back to this file.
+
+24
+00:01:27,610 --> 00:01:28,400
+All right.
+
+25
+00:01:28,540 --> 00:01:30,370
+So let's have an open here.
+
+26
+00:01:30,530 --> 00:01:34,520
+So now basically you've seen the standard way we do things right.
+
+27
+00:01:34,600 --> 00:01:39,580
+What I'm going to tell you about here though is that we're resizing the images to the two by two the
+
+28
+00:01:39,580 --> 00:01:41,240
+two pixels are quite small.
+
+29
+00:01:41,320 --> 00:01:42,410
+So let's see how it performs.
+
+30
+00:01:42,430 --> 00:01:43,270
+OK.
+
+31
+00:01:43,270 --> 00:01:50,140
+So we're doing some data augmentation the same usual thing as we do rotations with shifting horizontal
+
+32
+00:01:50,140 --> 00:01:52,410
+flipping so we can get to characters.
+
+33
+00:01:52,420 --> 00:01:54,110
+Adds a variety to the characters.
+
+34
+00:01:54,520 --> 00:01:59,490
+And then we declare our generators here and then this is the part I want to show you.
+
+35
+00:01:59,740 --> 00:02:02,350
+This is where we build our little Viji model.
+
+36
+00:02:02,350 --> 00:02:03,040
+All right.
+
+37
+00:02:03,040 --> 00:02:09,020
+So I believe you remember from diagram should units lights that this is how it's defined.
+
+38
+00:02:09,040 --> 00:02:11,240
+So we have to convolutional is here.
+
+39
+00:02:11,550 --> 00:02:18,090
+Sixty four filters each and all the filters all the kernel sizes here are tree by tree.
+
+40
+00:02:18,130 --> 00:02:20,640
+This is typical of Viji family model.
+
+41
+00:02:20,790 --> 00:02:21,730
+All right.
+
+42
+00:02:21,970 --> 00:02:25,720
+So we have these convolutional is here then itude set.
+
+43
+00:02:25,720 --> 00:02:28,940
+Second second set here and then another set here.
+
+44
+00:02:28,960 --> 00:02:37,380
+These are the ones with 256 filters and then we have a final F.C. density is here with some drop out.
+
+45
+00:02:37,420 --> 00:02:41,270
+We have two sets of dense dense connections here.
+
+46
+00:02:41,890 --> 00:02:48,530
+And then we finally go to 0 number Asaf Max classify with number of classes which is 20 which we defined
+
+47
+00:02:48,650 --> 00:02:49,350
+up here.
+
+48
+00:02:50,890 --> 00:02:52,330
+So let's run this actually.
+
+49
+00:02:52,330 --> 00:02:55,790
+Let me run the first one.
+
+50
+00:02:55,950 --> 00:02:56,680
+There we go.
+
+51
+00:02:57,450 --> 00:02:59,410
+And then let's run this one here.
+
+52
+00:03:00,870 --> 00:03:01,250
+So good.
+
+53
+00:03:01,260 --> 00:03:04,940
+So we just display our models quite smoothly as here.
+
+54
+00:03:05,100 --> 00:03:09,620
+But luckily this is a substantially smaller model compared to Visagie 16.
+
+55
+00:03:09,640 --> 00:03:12,090
+Is only 2.2 million parameters.
+
+56
+00:03:12,390 --> 00:03:15,510
+And if you want to take a look at our model we can plot it.
+
+57
+00:03:15,570 --> 00:03:16,320
+Remember how we did it.
+
+58
+00:03:16,320 --> 00:03:22,830
+Ilya Ylia chapters so this is a visualization of one model shows you the inputs and outputs of every
+
+59
+00:03:22,850 --> 00:03:25,090
+layer.
+
+60
+00:03:25,110 --> 00:03:31,590
+It's quite long but imagine how long each doing 19 would be okay.
+
+61
+00:03:31,720 --> 00:03:35,620
+So this is where I should put some notes in for you guys.
+
+62
+00:03:36,910 --> 00:03:44,160
+Training our little G.G. model right.
+
+63
+00:03:44,200 --> 00:03:48,300
+So we have all callbacks here seem typical call that we've used before.
+
+64
+00:03:48,430 --> 00:03:55,330
+Checkpointing really stopping and leading rate adjustments on Pluto and a number of samples that we've
+
+65
+00:03:55,330 --> 00:03:58,510
+gotten from the generators before actually did.
+
+66
+00:03:58,810 --> 00:04:01,610
+And we're going to train transfer at least to any box you can dream from.
+
+67
+00:04:01,990 --> 00:04:07,840
+Always recommend trying for more but if time is of the essence and this is more of a practical educational
+
+68
+00:04:07,840 --> 00:04:11,840
+exercise as opposed to us trying to get the best performance out of these models.
+
+69
+00:04:12,070 --> 00:04:15,650
+So what we're doing here we're just testing a little VDU model.
+
+70
+00:04:15,930 --> 00:04:20,080
+It's just one here just to see how he performs on our data set.
+
+71
+00:04:20,080 --> 00:04:23,200
+So I've run this for 10 bucks as I said.
+
+72
+00:04:23,470 --> 00:04:25,260
+And let's see how it performs.
+
+73
+00:04:25,330 --> 00:04:28,190
+Fifteen percent on validation accuracy.
+
+74
+00:04:28,240 --> 00:04:29,100
+Not great.
+
+75
+00:04:29,260 --> 00:04:29,740
+OK.
+
+76
+00:04:29,950 --> 00:04:31,040
+And again not great.
+
+77
+00:04:31,040 --> 00:04:37,720
+Are there any accuracy to that as we train steadily improves the good to this column here on the training
+
+78
+00:04:37,750 --> 00:04:40,920
+accuracy keeps going up and up.
+
+79
+00:04:41,200 --> 00:04:43,680
+And also a good validation accuracy.
+
+80
+00:04:43,780 --> 00:04:45,300
+It also keeps going up and up.
+
+81
+00:04:45,340 --> 00:04:52,310
+So I'm pretty sure if I left the trim maybe 50 bucks it could have gone 90s into percent accuracy.
+
+82
+00:04:52,330 --> 00:04:53,620
+So this is good to know.
+
+83
+00:04:53,830 --> 00:05:00,460
+So we do see the flexibility and power of a simple Viji model that honestly doesn't take that long to
+
+84
+00:05:00,460 --> 00:05:06,460
+train 500 something seconds says no to that pre-book.
+
+85
+00:05:06,630 --> 00:05:13,610
+All right so this is let's look at the performance of this model so let me just title this section Vigurs
+
+86
+00:05:13,740 --> 00:05:15,660
+to make this a bit cleaner.
+
+87
+00:05:21,130 --> 00:05:21,970
+OK.
+
+88
+00:05:22,280 --> 00:05:25,680
+My psyche is giving some of this use sticking about.
+
+89
+00:05:25,770 --> 00:05:28,380
+But look at the confusion metrics here.
+
+90
+00:05:28,550 --> 00:05:33,890
+So we can see a high number in the middle rows in a row here which is also good.
+
+91
+00:05:33,900 --> 00:05:40,440
+Always good but we do see some issues here with some characters being basically basically misclassified.
+
+92
+00:05:40,760 --> 00:05:43,740
+So we know this is a 77 percent accurate model.
+
+93
+00:05:43,850 --> 00:05:47,380
+So we know it is going to have some issues but it's generally going to be quite good.
+
+94
+00:05:47,900 --> 00:05:56,500
+So we can see which characters is performing poorly on and the 1 score to find that may be interesting.
+
+95
+00:05:56,900 --> 00:05:57,860
+All right.
+
+96
+00:05:58,040 --> 00:06:00,140
+And point one score for him.
+
+97
+00:06:00,410 --> 00:06:04,390
+But a super high precision rates means a lot of false positives.
+
+98
+00:06:04,730 --> 00:06:05,080
+Right.
+
+99
+00:06:05,150 --> 00:06:08,510
+And no one else seems to be nearly that bad.
+
+100
+00:06:10,490 --> 00:06:15,130
+Devon to see may be here being misclassified as Mo.
+
+101
+00:06:16,030 --> 00:06:17,290
+That is interesting.
+
+102
+00:06:17,290 --> 00:06:17,990
+All right.
+
+103
+00:06:18,220 --> 00:06:21,290
+But generally we see a nice smooth diagonal trolled here.
+
+104
+00:06:21,310 --> 00:06:25,640
+This is a much easier way to visualize this data.
+
+105
+00:06:25,690 --> 00:06:26,300
+OK.
+
+106
+00:06:26,900 --> 00:06:30,670
+So I'm going to look at our model and see if I'm a.
+
+107
+00:06:33,500 --> 00:06:35,110
+Takes about 10 seconds.
+
+108
+00:06:35,180 --> 00:06:37,790
+I hate malls for this reason.
+
+109
+00:06:37,790 --> 00:06:38,430
+There we go.
+
+110
+00:06:38,690 --> 00:06:42,340
+So this is a messy open see if you could create it.
+
+111
+00:06:42,530 --> 00:06:44,740
+However you can spend some time going through it.
+
+112
+00:06:44,900 --> 00:06:48,440
+It's not super difficult to understand and use similar code before.
+
+113
+00:06:48,620 --> 00:06:53,930
+But basically we're just going to display right good display predicted over.
+
+114
+00:06:53,990 --> 00:06:54,550
+True.
+
+115
+00:06:54,560 --> 00:06:56,730
+So let's see how it performs.
+
+116
+00:06:56,840 --> 00:06:57,330
+Good.
+
+117
+00:06:57,350 --> 00:07:01,850
+This is Mulhouse is is not homo.
+
+118
+00:07:01,960 --> 00:07:04,760
+This text is too big it can resize it after actually.
+
+119
+00:07:04,760 --> 00:07:07,850
+Let me show you how to resize it as we're here.
+
+120
+00:07:07,890 --> 00:07:10,390
+So as you see this is this function here.
+
+121
+00:07:10,380 --> 00:07:13,680
+Draw test is where we actually drawing a text.
+
+122
+00:07:13,710 --> 00:07:21,750
+So this is the font size so we can reduce the font size here and we see how it becomes so good.
+
+123
+00:07:21,750 --> 00:07:27,270
+If you're wondering though what this did to here this is the Ticknor sort of boldness of the top of
+
+124
+00:07:27,270 --> 00:07:28,030
+the font.
+
+125
+00:07:28,530 --> 00:07:31,050
+And there are a number of forms we can use an open C.v.
+
+126
+00:07:31,110 --> 00:07:33,540
+This is one of the nice looking ones in my opinion.
+
+127
+00:07:33,660 --> 00:07:36,430
+So let's take a look at testify results.
+
+128
+00:07:36,450 --> 00:07:43,530
+This is Charles Montgomery Burns and it is in fact Charles I mean it's a poor very good dresser declawing
+
+129
+00:07:43,620 --> 00:07:46,710
+should be easy to spot given his green hair.
+
+130
+00:07:46,710 --> 00:07:49,140
+Very good side show Bob pretty good.
+
+131
+00:07:49,200 --> 00:07:51,500
+Kristi again Mo.
+
+132
+00:07:52,530 --> 00:07:54,140
+This is clearly not principles.
+
+133
+00:07:54,210 --> 00:07:59,440
+This is Lisa however because she's wearing a cap that's probably why it got confuse.
+
+134
+00:07:59,490 --> 00:08:03,900
+Although the captor's is not similar to what Principal is going to wear.
+
+135
+00:08:03,900 --> 00:08:10,290
+So I'm not sure why or classify as well that this would be a good time to actually visualize how classifiers
+
+136
+00:08:10,290 --> 00:08:13,360
+perceived the character's pool.
+
+137
+00:08:13,390 --> 00:08:18,400
+Again at Ibraham or grandpa.
+
+138
+00:08:18,950 --> 00:08:19,600
+That's pretty good.
+
+139
+00:08:19,690 --> 00:08:20,840
+So for classified.
+
+140
+00:08:20,860 --> 00:08:25,680
+Seventy seven percent accurate it actually performed fairly well in our test data.
+
+141
+00:08:26,170 --> 00:08:26,950
+OK.
+
+142
+00:08:27,050 --> 00:08:30,100
+So I hope you had some fun playing with little Ujiji.
+
+143
+00:08:30,140 --> 00:08:35,990
+It's a very good model in my opinion and you can use Adeptus do a number of your applications if you
+
+144
+00:08:35,990 --> 00:08:36,750
+want.
+
+145
+00:08:37,180 --> 00:08:37,550
+OK.
+
+146
+00:08:37,760 --> 00:08:38,060
+Thank you.

16. Design Your Own CNN - LittleVGG A Simpsons Classifier/3.1 Download Simpsons Dataset.html

@@ -0,0 +1 @@
+<script type="text/javascript">window.location = "https://drive.google.com/file/d/1GmS93M5h5CHmQWKzMtXdAZmNW2jmxo01/view?usp=sharing";</script>

17. Advanced Activation Functions & Initializations/1. Chapter Introduction.srt

@@ -0,0 +1,27 @@
+1
+00:00:00,750 --> 00:00:07,020
+Hi and welcome to Chapter 17 where talk about advance activation functions and initializations those
+
+2
+00:00:07,020 --> 00:00:12,830
+are some features that Kurus allows us to configure when tuning or creating our CNN's.
+
+3
+00:00:12,840 --> 00:00:18,720
+So before we begin I'm going to talk about a dying real problem and introduce to you to why we need
+
+4
+00:00:18,730 --> 00:00:21,510
+liquorish real elu and previews as well.
+
+5
+00:00:21,840 --> 00:00:24,570
+And then I talk about advance initializations.
+
+6
+00:00:24,850 --> 00:00:25,150
+OK.
+
+7
+00:00:25,170 --> 00:00:26,400
+So let's get started.

17. Advanced Activation Functions & Initializations/2. Dying ReLU Problem and Introduction to Leaky ReLU, ELU and PReLUs.srt

@@ -0,0 +1,279 @@
+1
+00:00:00,560 --> 00:00:07,040
+Hi and welcome to chapter seventeen point one we are introduced to introduce to you some advance activation
+
+2
+00:00:07,040 --> 00:00:07,780
+functions.
+
+3
+00:00:09,870 --> 00:00:17,100
+So if you remember from earlier slides activation functions are what introduces the non linear linearity
+
+4
+00:00:17,490 --> 00:00:20,850
+that provide neural nets with incredible performance.
+
+5
+00:00:20,850 --> 00:00:26,940
+However what we've said before really is basically the activation unit of choice for all with CNN's.
+
+6
+00:00:27,090 --> 00:00:33,860
+And it basically left you at that and you probably will as usual if no one ever really isn't perfect.
+
+7
+00:00:34,020 --> 00:00:40,000
+I'll tell you why there's a problem and it's called the dying real problem.
+
+8
+00:00:40,090 --> 00:00:45,830
+So when training a number of real units can often die during training.
+
+9
+00:00:45,900 --> 00:00:47,250
+And what does that mean.
+
+10
+00:00:47,260 --> 00:00:49,240
+This happens when a large gritty inflows.
+
+11
+00:00:49,240 --> 00:00:55,210
+True that true that neuron which causes the weeds to update in such a way that the unit never activates
+
+12
+00:00:55,230 --> 00:00:55,790
+anymore.
+
+13
+00:00:55,990 --> 00:01:01,220
+So regardless of future input PERDIDA that really you basically is always going to be switched off.
+
+14
+00:01:01,240 --> 00:01:04,360
+So we're going to have a part of the network that's effectively dead.
+
+15
+00:01:04,750 --> 00:01:10,620
+So what happens is that exactly as it says here the output of this unit is always going to be zero.
+
+16
+00:01:10,810 --> 00:01:17,000
+So now we have basically wasted units and a wasted amount of connections and that work.
+
+17
+00:01:17,020 --> 00:01:22,760
+And apparently sometimes as much as 40 percent of them that can be dead because of these dying real
+
+18
+00:01:23,260 --> 00:01:24,020
+units.
+
+19
+00:01:25,850 --> 00:01:28,250
+So how do we fix a dying a real problem.
+
+20
+00:01:28,640 --> 00:01:35,240
+OK so fiercely Let's take a look at the different types of real functions are about to discuss you know
+
+21
+00:01:35,240 --> 00:01:40,130
+really it is a standard clamping basically at zero activation function.
+
+22
+00:01:40,190 --> 00:01:41,010
+Right.
+
+23
+00:01:41,060 --> 00:01:43,380
+Everything over zero is allowed to pass.
+
+24
+00:01:43,400 --> 00:01:46,250
+Everything negative is basically clamped at zero.
+
+25
+00:01:46,610 --> 00:01:53,120
+However what is leaky leaky real do has a small negative slope it's a green one here and basically it
+
+26
+00:01:53,120 --> 00:01:54,430
+is a linear function.
+
+27
+00:01:54,590 --> 00:02:00,830
+However it is a parameter in front of it that basically allows it to not grow that much.
+
+28
+00:02:00,830 --> 00:02:06,950
+So it's not going to have basically a factor that basically limits how much it can how big how negative
+
+29
+00:02:06,950 --> 00:02:08,910
+it can go.
+
+30
+00:02:08,960 --> 00:02:12,350
+What about you know previous tons of parametric.
+
+31
+00:02:12,950 --> 00:02:16,880
+And basically it is very similar to leaky Belu.
+
+32
+00:02:17,000 --> 00:02:23,810
+However parametric pre-New or parametric reel basically has a lot of function and that a parameter that
+
+33
+00:02:23,810 --> 00:02:31,820
+can control the steepness of the slope and Eliu which is exponential riu basically the negative portion
+
+34
+00:02:31,820 --> 00:02:35,830
+of it follows an exponential curve with these parameters here.
+
+35
+00:02:36,250 --> 00:02:37,050
+OK.
+
+36
+00:02:38,270 --> 00:02:42,660
+So it tends to be a good mix of the good parts of really and leaky really.
+
+37
+00:02:42,860 --> 00:02:46,660
+However it can saturate on large negative values as you can see here.
+
+38
+00:02:47,050 --> 00:02:47,460
+OK.
+
+39
+00:02:51,000 --> 00:02:56,120
+So there are some other exotic and exotic because they're not commonly used.
+
+40
+00:02:56,160 --> 00:03:03,600
+Activision functions there's Kerry-Lugar which combines concatenates Alpert's of tubular functions one
+
+41
+00:03:03,600 --> 00:03:05,240
+positive and one negative.
+
+42
+00:03:05,640 --> 00:03:07,050
+Doubling the output value.
+
+43
+00:03:07,050 --> 00:03:12,920
+Not entirely sure when you would use this but is a decent paper here you can read about it is also reduced
+
+44
+00:03:12,920 --> 00:03:13,600
+6.
+
+45
+00:03:13,680 --> 00:03:17,310
+So basically really six is just capped at negative six.
+
+46
+00:03:17,360 --> 00:03:22,350
+There is no special reason for selecting six other than it would be best for it is a four data set according
+
+47
+00:03:22,350 --> 00:03:23,490
+to this paper here.
+
+48
+00:03:23,880 --> 00:03:26,730
+And there are many others as well as Maxo at offset.
+
+49
+00:03:26,760 --> 00:03:27,820
+Do you get the idea.
+
+50
+00:03:28,050 --> 00:03:34,700
+However in practice I would suggest you use BQE real or relo.
+
+51
+00:03:34,880 --> 00:03:37,130
+So when you use something other than a redo.
+
+52
+00:03:37,370 --> 00:03:42,370
+So we've just seen some of the variations here but is no hard and fast rule and disciplining.
+
+53
+00:03:42,530 --> 00:03:46,340
+That's why some people called deepening Wolfer art than science.
+
+54
+00:03:46,430 --> 00:03:49,770
+That's kind of true because of the number of factors.
+
+55
+00:03:49,800 --> 00:03:54,210
+And basically there's so many interchangeable and dependencies.
+
+56
+00:03:54,380 --> 00:03:55,670
+How does your data look.
+
+57
+00:03:55,700 --> 00:03:57,670
+What is it going to be used on.
+
+58
+00:03:57,720 --> 00:03:59,840
+There's a lot of things to configure and network.
+
+59
+00:04:00,050 --> 00:04:01,880
+That's why a lot of people say it's an art.
+
+60
+00:04:01,880 --> 00:04:07,270
+However there are some general rules you can use that would generally get you good results.
+
+61
+00:04:07,390 --> 00:04:08,130
+OK.
+
+62
+00:04:08,270 --> 00:04:15,800
+So generally a good rule of thumb is to always use a real office and then you are just lending rates
+
+63
+00:04:15,800 --> 00:04:19,300
+to get the best accuracy you can with your CNN analyst.
+
+64
+00:04:19,400 --> 00:04:19,880
+OK.
+
+65
+00:04:20,300 --> 00:04:24,790
+Once that's done then you can start experimenting with different real functions.
+
+66
+00:04:24,920 --> 00:04:30,930
+You can go from leaky really to do as well as a nice progressive step you can take.
+
+67
+00:04:31,000 --> 00:04:35,200
+However in most cases it can skip leakage really and go straight to Eliu.
+
+68
+00:04:35,670 --> 00:04:40,340
+I always think others find I get better results with elu compared to real.
+
+69
+00:04:40,550 --> 00:04:46,800
+So if you want to just try to get the best network and best accuracy as possible use Elu.
+
+70
+00:04:46,910 --> 00:04:47,340
+OK.

17. Advanced Activation Functions & Initializations/3. Advanced Initializations.srt

@@ -0,0 +1,151 @@
+1
+00:00:00,510 --> 00:00:00,830
+OK.
+
+2
+00:00:00,840 --> 00:00:07,580
+So this brings us to Section seventeen point two where we talk about advance initializations.
+
+3
+00:00:07,700 --> 00:00:13,250
+So something we sort of glossed over in discourse is that we all the sales starting weights are random
+
+4
+00:00:13,400 --> 00:00:16,440
+or initials all the way it's but are it truly random.
+
+5
+00:00:16,490 --> 00:00:19,170
+And that means are they pulled from a uniform distribution.
+
+6
+00:00:19,460 --> 00:00:21,240
+Well kind of yes.
+
+7
+00:00:21,260 --> 00:00:23,040
+That is the default.
+
+8
+00:00:23,060 --> 00:00:26,970
+However there are a number of other initializations that Carrot's offers.
+
+9
+00:00:27,160 --> 00:00:28,420
+So let's take a look.
+
+10
+00:00:28,550 --> 00:00:33,000
+These are the many initialization functions Kurus can provide us with.
+
+11
+00:00:33,230 --> 00:00:34,780
+And let's take a look at how they look.
+
+12
+00:00:34,790 --> 00:00:35,490
+All right.
+
+13
+00:00:35,600 --> 00:00:43,720
+So this is what uniform distributions one would look like random normal random normal uniform some claret
+
+14
+00:00:43,940 --> 00:00:48,270
+garrote normal orthogonal identity glorify uniform.
+
+15
+00:00:48,410 --> 00:00:54,710
+So you can see that there are some definite differences in these functions you can do a normal distribution
+
+16
+00:00:54,710 --> 00:00:56,380
+style where it's more sent.
+
+17
+00:00:56,420 --> 00:01:01,160
+The bulk of the initializations would be centered around this area 0 you can do some which just randomly
+
+18
+00:01:01,160 --> 00:01:05,720
+between two point one to point zero one that sort of thing.
+
+19
+00:01:05,860 --> 00:01:06,650
+OK.
+
+20
+00:01:07,970 --> 00:01:10,820
+So we have tons of initializers.
+
+21
+00:01:10,820 --> 00:01:12,260
+Which one do we use.
+
+22
+00:01:12,260 --> 00:01:17,690
+So generally we always want to use a zero scented initialization within a small range example minus
+
+23
+00:01:17,690 --> 00:01:18,340
+1 to 1.
+
+24
+00:01:18,350 --> 00:01:19,490
+Typically best.
+
+25
+00:01:19,730 --> 00:01:26,330
+And this was recommended in Stanfords C-s to treat one costs can be division goes which is of course
+
+26
+00:01:26,330 --> 00:01:31,460
+I highly recommend you take it's pretty theoretical but it goes into a lot of detail especially in the
+
+27
+00:01:31,460 --> 00:01:35,170
+training part of CNN's and neural networks as well.
+
+28
+00:01:35,480 --> 00:01:41,810
+So some other good choices for initialises h e normal works pretty well when you're using real activations
+
+29
+00:01:42,470 --> 00:01:44,630
+Gourab normal works pretty well too.
+
+30
+00:01:44,840 --> 00:01:51,360
+And glue rotini form which is the Karris is a default random initialiser if you're going back.
+
+31
+00:01:51,380 --> 00:01:52,560
+That is what it is here.
+
+32
+00:01:54,610 --> 00:02:01,290
+So most times just so you know the initial dose of choice you choose it doesn't really impact your accuracy
+
+33
+00:02:01,290 --> 00:02:02,530
+as you get in the end.
+
+34
+00:02:02,670 --> 00:02:06,620
+However it can definitely impact a number of ebox we take to get there.
+
+35
+00:02:06,900 --> 00:02:14,070
+So it is something you can experiment with maybe after the fact if you if you think for some reason
+
+36
+00:02:14,070 --> 00:02:18,650
+that you read in the research paper that if you're using maybe an inception style model or resin that's
+
+37
+00:02:18,660 --> 00:02:23,940
+50 model it works better with this type of initialises initialization then you change it.
+
+38
+00:02:23,940 --> 00:02:26,360
+Otherwise I would just stick with the Cristi fault.

18 . Deep Survaliance - Build a Face Detector with Emotion, Age and Gender Recognition/18.2 Building an Emotion Detector with LittleVGG.ipynb

@@ -0,0 +1,723 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Using LittleVGG for Emotion Detection"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Training Emotion Detector"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 8,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Found 28709 images belonging to 7 classes.\n",
+      "Found 3589 images belonging to 7 classes.\n"
+     ]
+    }
+   ],
+   "source": [
+    "from __future__ import print_function\n",
+    "import tensorflow as tf\n",
+    "from tensorflow.keras.preprocessing.image import ImageDataGenerator\n",
+    "from tensorflow.keras.models import Sequential\n",
+    "from tensorflow.keras.layers import Dense, Dropout, Activation, Flatten, BatchNormalization\n",
+    "from tensorflow.keras.layers import Conv2D, MaxPooling2D\n",
+    "from tensorflow.keras.preprocessing.image import ImageDataGenerator\n",
+    "import os\n",
+    "\n",
+    "num_classes = 7\n",
+    "img_rows, img_cols = 48, 48\n",
+    "batch_size = 16\n",
+    "\n",
+    "train_data_dir = './fer2013/train'\n",
+    "validation_data_dir = './fer2013/validation'\n",
+    "\n",
+    "# Let's use some data augmentaiton \n",
+    "train_datagen = ImageDataGenerator(\n",
+    "      rescale=1./255,\n",
+    "      rotation_range=30,\n",
+    "      shear_range=0.3,\n",
+    "      zoom_range=0.3,\n",
+    "      width_shift_range=0.4,\n",
+    "      height_shift_range=0.4,\n",
+    "      horizontal_flip=True,\n",
+    "      fill_mode='nearest')\n",
+    " \n",
+    "validation_datagen = ImageDataGenerator(rescale=1./255)\n",
+    " \n",
+    "train_generator = train_datagen.flow_from_directory(\n",
+    "        train_data_dir,\n",
+    "        color_mode = 'grayscale',\n",
+    "        target_size=(img_rows, img_cols),\n",
+    "        batch_size=batch_size,\n",
+    "        class_mode='categorical',\n",
+    "        shuffle=True)\n",
+    " \n",
+    "validation_generator = validation_datagen.flow_from_directory(\n",
+    "        validation_data_dir,\n",
+    "        color_mode = 'grayscale',\n",
+    "        target_size=(img_rows, img_cols),\n",
+    "        batch_size=batch_size,\n",
+    "        class_mode='categorical',\n",
+    "        shuffle=True)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Our Keras Imports"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 9,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from tensorflow.keras.models import Sequential\n",
+    "from tensorflow.keras.layers import BatchNormalization\n",
+    "from tensorflow.keras.layers import Conv2D, MaxPooling2D\n",
+    "from tensorflow.keras.layers import ELU\n",
+    "from tensorflow.keras.layers import Activation, Flatten, Dropout, Dense"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Keras LittleVGG Model"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 10,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Model: \"sequential_1\"\n",
+      "_________________________________________________________________\n",
+      "Layer (type)                 Output Shape              Param #   \n",
+      "=================================================================\n",
+      "conv2d_8 (Conv2D)            (None, 48, 48, 32)        320       \n",
+      "_________________________________________________________________\n",
+      "activation_11 (Activation)   (None, 48, 48, 32)        0         \n",
+      "_________________________________________________________________\n",
+      "batch_normalization_10 (Batc (None, 48, 48, 32)        128       \n",
+      "_________________________________________________________________\n",
+      "conv2d_9 (Conv2D)            (None, 48, 48, 32)        9248      \n",
+      "_________________________________________________________________\n",
+      "activation_12 (Activation)   (None, 48, 48, 32)        0         \n",
+      "_________________________________________________________________\n",
+      "batch_normalization_11 (Batc (None, 48, 48, 32)        128       \n",
+      "_________________________________________________________________\n",
+      "max_pooling2d_4 (MaxPooling2 (None, 24, 24, 32)        0         \n",
+      "_________________________________________________________________\n",
+      "dropout_6 (Dropout)          (None, 24, 24, 32)        0         \n",
+      "_________________________________________________________________\n",
+      "conv2d_10 (Conv2D)           (None, 24, 24, 64)        18496     \n",
+      "_________________________________________________________________\n",
+      "activation_13 (Activation)   (None, 24, 24, 64)        0         \n",
+      "_________________________________________________________________\n",
+      "batch_normalization_12 (Batc (None, 24, 24, 64)        256       \n",
+      "_________________________________________________________________\n",
+      "conv2d_11 (Conv2D)           (None, 24, 24, 64)        36928     \n",
+      "_________________________________________________________________\n",
+      "activation_14 (Activation)   (None, 24, 24, 64)        0         \n",
+      "_________________________________________________________________\n",
+      "batch_normalization_13 (Batc (None, 24, 24, 64)        256       \n",
+      "_________________________________________________________________\n",
+      "max_pooling2d_5 (MaxPooling2 (None, 12, 12, 64)        0         \n",
+      "_________________________________________________________________\n",
+      "dropout_7 (Dropout)          (None, 12, 12, 64)        0         \n",
+      "_________________________________________________________________\n",
+      "conv2d_12 (Conv2D)           (None, 12, 12, 128)       73856     \n",
+      "_________________________________________________________________\n",
+      "activation_15 (Activation)   (None, 12, 12, 128)       0         \n",
+      "_________________________________________________________________\n",
+      "batch_normalization_14 (Batc (None, 12, 12, 128)       512       \n",
+      "_________________________________________________________________\n",
+      "conv2d_13 (Conv2D)           (None, 12, 12, 128)       147584    \n",
+      "_________________________________________________________________\n",
+      "activation_16 (Activation)   (None, 12, 12, 128)       0         \n",
+      "_________________________________________________________________\n",
+      "batch_normalization_15 (Batc (None, 12, 12, 128)       512       \n",
+      "_________________________________________________________________\n",
+      "max_pooling2d_6 (MaxPooling2 (None, 6, 6, 128)         0         \n",
+      "_________________________________________________________________\n",
+      "dropout_8 (Dropout)          (None, 6, 6, 128)         0         \n",
+      "_________________________________________________________________\n",
+      "conv2d_14 (Conv2D)           (None, 6, 6, 256)         295168    \n",
+      "_________________________________________________________________\n",
+      "activation_17 (Activation)   (None, 6, 6, 256)         0         \n",
+      "_________________________________________________________________\n",
+      "batch_normalization_16 (Batc (None, 6, 6, 256)         1024      \n",
+      "_________________________________________________________________\n",
+      "conv2d_15 (Conv2D)           (None, 6, 6, 256)         590080    \n",
+      "_________________________________________________________________\n",
+      "activation_18 (Activation)   (None, 6, 6, 256)         0         \n",
+      "_________________________________________________________________\n",
+      "batch_normalization_17 (Batc (None, 6, 6, 256)         1024      \n",
+      "_________________________________________________________________\n",
+      "max_pooling2d_7 (MaxPooling2 (None, 3, 3, 256)         0         \n",
+      "_________________________________________________________________\n",
+      "dropout_9 (Dropout)          (None, 3, 3, 256)         0         \n",
+      "_________________________________________________________________\n",
+      "flatten_1 (Flatten)          (None, 2304)              0         \n",
+      "_________________________________________________________________\n",
+      "dense_3 (Dense)              (None, 64)                147520    \n",
+      "_________________________________________________________________\n",
+      "activation_19 (Activation)   (None, 64)                0         \n",
+      "_________________________________________________________________\n",
+      "batch_normalization_18 (Batc (None, 64)                256       \n",
+      "_________________________________________________________________\n",
+      "dropout_10 (Dropout)         (None, 64)                0         \n",
+      "_________________________________________________________________\n",
+      "dense_4 (Dense)              (None, 64)                4160      \n",
+      "_________________________________________________________________\n",
+      "activation_20 (Activation)   (None, 64)                0         \n",
+      "_________________________________________________________________\n",
+      "batch_normalization_19 (Batc (None, 64)                256       \n",
+      "_________________________________________________________________\n",
+      "dropout_11 (Dropout)         (None, 64)                0         \n",
+      "_________________________________________________________________\n",
+      "dense_5 (Dense)              (None, 7)                 455       \n",
+      "_________________________________________________________________\n",
+      "activation_21 (Activation)   (None, 7)                 0         \n",
+      "=================================================================\n",
+      "Total params: 1,328,167\n",
+      "Trainable params: 1,325,991\n",
+      "Non-trainable params: 2,176\n",
+      "_________________________________________________________________\n",
+      "None\n"
+     ]
+    }
+   ],
+   "source": [
+    "model = Sequential()\n",
+    "\n",
+    "model.add(Conv2D(32, (3, 3), padding = 'same', kernel_initializer=\"he_normal\",\n",
+    "                 input_shape = (img_rows, img_cols, 1)))\n",
+    "model.add(Activation('elu'))\n",
+    "model.add(BatchNormalization())\n",
+    "model.add(Conv2D(32, (3, 3), padding = \"same\", kernel_initializer=\"he_normal\", \n",
+    "                 input_shape = (img_rows, img_cols, 1)))\n",
+    "model.add(Activation('elu'))\n",
+    "model.add(BatchNormalization())\n",
+    "model.add(MaxPooling2D(pool_size=(2, 2)))\n",
+    "model.add(Dropout(0.2))\n",
+    "\n",
+    "# Block #2: second CONV => RELU => CONV => RELU => POOL\n",
+    "# layer set\n",
+    "model.add(Conv2D(64, (3, 3), padding=\"same\", kernel_initializer=\"he_normal\"))\n",
+    "model.add(Activation('elu'))\n",
+    "model.add(BatchNormalization())\n",
+    "model.add(Conv2D(64, (3, 3), padding=\"same\", kernel_initializer=\"he_normal\"))\n",
+    "model.add(Activation('elu'))\n",
+    "model.add(BatchNormalization())\n",
+    "model.add(MaxPooling2D(pool_size=(2, 2)))\n",
+    "model.add(Dropout(0.2))\n",
+    "\n",
+    "# Block #3: third CONV => RELU => CONV => RELU => POOL\n",
+    "# layer set\n",
+    "model.add(Conv2D(128, (3, 3), padding=\"same\", kernel_initializer=\"he_normal\"))\n",
+    "model.add(Activation('elu'))\n",
+    "model.add(BatchNormalization())\n",
+    "model.add(Conv2D(128, (3, 3), padding=\"same\", kernel_initializer=\"he_normal\"))\n",
+    "model.add(Activation('elu'))\n",
+    "model.add(BatchNormalization())\n",
+    "model.add(MaxPooling2D(pool_size=(2, 2)))\n",
+    "model.add(Dropout(0.2))\n",
+    "\n",
+    "# Block #4: third CONV => RELU => CONV => RELU => POOL\n",
+    "# layer set\n",
+    "model.add(Conv2D(256, (3, 3), padding=\"same\", kernel_initializer=\"he_normal\"))\n",
+    "model.add(Activation('elu'))\n",
+    "model.add(BatchNormalization())\n",
+    "model.add(Conv2D(256, (3, 3), padding=\"same\", kernel_initializer=\"he_normal\"))\n",
+    "model.add(Activation('elu'))\n",
+    "model.add(BatchNormalization())\n",
+    "model.add(MaxPooling2D(pool_size=(2, 2)))\n",
+    "model.add(Dropout(0.2))\n",
+    "\n",
+    "# Block #5: first set of FC => RELU layers\n",
+    "model.add(Flatten())\n",
+    "model.add(Dense(64, kernel_initializer=\"he_normal\"))\n",
+    "model.add(Activation('elu'))\n",
+    "model.add(BatchNormalization())\n",
+    "model.add(Dropout(0.5))\n",
+    "\n",
+    "# Block #6: second set of FC => RELU layers\n",
+    "model.add(Dense(64, kernel_initializer=\"he_normal\"))\n",
+    "model.add(Activation('elu'))\n",
+    "model.add(BatchNormalization())\n",
+    "model.add(Dropout(0.5))\n",
+    "\n",
+    "# Block #7: softmax classifier\n",
+    "model.add(Dense(num_classes, kernel_initializer=\"he_normal\"))\n",
+    "model.add(Activation(\"softmax\"))\n",
+    "\n",
+    "print(model.summary())"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Training our model"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 12,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "WARNING:tensorflow:sample_weight modes were coerced from\n",
+      "  ...\n",
+      "    to  \n",
+      "  ['...']\n",
+      "Train for 1795 steps\n",
+      "1795/1795 [==============================] - 607s 338ms/step - loss: 2.0255 - accuracy: 0.2012\n"
+     ]
+    }
+   ],
+   "source": [
+    "from tensorflow.keras.optimizers import RMSprop, SGD, Adam\n",
+    "from tensorflow.keras.callbacks import ModelCheckpoint, EarlyStopping, ReduceLROnPlateau\n",
+    "\n",
+    "                     \n",
+    "checkpoint = ModelCheckpoint(\"emotion_little_vgg.h5\",\n",
+    "                             monitor=\"val_loss\",\n",
+    "                             mode=\"min\",\n",
+    "                             save_best_only = True,\n",
+    "                             verbose=1)\n",
+    "\n",
+    "earlystop = EarlyStopping(monitor = 'val_loss', \n",
+    "                          min_delta = 0, \n",
+    "                          patience = 3,\n",
+    "                          verbose = 1,\n",
+    "                          restore_best_weights = True)\n",
+    "\n",
+    "reduce_lr = ReduceLROnPlateau(monitor = 'val_loss', factor = 0.2, patience = 3, verbose = 1, min_delta = 0.0001)\n",
+    "\n",
+    "# we put our call backs into a callback list\n",
+    "callbacks = [earlystop, checkpoint] #reduce_lr]\n",
+    "\n",
+    "# We use a very small learning rate \n",
+    "model.compile(loss = 'categorical_crossentropy',\n",
+    "              optimizer = Adam(lr=0.001),\n",
+    "              metrics = ['accuracy'])\n",
+    "\n",
+    "nb_train_samples = 28273\n",
+    "nb_validation_samples = 3534\n",
+    "epochs = 5\n",
+    "\n",
+    "history = model.fit(\n",
+    "    train_generator,\n",
+    "    epochs = epochs)\n",
+    "    callbacks = callbacks)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 15,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Found 3589 images belonging to 7 classes.\n",
+      "Confusion Matrix\n",
+      "[[  0   0   0 439   0  52   0]\n",
+      " [  0   0   0  52   0   3   0]\n",
+      " [  0   0   0 486   0  42   0]\n",
+      " [  0   0   0 790   0  89   0]\n",
+      " [  0   0   0 565   0  61   0]\n",
+      " [  0   0   0 496   0  98   0]\n",
+      " [  0   0   0 401   0  15   0]]\n",
+      "Classification Report\n",
+      "              precision    recall  f1-score   support\n",
+      "\n",
+      "       Angry       0.00      0.00      0.00       491\n",
+      "     Disgust       0.00      0.00      0.00        55\n",
+      "        Fear       0.00      0.00      0.00       528\n",
+      "       Happy       0.24      0.90      0.38       879\n",
+      "     Neutral       0.00      0.00      0.00       626\n",
+      "         Sad       0.27      0.16      0.21       594\n",
+      "    Surprise       0.00      0.00      0.00       416\n",
+      "\n",
+      "    accuracy                           0.25      3589\n",
+      "   macro avg       0.07      0.15      0.08      3589\n",
+      "weighted avg       0.10      0.25      0.13      3589\n",
+      "\n"
+     ]
+    },
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "C:\\ProgramData\\Anaconda3\\envs\\cv\\lib\\site-packages\\sklearn\\metrics\\classification.py:1437: UndefinedMetricWarning: Precision and F-score are ill-defined and being set to 0.0 in labels with no predicted samples.\n",
+      "  'precision', 'predicted', average, warn_for)\n"
+     ]
+    },
+    {
+     "data": {
+      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAekAAAHHCAYAAACbaKDRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3debglVXnv8e8PZFAREBCCgOKAUxwQUCESgxoTp4hjnHJFQtLG2Rhzg8NVork3Js6axKQjKniNI0FwuAoiOJAANtiCiCIiSgcCNggOINrd7/2j6tDb9gzd5+yza1fx/TxPPadqVe3aa/dw3v2uWkOqCkmSNH226roCkiRpdgZpSZKmlEFakqQpZZCWJGlKGaQlSZpSBmlJkqbUrbqugCRJS/X7D79tXXPt+rHf99zzb/pcVT167DfeTAZpSVLvXXPtes753J3Gft+t9/zObmO/6RYwSEuSeq+ADWzouhpj5zNpSZKmlJm0JGkAivVlJi1JkibETFqS1HvNM+nhLRhlkJYkDYIdxyRJ0sSYSUuSeq8o1tfwmrvNpCVJmlJm0pKkQbDjmCRJU6iA9QMM0jZ3S5I0pcykJUmDMMTmbjNpSZKmlJm0JKn3CgY5BMsgLUkahOHNN2ZztyRJU8tMWpLUe0U5BEuSJE2OmbQkqf8K1g8vkTaTliRpWplJS5J6rxhm726DtCRpAMJ60nUlxs7mbkmSppSZtCSp9wrYYMcxSZI0KWbSkqRBGOIzaYO0JKn3imEGaZu7JUmaUmbSkqRB2FBm0pIkaULMpCVJvTfUZ9IGaUlS7xVh/QAbh4f3iSRJGggzaUnSINhxTJIkTYyZtCSp9+w4JknS1Arra3iNw8P7RJIkDYSZtCSp9wrYMMC8c3ifSJKkgTCTliQNwhA7jplJS5I0pcykJUm9V2XvbkmSptYGMvZtIUnumWT1yPbjJC9LskuSU5N8p/15+/b6JHlnkkuSnJ/kgPnub5CWJGmRqurbVbV/Ve0PHAjcAJwIHA2cVlX7Aae1xwCPAfZrtxXAu+e7v0FaktR7zYxjW41920KPBL5bVd8HDgeOa8uPA57Y7h8OHF+Ns4Cdk+w51w0N0pIkzW23JKtGthXzXPsM4EPt/h5VdSVA+3P3tnwv4PKR16xpy2ZlxzFJ0gAsW8extVV10ILvnmwLPAF45UKXzlJWc11skJYk9d4UzDj2GOC8qrqqPb4qyZ5VdWXbnH11W74G2GfkdXsDV8x1U5u7JUlaumeysakb4GTgiHb/COCkkfLntL28Dwaun2kWn42ZtCRpENZXNzOOJbkN8CjgeSPFbwQ+muQo4AfA09ryzwCPBS6h6Ql+5Hz3NkhLkrQEVXUDsOsmZdfQ9Pbe9NoCXri59zZIS5J6r8hihkxNPYO0JGkQNjgtqCRJmhQzaUlS783MODY0w/tEkiQNhJm0JKn3inQ2BGs5LUsmneRJSSrJvZbj/pIk3RIsV3P3M4Gv0Ew2vmRJzPglSfPawFZj37o29uCXZAfgocDDaaY/OybJYcAxwFrgvsC5wB9VVSV5LPDW9tx5wF2r6vFJjgHuCOwLrE2yD/Diqlrdvs+ZwPOr6vxxfwZJUr9UsVwLbHRqOTLUJwKfraqLk1yb5IC2/IHAb9JMJH4m8NAkq4B/AR5WVd9L8qFN7nUgcGhV3ZjkCOC5wMuS3APYbq4A3S4ltgJga7Y+8DbsOOaPKG10051v03UVxma779/QdRXGKrcaTiNcrVvXdRXG4uf8jF/UTcN7eLxMluNf8DOBt7f7H26PPw2cU1VrAJKspsmQfwpcWlXfa6//EG1wbZ1cVTe2+x8D/leSvwT+GHj/XBWoqpXASoAds0s9JL82M5s0Nhe/dsFV7HrjHn+yqusqjNXWu+2+8EU9sf6qqxe+qAfOrtOW6c5hw6yrQPbbWIN0kl2BRwD3TVLA1jTD1z4D3DRy6fr2vRf6E/3ZzE5V3ZDkVOBw4A+B4fxmlCRpFuPOpJ8KHF9VN68EkuSLwKFzXP8t4K5J9q2qy4CnL3D/9wCfBL5cVdeOob6SpAEofCa9OZ5JszzXqBOA5wPf3fTi9lnzC4DPJlkLnDPfzavq3CQ/Bt43pvpKkgZiiDOOjTVIV9Vhs5S9E3jnJmUvGjk8varulSTAPwKr2muO2fReSe5IM2zslPHVWpKk6TQNXzv+tO1IdiGwE01v71+T5DnA2cCrq2rDBOsnSZpyRdhQ49+61vn4hKp6G/C2zbjueOD45a+RJEnTofMgLUnSOPhMWpKkKVTAhgH27h7eJ5IkaSDMpCVJAxDWD3DGMTNpSZKmlJm0JKn3fCYtSZImykxakjQIQ3wmbZCWJPVeVWzuliRJk2MmLUkahCEuVTm8TyRJ0kCYSUuSeq+ADXYckyRpGsXmbkmSNDlm0pKk3mtmHLO5W9Im7v3qH3RdhbFZ33UFxmz9VVd3XQVpSQzSkqRBWD/AJ7gGaUlS7xUZZHP38L52SJI0EGbSkqRB2DDAvHN4n0iSpIEwk5Yk9V4VrPeZtCRJmhQzaUnSIAyxd7dBWpLUe80QrOE1Dg/vE0mSNBBm0pKkQVg/wKUqzaQlSZpSBmlJUu/NrII17m1zJNk5yceTfCvJRUkOSbJLklOTfKf9efv22iR5Z5JLkpyf5ID57m2QliQNQNNxbNzbZnoH8NmquhfwAOAi4GjgtKraDzitPQZ4DLBfu60A3j3fjQ3SkiQtUpIdgYcBxwJU1S+q6jrgcOC49rLjgCe2+4cDx1fjLGDnJHvOdX+DtCRpEDaQsW+b4a7AD4H3JflakvckuS2wR1VdCdD+3L29fi/g8pHXr2nLZmWQliRpbrslWTWyrdjk/K2AA4B3V9UDgZ+xsWl7NrNF/prrYodgSZJ6bxnn7l5bVQfNc34NsKaqzm6PP04TpK9KsmdVXdk2Z189cv0+I6/fG7hirpubSUuSBqGLjmNV9d/A5Unu2RY9EvgmcDJwRFt2BHBSu38y8Jy2l/fBwPUzzeKzMZOWJGlpXgx8MMm2wKXAkTRJ8EeTHAX8AHhae+1ngMcClwA3tNfOySAtSeq9Zu7ubmYcq6rVwGxN4o+c5doCXri597a5W5KkKWUmLUkahM0cMtUrW5xJJ1mfZHWSC5N8PcnLk2zVnjsoyTvHX81fq8O+SZ613O8jSVKXFpNJ31hV+wMk2R34N2An4HVVtQpYNcb6zWVf4Fnte0uSbuFm5u4emiU9k66qq2nmHn1R2538sCSfAkjyO23GvbqdheV2SbZK8k9tFv6pJJ9J8tT2+suS7NbuH5TkjLnuA7wR+O227M+X8hkkScPQ4dzdy2bJz6Sr6tK2uXv3TU69AnhhVZ2ZZAfg58CTabLg+7XXXwS8d4G3mO0+RwOvqKrHz/aCdkaYFQDbc5tFfS5Jkro2rq8Js7UxnAm8NclLgJ2rah1wKPCxqtrQDgA/fTPuPdt95lVVK6vqoKo6aBu224KPIUnqpWVYpnIams+XHKST3BVYz8YpzwCoqjcCfwLcGjgryb2YPZjPWDdSn+0XuI8kSYO3pCCd5A7APwP/0A7QHj13t6q6oKr+jqYz2b2ArwBPaZ9N7wEcNvKSy4AD2/2nLHCfnwC3W0rdJUnDUXS2CtayWswz6VsnWQ1sQ5P9fgB46yzXvSzJw2my7G8C/w/4Jc0MLN8ALgbOBq5vr/9r4Ngkr2rL57vPBmBdkq8D76+qty3ic0iSBmQamqfHbYuDdFVtPc+5M4Az2v0Xz3ZNkldU1U+T7AqcA1zQXv9l4B6z3HPW+zDLdGuSJA1JFzOOfSrJzsC2wBvaDmSSJC3aUMdJTzxIV9Vhk35PSZL6yLm7JUmDYCYtSdIU6nKpyuXU/ZxnkiRpVmbSkqRBmIZxzeNmJi1J0pQyk5Yk9V8Ns+OYmbQkSVPKTFqS1HtOZiJJ0hQbYpC2uVuSpCllJi1J6j0nM5EkSRNlJi1JGoQaYCZtkJYkDYIzjkmSpIkxk5Yk9V4545gkSZokM2lpiS591x5dV2Fs7vyHV3ddhbHaetdduq7C2Ky/5tquqzD17DgmSdJUcpy0JEmaIDNpSdIgDLG520xakqQpZSYtSeq9oS5VaSYtSdKUMpOWJPVfNROaDI1BWpI0CM7dLUmSJsZMWpLUe4VDsCRJ0gSZSUuSBmCY04IapCVJgzDE3t02d0uSNKXMpCVJg2DHMUmS9CuSXJbkgiSrk6xqy3ZJcmqS77Q/b9+WJ8k7k1yS5PwkB8x3b4O0JKn3qppMetzbFnh4Ve1fVQe1x0cDp1XVfsBp7THAY4D92m0F8O75bmqQliQNwobK2LclOBw4rt0/DnjiSPnx1TgL2DnJnnPdxCAtSdLSFHBKknOTrGjL9qiqKwHan7u35XsBl4+8dk1bNis7jkmSBmGZhmDtNvOcubWyqlZucs1Dq+qKJLsDpyb51jz3my09n7PmBmlJkua2duQ586yq6or259VJTgQeDFyVZM+qurJtzr66vXwNsM/Iy/cGrpjr3jZ3S5IGoYuOY0lum+R2M/vA7wHfAE4GjmgvOwI4qd0/GXhO28v7YOD6mWbx2Uw0k06yHrhgpOiJVXXZJOsgSRqeYot7Y4/LHsCJSaCJqf9WVZ9N8lXgo0mOAn4APK29/jPAY4FLgBuAI+e7+aSbu2+sqv3HdbM0fyqpqg3juqckSZurqi4FHjBL+TXAI2cpL+CFm3v/zpu7k2yd5E1JvtoO7H5eW75DktOSnNcOEj+8Ld83yUVJ/gk4j19t25ck3ULVMmxdm3Qmfeskq9v971XVk4CjaNrkH5RkO+DMJKfQdFF/UlX9OMluwFlJTm5fe0/gyKp6wWxv0naBXwGwPbdZzs8jSdKymYbm7t8D7p/kqe3xTjQzsawB/k+ShwEbaMaR7dFe8/12EPis2u7xKwF2zC7T8GVIkrScaphzd0/DEKwAL66qz/1KYfJc4A7AgVX1yySXAdu3p3820RpKktSBzp9JA58Dnp9kG4Ak92i7se8EXN0G6IcDd+6ykpKkKTfAh9LTkEm/B9gXOK/trf1DmjlOPwh8sp3pZTUw3wwukqRbOJu7l6iqdpilbAPwqnbb1CFz3Oq+46yXJEnTaBoyaUmSlmyZ5u7u1DQ8k5YkSbMwk5Yk9V7hM2lJkqZTAQMM0jZ3S5I0pcykJUmDYMcxSZI0MWbSkqRhGGAmbZCWJA1ABtm72+ZuSZKmlJm0JGkYBtjcbSYtSdKUMpOWJPVfDXPGMTNpSZKmlJm0JGkYBvhM2iAtSRoIm7slSdKEmElLkoZhgM3dZtKSJE0pM2lpib516Ae6rsLY/D77d12F8dr19l3XYHyuubbrGky/AWbSBmlJUv8V4DhpSZI0KWbSkqRBqAE2d5tJS5I0pcykJUnDMMBM2iAtSRoGO45JkqRJMZOWJA1CBtjcbSYtSdKUMpOWJPVfMciOY2bSkiRNKTNpSdIAZJC9uw3SkqRhsLlbkiRNipm0JGkYzKQlSdKkmElLkoZhgJm0QVqS1H/FIHt329wtSdKUMkhLkgYhNf5ts9432TrJ15J8qj2+S5Kzk3wnyUeSbNuWb9ceX9Ke33ehexukJUlampcCF40c/x3wtqraD/gRcFRbfhTwo6q6O/C29rp5GaQlScNQy7AtIMnewOOA97THAR4BfLy95Djgie3+4e0x7flHttfPaWxBOslPNzl+bpJ/GNf9JUmaQm8H/iewoT3eFbiuqta1x2uAvdr9vYDLAdrz17fXz8lMWpKkue2WZNXItmLmRJLHA1dX1bkj18+WGddmnJvVRIZgJfkD4DXAtsA1wLOr6qokxwB3o/l2sQ/w91X1r0kOA17fXntP4EvAC4AjgftW1Z+39/1T4N5V9fJJfA5J0vTa3I5eW2htVR00x7mHAk9I8lhge2BHmsx65yS3arPlvYEr2uvX0MS6NUluBewEXDvfm48zk751ktUzG02QnfEV4OCqeiDwYZqmgRn3p2nPPwR4bZI7tuUPBv4CuB9NIH9y+9onJNmmveZI4H2bViTJiplvPb/kpvF9QkmSWlX1yqrau6r2BZ4BfKGqng2cDjy1vewI4KR2/+T2mPb8F6pqYpn0jVW1/8xBkucCM98+9gY+kmRPmmz6eyOvO6mqbgRuTHI6TXC+Djinqi5t7/Uh4NCq+niSLwCPT3IRsE1VXbBpRapqJbASYMfsMsA5aCRJv2Z6JjP5K+DDSf4G+BpwbFt+LPCBJJfQZNDPWOhGk5px7F3AW6vq5LYp+5iRc5sG0Vqg/D3Aq4BvMUsWLUnSpFXVGcAZ7f6lNAnnptf8HHjaltx3Uh3HdgL+q90/YpNzhyfZPsmuwGHAV9vyB7cDwrcCnk7TZE5VnU3Tpv8s4EPLXXFJUg8sx/CrKWiHnVSQPgb4WJIvA2s3OXcO8GngLOANVTXzgP0/gTcC36BpHj9x5DUfBc6sqh8tZ6UlST0ywCA9tubuqtphk+P3A+9v909i44PzTV1cVStmKb+hqp4+x2sOpZmtRZKkwerVOOkkOye5mKaT2mld10eSND26mrt7OXW6VGVVHTNH+Rm0D+A3Kb8OuMeyVkqSpCnhetKSpGGYgsx33AzSkqRhGGCQ7tUzaUmSbknMpCVJvTctHb3GzUxakqQpZSYtSRqG6Zm7e2wM0pKkYbC5W5IkTYqZtCRpEOw4JkmSJsZMWpI0DGbSkiRpUsykJUn9N9DJTAzSkqRhGGCQtrlbkqQpZSYtSRoGM2lJkjQpZtLSEj3hO4/uugpj9N9dV2Cs8rMbu66CJmiIHcfMpCVJmlIGaUmSppTN3ZKkYbC5W5IkTYqZtCSp/5xxTJKkKTbAIG1ztyRJU8pMWpI0DGbSkiRpUsykJUm9F4bZccxMWpKkKWUmLUkahgFm0gZpSVL/DXSctM3dkiRNKTNpSdIwmElLkqRJMZOWJA3DADNpg7QkaRDsOCZJkibGTFqSNAxm0pIkaVLMpCVJ/VeYSc9IUkneMnL8iiTHLPJeOyd5wSJfe1mS3RbzWknSsKTGv3Vtsc3dNwFPHlOA3BmYNUgn2XoM95ckaeySbJ/knCRfT3Jhkr9uy++S5Owk30nykSTbtuXbtceXtOf3Xeg9Fhuk1wErgT+fpdJ3SHJCkq+220Pb8mOSvGLkum+0FXwjcLckq5O8KclhSU5P8m/ABe21n0hybvuHsGKRdZYkDVktwza/m4BHVNUDgP2BRyc5GPg74G1VtR/wI+Co9vqjgB9V1d2Bt7XXzWspHcf+EXh2kp02KX9HW7kHAU8B3rPAfY4GvltV+1fVX7ZlDwZeXVX3aY//uKoOBA4CXpJk1/lumGRFklVJVv2Sm7bkM0mStFmq8dP2cJt2K+ARwMfb8uOAJ7b7h7fHtOcfmSTzvceiO45V1Y+THA+8BLhx5NTvAvcZed8dk9xuC29/TlV9b+T4JUme1O7vA+wHXDNP3VbSZPrsmF2m4KmCJGm5dfEMuX0sey5wd5rk9bvAdVW1rr1kDbBXu78XcDlAVa1Lcj2wK7B2rvsvtXf324HzgPeNlG0FHFJVo4GbJOv41cx9+3nu+7OR1x1GE/gPqaobkpyxwGslSRqX3ZKsGjle2SaCAFTVemD/JDsDJwL3nuUeM18fZsua5/1qsaRx0lV1LfBRNra3A5wCvGjmIMn+7e5lwAFt2QHAXdrynwDzZdo70bTh35DkXsDBS6mzJGmglueZ9NqqOmhkW8ksquo64AyaGLVzkpkkeG/ginZ/DU1rMO35nYBr5/tI45jM5C3AaC/vlwAHJTk/yTeBP2vLTwB2SbIaeD5wMUBVXQOc2XYke9Ms9/8scKsk5wNvAM4aQ50lSUOyHAF6gebztqP0zu3+rWlafS8CTgee2l52BHBSu39ye0x7/gtVNe+7LKq5u6p2GNm/CrjNyPFa4OmzvOZG4PfmuN+zNik6Y+TcTcBj5njdvltQbUmSxmlP4Lj2ufRWwEer6lNtgvrhJH8DfA04tr3+WOADSS6hyaCfsdAbOOOYJKn3wuwPfJdTVZ0PPHCW8ktpRiltWv5z4Glb8h7O3S1J0pQyk5YkDcMAB9wapCVJgzANc22Pm83dkiRNKTNpSdIwmElLkqRJMZOWJA3DADNpg7Qkqf/KjmOSJGmCzKQlScNgJi1JkibFTFqSNAg+k5YkSRNjJi1JGoYBZtIGaUnSINjcLUmSJsZMWpLUf8Ugm7vNpCVJmlJm0tISfXftrl1XYWz25r+7rsJY1U47dF2F8fmvrivQAwPMpA3SkqTeC3YckyRJE2QmLUkaBjNpSZI0KWbSkqRBSA0vlTZIS5L6z3HSkiRpksykJUmD4BAsSZI0MWbSkqRhGGAmbZCWJA2Czd2SJGlizKQlScNgJi1JkibFTFqS1H/lM2lJkjRBZtKSpGEYYCZtkJYk9V6wuVuSJE2QmbQkaRgGuFSlmbQkSVPKTFqSNAg+kx6zJK9OcmGS85OsTvKQzXzdvkm+sdz1kyT1RC3T1rHOMukkhwCPBw6oqpuS7AZs21V9JEmaNl02d+8JrK2qmwCqai1AktcCfwDcGvgP4HlVVUkOBN4L3AB8pZsqS5KmVTZ0XYPx67K5+xRgnyQXJ/mnJL/Tlv9DVT2oqu5LE6gf35a/D3hJVR2y0I2TrEiyKsmqX3LT8tRekqRl1lmQrqqfAgcCK4AfAh9J8lzg4UnOTnIB8AjgN5PsBOxcVV9sX/6BBe69sqoOqqqDtmG75fsQkqTp4TPp8aqq9cAZwBltUH4ecH/goKq6PMkxwPY0k8lMwR+XJGladdG7O8k+wPHAbwAbgJVV9Y4kuwAfAfYFLgP+sKp+lCTAO4DH0jy+fW5VnTfX/TvLpJPcM8l+I0X7A99u99cm2QF4KkBVXQdcn+TQ9vyzJ1dTSZLmtA74i6q6N3Aw8MIk9wGOBk6rqv2A09pjgMcA+7XbCuDd8928y0x6B+BdSXam+ZCX0FT4OuACmm8eXx25/kjgvUluAD432apKkqZa0cmMY1V1JXBlu/+TJBcBewGHA4e1lx1H02r8V2358VVVwFlJdk6yZ3ufX9NZkK6qc4HfmuXUa9pttusfMFJ0zPLUTJKkLZdkX+CBwNnAHjOBt6quTLJ7e9lewOUjL1vTlk1XkJYkaZyW6Zn0bklWjRyvrKqVv/bezSPaE4CXVdWPm0fPs5rtxJw1N0hLkjS3tVV10HwXJNmGJkB/sKr+vS2+aqYZO8mewNVt+Rpgn5GX7w1cMde9XWBDkjQMHQzBantrHwtcVFVvHTl1MnBEu38EcNJI+XPSOBi4fq7n0WAmLUkagNDZAhsPBf4HcEGS1W3Zq4A3Ah9NchTwA+Bp7bnP0Ay/uoRmCNaR893cIC1J0iJV1VeY/TkzwCNnub6AF27u/Q3SkqT+q+pkCNZy85m0JElTykxakjQIHT2TXlYGaUnSMAwwSNvcLUnSlDKTliQNwhCbu82kJUmaUmbSkqT+K2DD8FJpg7QkaRiGF6Nt7pYkaVqZSUuSBsGOY5IkaWLMpCVJw+Dc3ZIkaVLMpKUl+vnlt+u6CprDhu9c1nUVNEFDfCZtkJYk9V/hECxJkjQ5ZtKSpN4LEDuOSZKkSTGTliQNw4auKzB+BmlJ0iDY3C1JkibGTFqS1H8OwZIkSZNkJi1JGoAa5NzdBmlJ0iAMcVpQm7slSZpSZtKSpGEYYHO3mbQkSVPKTFqS1H8FGeCMY2bSkiRNKTNpSdIwDPCZtEFakjQMw4vRNndLkjStzKQlSYPgKliSJGlizKQlScNwS82kk7w6yYVJzk+yOslDlqMyST6TZOfluLckacAK2LAMW8cWzKSTHAI8Hjigqm5Kshuw7ebcPMmtqmrdZlwXIFX12M25ryRJtwSbk0nvCaytqpsAqmptVV2R5LI2YJPkoCRntPvHJFmZ5BTg+CTPTXJSks8m+XaS17XX7ZvkoiT/BJwH7DNzzyS3TfLpJF9P8o0kT29fc2CSLyY5N8nnkuw5/j8SSVLfhCI1/q1rm/NM+hTgtUkuBj4PfKSqvrjAaw4EDq2qG5M8F3gwcF/gBuCrST4NrAXuCRxZVS8AaBJqAB4NXFFVj2vLd0qyDfAu4PCq+mEbuP838MebvnmSFcCK9vCnn6+Pf3szPudS7EbzeYbAz7KlXvrxZX8LJvRZLlvuN2hM7t/YL5b9Hfz/suXuPIH3GIwFg3RV/TTJgcBvAw8HPpLk6AVednJV3ThyfGpVXQOQ5N+BQ4FPAN+vqrNmef0FwJuT/B3wqar6cpL70gT6U9tgvjVw5Rx1XgmsXOizjUuSVVV10KTebzn5WaaTn2U6+VmmzBRkvuO2Wb27q2o9cAZwRpILgCOAdWxsLt9+k5f8bNNbzHG86XUz73dx+8XgscDftk3nJwIXVtUhm1NnSdItzACD9ILPpJPcM8l+I0X7A9+naRk7sC17ygK3eVSSXZLcGngicOYC73lH4Iaq+r/Am4EDgG8Dd2g7spFkmyS/uVD9JUnqq83JpHcA3tUOjVoHXELzvPfewLFJXgWcvcA9vgJ8ALg78G9VtSrJvvNcfz/gTUk2AL8Enl9Vv0jyVOCdSXZq6/524MLN+AzLbWJN6xPgZ5lOfpbp5GeZFjNDsAYmtczNA23HsYOq6kXL+kaSpFusnW5zxzr4nn869vuesvr153b5rN5pQSVJg9DFEKwk701ydZJvjJTtkuTUJN9pf96+LU+Sdya5pJ0c7ICF7r/sQbqq3m8WLUkaqPfTDBsedTRwWlXtB5zWHgM8Btiv3VYA717o5mbSkqRhqBr/tuBb1peAazcpPhw4rt0/jqbD9Ez58dU4C9h5oUm5DNKL0I7ZHowkL92csmnXNiXt03U9JHVhGQL04vts7VFVVwK0P3dvy/cCLh+5bk1bNieD9OL8c5JzkrxgIAuCHDFL2XMnXYmlqqYX5Ce6rse4JHlz34cZts/m5ty6rt+WSHJB+xxx1q3r+i1Gkj2SHJvk/7XH90lyVNf1mjK7JVk1sq1Y+CVzyixl834TcKnKRaiqQ9ux438MrEpyDvC+qjq146ptkSTPBJ4F3CXJySOndgSu6aZWS3ZWkgdV1Ve7rsgYfAtYmeRWwPuAD1XV9R3XaUudS/NLaK5fTnedbJJWAskAAA4rSURBVHWW5PHtzxe2Pz/Q/nw2zZTHffR+mn9br26PLwY+AhzbVYUWrViuyUzWLqJ391VJ9qyqK9vm7Kvb8jXAaGvf3sAV893IIL1IVfWdJK8BVgHvBB6YZr7SV1XVv3dbu832HzRTq+4GvGWk/CdALzMDmqlrn5fk+zQz2oUmyb5/t9XaclX1HuA9Se4JHAmcn+RM4F+r6vRua7d5quouXddhXKrq+wBJHlpVDx05dXT79/L6bmq2JLtV1UeTvBKgqtYlWd91pQbgZJoWyje2P08aKX9Rkg8DDwGun2kWn4tBehGS3J/ml+bjgFOBP6iq89qZ0v4T6EWQbn/pfD/J7wI3VtWGJPcA7kUzf3ofPabrCoxTkq1p/j7uRbP4wdeBlyd5XlU9o9PKbaF2GMp+jEwj3Ha66ZvbJjm0qr4CkOS3gNt2XKfF+lmSXWmbXJMcDPSttWajDiYzSfIh4DCaZvE1wOtogvNH20cHPwCe1l7+GZrpri+haX05cqH7G6QX5x+Af6XJmm9eSKRdwvM13VVr0b4E/Hb7S/Q0mtaBp9M04/XKSLazO78+p3yvJHkr8ASav5P/U1XntKf+Lslyr+w2Vkn+BHgpTfPeauBgmi+0j+iyXot0FPDeduZDgOuYZTW+nng5TXZ3t7Y14A7AU7ut0uJ1sbRkVT1zjlOPnOXaYuPjks1ikN5CbWZzeVV9YLbzc5VPuVTVDe23vndV1d8n+VrXlVqMJE+gabq/I81zoDsDFwF97ID1DeA1VTXb884HT7oyS/RS4EHAWVX18CT3Av664zotSlWdCzwgyY40/3d6m3m2LYC/Q7NscIBvV9UvO66WRhikt1BVrU+ya5Jtq2r5V6udjLQLlzybJkuA/v7beANNlvb5qnpgkocDc33TnXbvA56U5FCa5sivVNWJAD0MDD+vqp8nIcl2VfWt9ll7LyV5HM0Xv+2brihQVb17Jp3kacBnq+rCthXwgCR/U1XndV23RRngKlh9/UXcte8DZ7Y9om9ebrOq3tpdlZbkZcArgRPb/6x3BXrRMWkWv6yqa5JslWSrqjo9zbrkffSPNIvSfKg9fl6S362qLWoumxJr2uGKn6BZE/5HLNCrdVol+WfgNjSdFN9D0zx8zrwvml7/q6o+1n4R/H2aVQffTdOpSVPAIL04V7TbVsDtOq7LklXVF4EvjhxfCrykuxotyXVJdgC+DHwwydU0q7f10e8A922fY5HkOHraoa+qntTuHpPkdGAn4LMdVmkpfquq7p/k/Kr66yRvoSedRWcx05P7ccC7q+qkJMd0WJ/FK2CDmbSAqurls7S5tL80f+1fd1X1sVPP4cCNNK0Dz6YJBr1rhmx9G7gTTcsNNOMrezc0LslWwPlVdV+4+Uthn810Fr2hHdFxLdDXoWb/leRfgN+l6ZC4Hb2d5GpJM4RNLYP0IiT5JL8e1K6n6RX9L1X188nXakleMbK/PfAUepp9VtXPktwZ2K+qjktyG2Drruu1SLsCF7WT5UDT8eo/ZyaeqaondFazLdAO7ft6kjtV1Q+6rs8YfKptuv97mslaoGn27qM/pFkc4s1VdV078cZfdlwnjTBIL86lNEMVZp4VPh24CrgHzdCs/9FRvRal7a066swkvcx2kvwpzeoyuwB3o5kX95+ZZThED7y26wqM0Z7Ahe0XjtF+HL34ogGQ5EE0Izve0B7vQPP44VvA27qs25ZKsmNV/ZjmS/kZbdkuwE00yUY/mUmr9cCqetjI8SeTfKmqHpbkws5qtUibzKG8FXAg8BsdVWepXkgzPOlsuHlmuN3nf8l0qqovJvkNms9TwFer6r87rtZiDeER0UyzMEkeRjNhxYuB/YGV9Gt88b/RTHM627StfZuuddAM0otzh9GmuyR3oplaE6CPw7JG/6OuA77HxqFYfXNTVf1iZlhMO+91L79etxOAvBb4As3fzbuSvL6q3tttzRblsVX1V6MFba/7PrXYbF1VM0sSPh1YWVUnACckWd1hvbZYVT2+ncb4dwbyCKJhJq3WXwBfSfJdml+edwFekOS2bFxDtDeGNL8y8MUkrwJuneRRwAuAT3Zcp8X6S5pWm2sA2ukb/wPoY5B+FPBXm5Q9ZpayabZ1kltV1TqaxyejqyH17ndpVVWSE2lazjSlevcPaxpU1WfSrIJ1L5og/a2RzmJv765mi5PkybMUXw9cUFVXz3Jumh1N0wpwAfA8mrly+9qpZw3NYiczfsKvrkU79ZI8n+aL0t3yq8s53o7mC0effIjmS+Bamh7eXwZIcnf6O9/1cFaNG+gQrNQAmwcmoZ1Uf19GvuhU1fGdVWgJknwaOISNE5gcBpxF0xHu9X2Y6nRAPYdvluR44H40K+gUzfCyc2iWE+zF5Dnt/Na3B/6W5gvUjJ+MNB33RrsAxZ7AKVX1s7bsHsAOfZylK8k3af6f937VuJ2226N+647jX27gs5e97dxFLFU5NmbSi5DkAzQ9h1ezcTKAAnoZpGnWjrl3VV0FzULwbJx16EtsXDd3mn0COAAgyQlV9ZSO6zMO3223GTPL3fVmAp12+tLrk2zarL1Dkh369sWqqs6apeziLuoyJoNaNW6IDNKLcxBwnxpOM8S+MwG6dTVwj6q6NklfJtsf7Z06iJ6pA5s059Ns7Jy4PU0/jm/Tz4VPBqOqvp/kAGBmfvgz+9gicLPB/EreyCC9ON+gGaI072LdPfLlJJ8CPtYePxX4UtsR7rruqrVFao793kpyB+B/0i7kMFPex5ngqup+o8dtYHheR9VRK8lradY6npnW9H1JPlZVf9NhtTTCIL04uwHfbCdmuKktq6o6vMM6LcULgSfTfJsOTQ/1E9qWgod3WbEt8IAkP6ap/63bfdj4jG3H7qq2aB8EPkIznvXPgCOAH3ZaozFpl0h8UNf1EM+kGUHwc4AkbwTOA/oXpAfaccwgvTjHjOyHJrj1dTlEaFb0+URVndAuH3hPmn8bfWnqpqr6OvXnfHatqmOTvHRmEZQezwT38pHDrWj6DwziC0fPXUbTSjMzOmU7frUfRL/Y3C24eSao/YFn0cx9+z2aqSf76kvAbye5PfB5mmkBn06zQIW6M/Ml6cp2/eIrgL07rM9SjHZ2W0fzjPqEjuqijW6ima71VJpc9FE0c0C8E6Cq+roa3mAYpLdAO9TiGTRZ8zU0TZGpqr40Cc8lVXVDkqOAd1XV3yf5WteVEn/TDmH6C+BdwI7An3dbpcWZ6QSX5LYzQ5c0FU5stxlndFSP8TCTvsX7Fs0EBn9QVZcAJOnlL81NJMkhNJnzzHSg/tvoWFV9qt29nv70DZhV++/rWGAH4E5JHgA8r6pe0G3NbrmSbA08qqr+qOu6aG7+It4yT6HJpE9P8lngw/zq0J++ehnwSuDEqrowyV3ZOLGJJizJu5inh3pPmyDfDvw+MLPM5tfbRSrUkapan+QOSbatqj6uObAJ15O+xauqE4ET26FJT6RpetwjybtpAtwpnVZwkWY6JY0cXwr0MRAMxehSgX8NvK6rioxTVV0+s/BJa/1c12piLqNZmvZkfnUJ0amfze7XFLBhQ9e1GDuD9CK0z9Q+CHywXebxaTRTHvYqSCd5e1W9LMknmSVz69Nav0NSVTcv0pLkZaPHPXZ5O5VuJdmW5kvgRR3XSU1nxCtoetz3Zia7WxKD9BK18w//S7v1zcx0n2/utBaaz1Da7/4MeAewF83CIafQjM9XhwY2q53N3RqWqjq3/fnFdnYrqsqxqxq7qlqLQ/qmTpLTmb0VrXez2g2VQfoWrF30/XXAi2g6wG2VZB3NMKzXd1q5W7AkP2HjL87b9Hn2tHbayblUVb1hYpXRbF4xsr89TefYdR3VZenMpDUwLwMeCjyoqr4H0PbsfneSP6+qt3Vau1uoqhrSs8HZxkTflmao366AQbpDM61pI87s66x2Q2WQvmV7Ds04ybUzBVV1aZI/onlmaJDWklTVW2b2k9wOeClwJM3wxbfM9TpNRtvxdcZWNCv8/UZH1Vmicu5uDc42owF6RlX9MMk2XVRIw9MGgpfTPJM+Djigqn7Uba3UOpeNj1bW0QzJOmrOq6dZQZVDsDQs801gMIDJDdS1JG+iWWFtJXC/qvppx1US0K5AdnlV3aU9PoLmefRlwDc7rJo2sVXXFVCnHpDkx7NsPwHut+CrpYX9BXBH4DXAFaP/xkY6xGny/oX2i3g789vf0rRyXE/zhaqfNtT4t46ZSd+CDXR5R02RqjIRmE5bt3M8QLPi3cqqOgE4IcnqDuulTfgfSJJuebZOMpOkPRL4wsi5/iZvVePfOtbfvwxJ0mJ9CPhikrXAjTSr+5Hk7jRN3v1T5dzdkqT+q6r/neQ0YE/glKqbU8atgBd3VzNtyiAtSbdAVXXWLGUXd1GXsZmC5ulx85m0JElTykxakjQI5TNpSZKm0XT0xh43m7slSZpSZtKSpP4rpmKGsHEzk5YkaUqZSUuShmGAq2CZSUuSNKXMpCVJvVdADfCZtEFaktR/VTZ3S5KkjZI8Osm3k1yS5Ohx399MWpI0CJNu7k6yNfCPwKOANcBXk5xcVd8c13uYSUuStDgPBi6pqkur6hfAh4HDx/kGZtKSpGGY/DPpvYDLR47XAA8Z5xsYpCVJvfcTfvS5z9fHd1uGW2+fZNXI8cqqWtnuZ5brx9rmbpCWJPVeVT26g7ddA+wzcrw3cMU438Bn0pIkLc5Xgf2S3CXJtsAzgJPH+QZm0pIkLUJVrUvyIuBzwNbAe6vqwnG+R2qA629KkjQENndLkjSlDNKSJE0pg7QkSVPKIC1J0pQySEuSNKUM0pIkTSmDtCRJU8ogLUnSlPr/Z6bR2NdNyw4AAAAASUVORK5CYII=\n",
+      "text/plain": [
+       "<Figure size 576x576 with 2 Axes>"
+      ]
+     },
+     "metadata": {
+      "needs_background": "light"
+     },
+     "output_type": "display_data"
+    }
+   ],
+   "source": [
+    "import matplotlib.pyplot as plt\n",
+    "import sklearn\n",
+    "from sklearn.metrics import classification_report, confusion_matrix\n",
+    "import numpy as np\n",
+    "\n",
+    "nb_train_samples = 28273\n",
+    "nb_validation_samples = 3534\n",
+    "\n",
+    "# We need to recreate our validation generator with shuffle = false\n",
+    "validation_generator = validation_datagen.flow_from_directory(\n",
+    "        validation_data_dir,\n",
+    "        color_mode = 'grayscale',\n",
+    "        target_size=(img_rows, img_cols),\n",
+    "        batch_size=batch_size,\n",
+    "        class_mode='categorical',\n",
+    "        shuffle=False)\n",
+    "\n",
+    "class_labels = validation_generator.class_indices\n",
+    "class_labels = {v: k for k, v in class_labels.items()}\n",
+    "classes = list(class_labels.values())\n",
+    "\n",
+    "#Confution Matrix and Classification Report\n",
+    "Y_pred = model.predict(validation_generator)\n",
+    "y_pred = np.argmax(Y_pred, axis=1)\n",
+    "\n",
+    "print('Confusion Matrix')\n",
+    "print(confusion_matrix(validation_generator.classes, y_pred))\n",
+    "print('Classification Report')\n",
+    "target_names = list(class_labels.values())\n",
+    "print(classification_report(validation_generator.classes, y_pred, target_names=target_names))\n",
+    "\n",
+    "plt.figure(figsize=(8,8))\n",
+    "cnf_matrix = confusion_matrix(validation_generator.classes, y_pred)\n",
+    "\n",
+    "plt.imshow(cnf_matrix, interpolation='nearest')\n",
+    "plt.colorbar()\n",
+    "tick_marks = np.arange(len(classes))\n",
+    "_ = plt.xticks(tick_marks, classes, rotation=90)\n",
+    "_ = plt.yticks(tick_marks, classes)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Loading our saved model"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 20,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from tensorflow.keras.models import load_model\n",
+    "\n",
+    "classifier = load_model('emotion_little_vgg.h5')"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Get our class labels"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 21,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Found 3589 images belonging to 7 classes.\n",
+      "{0: 'Angry', 1: 'Disgust', 2: 'Fear', 3: 'Happy', 4: 'Neutral', 5: 'Sad', 6: 'Surprise'}\n"
+     ]
+    }
+   ],
+   "source": [
+    "validation_generator = validation_datagen.flow_from_directory(\n",
+    "        validation_data_dir,\n",
+    "        color_mode = 'grayscale',\n",
+    "        target_size=(img_rows, img_cols),\n",
+    "        batch_size=batch_size,\n",
+    "        class_mode='categorical',\n",
+    "        shuffle=False)\n",
+    "\n",
+    "class_labels = validation_generator.class_indices\n",
+    "class_labels = {v: k for k, v in class_labels.items()}\n",
+    "classes = list(class_labels.values())\n",
+    "print(class_labels)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Let's test on some of validation images"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 25,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from tensorflow.keras.models import load_model\n",
+    "from tensorflow.keras.optimizers import RMSprop, SGD, Adam\n",
+    "from tensorflow.keras.preprocessing import image\n",
+    "import numpy as np\n",
+    "import os\n",
+    "import cv2\n",
+    "import numpy as np\n",
+    "from os import listdir\n",
+    "from os.path import isfile, join\n",
+    "import re\n",
+    "\n",
+    "def draw_test(name, pred, im, true_label):\n",
+    "    BLACK = [0,0,0]\n",
+    "    expanded_image = cv2.copyMakeBorder(im, 160, 0, 0, 300 ,cv2.BORDER_CONSTANT,value=BLACK)\n",
+    "    cv2.putText(expanded_image, \"predited - \"+ pred, (20, 60) , cv2.FONT_HERSHEY_SIMPLEX,1, (0,0,255), 2)\n",
+    "    cv2.putText(expanded_image, \"true - \"+ true_label, (20, 120) , cv2.FONT_HERSHEY_SIMPLEX,1, (0,255,0), 2)\n",
+    "    cv2.imshow(name, expanded_image)\n",
+    "\n",
+    "\n",
+    "def getRandomImage(path, img_width, img_height):\n",
+    "    \"\"\"function loads a random images from a random folder in our test path \"\"\"\n",
+    "    folders = list(filter(lambda x: os.path.isdir(os.path.join(path, x)), os.listdir(path)))\n",
+    "    random_directory = np.random.randint(0,len(folders))\n",
+    "    path_class = folders[random_directory]\n",
+    "    file_path = path + path_class\n",
+    "    file_names = [f for f in listdir(file_path) if isfile(join(file_path, f))]\n",
+    "    random_file_index = np.random.randint(0,len(file_names))\n",
+    "    image_name = file_names[random_file_index]\n",
+    "    final_path = file_path + \"/\" + image_name\n",
+    "    return image.load_img(final_path, target_size = (img_width, img_height),grayscale=True), final_path, path_class\n",
+    "\n",
+    "# dimensions of our images\n",
+    "img_width, img_height = 48, 48\n",
+    "\n",
+    "# We use a very small learning rate \n",
+    "model.compile(loss = 'categorical_crossentropy',\n",
+    "              optimizer = RMSprop(lr = 0.001),\n",
+    "              metrics = ['accuracy'])\n",
+    "\n",
+    "files = []\n",
+    "predictions = []\n",
+    "true_labels = []\n",
+    "\n",
+    "# predicting images\n",
+    "for i in range(0, 10):\n",
+    "    path = './fer2013/validation/' \n",
+    "    img, final_path, true_label = getRandomImage(path, img_width, img_height)\n",
+    "    files.append(final_path)\n",
+    "    true_labels.append(true_label)\n",
+    "    x = image.img_to_array(img)\n",
+    "    x = x * 1./255\n",
+    "    x = np.expand_dims(x, axis=0)\n",
+    "    images = np.vstack([x])\n",
+    "    classes = model.predict_classes(images, batch_size = 10)\n",
+    "    predictions.append(classes)\n",
+    "    \n",
+    "for i in range(0, len(files)):\n",
+    "    image = cv2.imread((files[i]))\n",
+    "    image = cv2.resize(image, None, fx=3, fy=3, interpolation = cv2.INTER_CUBIC)\n",
+    "    draw_test(\"Prediction\", class_labels[predictions[i][0]], image, true_labels[i])\n",
+    "    cv2.waitKey(0)\n",
+    "\n",
+    "cv2.destroyAllWindows()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Test on a single image"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 27,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from tensorflow.keras.models import load_model\n",
+    "from tensorflow.keras.preprocessing import image\n",
+    "import numpy as np\n",
+    "import os\n",
+    "import cv2\n",
+    "import numpy as np\n",
+    "from os import listdir\n",
+    "from os.path import isfile, join\n",
+    "from tensorflow.keras.preprocessing.image import img_to_array\n",
+    "\n",
+    "face_classifier = cv2.CascadeClassifier('./Haarcascades/haarcascade_frontalface_default.xml')\n",
+    "\n",
+    "def face_detector(img):\n",
+    "    # Convert image to grayscale\n",
+    "    gray = cv2.cvtColor(img.copy(),cv2.COLOR_BGR2GRAY)\n",
+    "    faces = face_classifier.detectMultiScale(gray, 1.3, 5)\n",
+    "    if faces is ():\n",
+    "        return (0,0,0,0), np.zeros((48,48), np.uint8), img\n",
+    "    \n",
+    "    allfaces = []   \n",
+    "    rects = []\n",
+    "    for (x,y,w,h) in faces:\n",
+    "        cv2.rectangle(img,(x,y),(x+w,y+h),(255,0,0),2)\n",
+    "        roi_gray = gray[y:y+h, x:x+w]\n",
+    "        roi_gray = cv2.resize(roi_gray, (48, 48), interpolation = cv2.INTER_AREA)\n",
+    "        allfaces.append(roi_gray)\n",
+    "        rects.append((x,w,y,h))\n",
+    "    return rects, allfaces, img\n",
+    "\n",
+    "img = cv2.imread(\"rajeev.jpg\")\n",
+    "rects, faces, image = face_detector(img)\n",
+    "\n",
+    "i = 0\n",
+    "for face in faces:\n",
+    "    roi = face.astype(\"float\") / 255.0\n",
+    "    roi = img_to_array(roi)\n",
+    "    roi = np.expand_dims(roi, axis=0)\n",
+    "\n",
+    "    # make a prediction on the ROI, then lookup the class\n",
+    "    preds = classifier.predict(roi)[0]\n",
+    "    label = class_labels[preds.argmax()]   \n",
+    "\n",
+    "    #Overlay our detected emotion on our pic\n",
+    "    label_position = (rects[i][0] + int((rects[i][1]/2)), abs(rects[i][2] - 10))\n",
+    "    i =+ 1\n",
+    "    cv2.putText(image, label, label_position , cv2.FONT_HERSHEY_SIMPLEX,1, (0,255,0), 2)\n",
+    "    \n",
+    "cv2.imshow(\"Emotion Detector\", image)\n",
+    "cv2.waitKey(0)\n",
+    "\n",
+    "cv2.destroyAllWindows()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Let's try this on our webcam\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 29,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import cv2\n",
+    "import numpy as np\n",
+    "from time import sleep\n",
+    "from tensorflow.keras.preprocessing.image import img_to_array\n",
+    "\n",
+    "face_classifier = cv2.CascadeClassifier('./Haarcascades/haarcascade_frontalface_default.xml')\n",
+    "\n",
+    "def face_detector(img):\n",
+    "    # Convert image to grayscale\n",
+    "    gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)\n",
+    "    faces = face_classifier.detectMultiScale(gray, 1.3, 5)\n",
+    "    if faces is ():\n",
+    "        return (0,0,0,0), np.zeros((48,48), np.uint8), img\n",
+    "    \n",
+    "    for (x,y,w,h) in faces:\n",
+    "        cv2.rectangle(img,(x,y),(x+w,y+h),(255,0,0),2)\n",
+    "        roi_gray = gray[y:y+h, x:x+w]\n",
+    "\n",
+    "    try:\n",
+    "        roi_gray = cv2.resize(roi_gray, (48, 48), interpolation = cv2.INTER_AREA)\n",
+    "    except:\n",
+    "        return (x,w,y,h), np.zeros((48,48), np.uint8), img\n",
+    "    return (x,w,y,h), roi_gray, img\n",
+    "\n",
+    "cap = cv2.VideoCapture(0)\n",
+    "\n",
+    "while True:\n",
+    "\n",
+    "    ret, frame = cap.read()\n",
+    "    rect, face, image = face_detector(frame)\n",
+    "    if np.sum([face]) != 0.0:\n",
+    "        roi = face.astype(\"float\") / 255.0\n",
+    "        roi = img_to_array(roi)\n",
+    "        roi = np.expand_dims(roi, axis=0)\n",
+    "\n",
+    "        # make a prediction on the ROI, then lookup the class\n",
+    "        preds = classifier.predict(roi)[0]\n",
+    "        label = class_labels[preds.argmax()]  \n",
+    "        label_position = (rect[0] + int((rect[1]/2)), rect[2] + 25)\n",
+    "        cv2.putText(image, label, label_position , cv2.FONT_HERSHEY_SIMPLEX,2, (0,255,0), 3)\n",
+    "    else:\n",
+    "        cv2.putText(image, \"No Face Found\", (20, 60) , cv2.FONT_HERSHEY_SIMPLEX,2, (0,255,0), 3)\n",
+    "        \n",
+    "    cv2.imshow('All', image)\n",
+    "    if cv2.waitKey(1) == 13: #13 is the Enter Key\n",
+    "        break\n",
+    "        \n",
+    "cap.release()\n",
+    "cv2.destroyAllWindows()      "
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.7.4"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}

18 . Deep Survaliance - Build a Face Detector with Emotion, Age and Gender Recognition/18.3A - Age, Gender Detection.ipynb

@@ -0,0 +1,174 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Let's run our Age and Gender Detector\n",
+    "\n",
+    "- Please see https://github.com/yu4u/age-gender-estimation for source code project.\n",
+    "- In this notebook we re-use the model trained by yu4u"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Downloading data from https://github.com/yu4u/age-gender-estimation/releases/download/v0.5/weights.28-3.73.hdf5\n",
+      "195854336/195848088 [==============================] - 173s 1us/step\n"
+     ]
+    }
+   ],
+   "source": [
+    "from pathlib import Path\n",
+    "import cv2\n",
+    "import dlib\n",
+    "import sys\n",
+    "import numpy as np\n",
+    "import argparse\n",
+    "from contextlib import contextmanager\n",
+    "from wide_resnet import WideResNet\n",
+    "from tensorflow.keras.utils import get_file\n",
+    "\n",
+    "# Load our cassade classifier for faces\n",
+    "face_classifier = cv2.CascadeClassifier('./Haarcascades/haarcascade_frontalface_default.xml')\n",
+    "\n",
+    "# Load our pretrained model for Gender and Age Detection\n",
+    "pretrained_model = \"https://github.com/yu4u/age-gender-estimation/releases/download/v0.5/weights.28-3.73.hdf5\"\n",
+    "modhash = 'fbe63257a054c1c5466cfd7bf14646d6'\n",
+    "\n",
+    "# Face Detection function\n",
+    "def face_detector(img):\n",
+    "    # Convert image to grayscale for faster detection\n",
+    "    gray = cv2.cvtColor(img.copy(),cv2.COLOR_BGR2GRAY)\n",
+    "    faces = face_classifier.detectMultiScale(gray, 1.3, 5)\n",
+    "    if faces is ():\n",
+    "        return False ,(0,0,0,0), np.zeros((1,48,48,3), np.uint8), img\n",
+    "    \n",
+    "    allfaces = []   \n",
+    "    rects = []\n",
+    "    for (x,y,w,h) in faces:\n",
+    "        cv2.rectangle(img,(x,y),(x+w,y+h),(255,0,0),2)\n",
+    "        roi = img[y:y+h, x:x+w]\n",
+    "        roi_groiray = cv2.resize(roi, (64, 64), interpolation = cv2.INTER_AREA)\n",
+    "        allfaces.append(roi)\n",
+    "        rects.append((x,w,y,h))\n",
+    "    return True, rects, allfaces, img\n",
+    "\n",
+    "# Define our model parameters\n",
+    "depth = 16\n",
+    "k = 8\n",
+    "weight_file = None\n",
+    "margin = 0.4\n",
+    "image_dir = None\n",
+    "\n",
+    "# Get our weight file \n",
+    "if not weight_file:\n",
+    "    weight_file = get_file(\"weights.28-3.73.hdf5\", pretrained_model, cache_subdir=\"pretrained_models\",\n",
+    "                           file_hash=modhash, cache_dir=Path(sys.argv[0]).resolve().parent)\n",
+    "\n",
+    "# load model and weights\n",
+    "img_size = 64\n",
+    "model = WideResNet(img_size, depth=depth, k=k)()\n",
+    "model.load_weights(weight_file)\n",
+    "\n",
+    "# Initialize Webcam\n",
+    "cap = cv2.VideoCapture(0)\n",
+    "\n",
+    "while True:\n",
+    "    ret, frame = cap.read()\n",
+    "    ret, rects, faces, image = face_detector(frame)\n",
+    "    preprocessed_faces = []\n",
+    "    i = 0\n",
+    "    if ret:\n",
+    "        for (i,face) in enumerate(faces):\n",
+    "            face = cv2.resize(face, (64, 64), interpolation = cv2.INTER_AREA)\n",
+    "            preprocessed_faces.append(face)\n",
+    "\n",
+    "        # make a prediction on the faces detected\n",
+    "        results = model.predict(np.array(preprocessed_faces))\n",
+    "        predicted_genders = results[0]\n",
+    "        ages = np.arange(0, 101).reshape(101, 1)\n",
+    "        predicted_ages = results[1].dot(ages).flatten()\n",
+    "\n",
+    "        # draw results\n",
+    "        for (i, f) in enumerate(faces):\n",
+    "            label = \"{}, {}\".format(int(predicted_ages[i]),\n",
+    "                                        \"F\" if predicted_genders[i][0] > 0.5 else \"M\")\n",
+    "\n",
+    "        #Overlay our detected emotion on our pic\n",
+    "        label_position = (rects[i][0] + int((rects[i][1]/2)), abs(rects[i][2] - 10))\n",
+    "        i =+ 1\n",
+    "        cv2.putText(image, label, label_position , cv2.FONT_HERSHEY_SIMPLEX,1, (0,255,0), 2)\n",
+    "\n",
+    "    cv2.imshow(\"Emotion Detector\", image)\n",
+    "    if cv2.waitKey(1) == 13: #13 is the Enter Key\n",
+    "        break\n",
+    "\n",
+    "cap.release()\n",
+    "cv2.destroyAllWindows()      "
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Note if you get the following error, you need to enable your webcam \n",
+    "<img src=\"error.jpg\">\n",
+    "### Enable your webcam by doing the following:\n",
+    "<img src=\"webcam.jpg\">"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Run these lines if your webcam fails to be realesed due to error in code\n",
+    "cap.release()\n",
+    "cv2.destroyAllWindows()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.7.4"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}

18 . Deep Survaliance - Build a Face Detector with Emotion, Age and Gender Recognition/18.3B Age, Gender with Emotion.ipynb

@@ -0,0 +1,526 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Age, Gender and Emotion Detection\n",
+    "\n",
+    "### Let's load our classfiers"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from pathlib import Path\n",
+    "import cv2\n",
+    "import dlib\n",
+    "import sys\n",
+    "import numpy as np\n",
+    "import argparse\n",
+    "from contextlib import contextmanager\n",
+    "from wide_resnet import WideResNet\n",
+    "from tensorflow.keras.utils import get_file\n",
+    "from tensorflow.keras.models import load_model\n",
+    "from tensorflow.keras.preprocessing.image import img_to_array\n",
+    "\n",
+    "classifier = load_model('emotion_little_vgg.h5')\n",
+    "face_classifier = cv2.CascadeClassifier('./Haarcascades/haarcascade_frontalface_default.xml')\n",
+    "pretrained_model = \"https://github.com/yu4u/age-gender-estimation/releases/download/v0.5/weights.28-3.73.hdf5\""
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Testing our Emotion, Age and Gender Detector - Using Webcam"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "modhash = 'fbe63257a054c1c5466cfd7bf14646d6'\n",
+    "emotion_classes = {0: 'Angry', 1: 'Fear', 2: 'Happy', 3: 'Neutral', 4: 'Sad', 5: 'Surprise'}\n",
+    "\n",
+    "def face_detector(img):\n",
+    "    # Convert image to grayscale for faster detection\n",
+    "    gray = cv2.cvtColor(img.copy(),cv2.COLOR_BGR2GRAY)\n",
+    "    faces = face_classifier.detectMultiScale(gray, 1.3, 5)\n",
+    "    if faces is ():\n",
+    "        return False ,(0,0,0,0), np.zeros((1,48,48,3), np.uint8), img\n",
+    "    \n",
+    "    allfaces = []   \n",
+    "    rects = []\n",
+    "    for (x,y,w,h) in faces:\n",
+    "        cv2.rectangle(img,(x,y),(x+w,y+h),(255,0,0),2)\n",
+    "        roi = img[y:y+h, x:x+w]\n",
+    "        allfaces.append(roi)\n",
+    "        rects.append((x,w,y,h))\n",
+    "    return True, rects, allfaces, img\n",
+    "\n",
+    "# Define our model parameters\n",
+    "depth = 16\n",
+    "k = 8\n",
+    "weight_file = None\n",
+    "margin = 0.4\n",
+    "image_dir = None\n",
+    "\n",
+    "# Get our weight file \n",
+    "if not weight_file:\n",
+    "    weight_file = get_file(\"weights.28-3.73.hdf5\", pretrained_model, cache_subdir=\"pretrained_models\",\n",
+    "                           file_hash=modhash, cache_dir=Path(sys.argv[0]).resolve().parent)\n",
+    "\n",
+    "# load model and weights\n",
+    "img_size = 64\n",
+    "model = WideResNet(img_size, depth=depth, k=k)()\n",
+    "model.load_weights(weight_file)\n",
+    "\n",
+    "# Initialize Webcam\n",
+    "cap = cv2.VideoCapture(0)\n",
+    "\n",
+    "while True:\n",
+    "    ret, frame = cap.read()\n",
+    "    ret, rects, faces, image = face_detector(frame)\n",
+    "    preprocessed_faces_ag = []\n",
+    "    preprocessed_faces_emo = []\n",
+    "    \n",
+    "    if ret:\n",
+    "        for (i,face) in enumerate(faces):\n",
+    "            face_ag = cv2.resize(face, (64, 64), interpolation = cv2.INTER_AREA)\n",
+    "            preprocessed_faces_ag.append(face_ag)\n",
+    "\n",
+    "            face_gray_emo = cv2.cvtColor(face, cv2.COLOR_BGR2GRAY)\n",
+    "            face_gray_emo = cv2.resize(face_gray_emo, (48, 48), interpolation = cv2.INTER_AREA)\n",
+    "            face_gray_emo = face_gray_emo.astype(\"float\") / 255.0\n",
+    "            face_gray_emo = img_to_array(face_gray_emo)\n",
+    "            face_gray_emo = np.expand_dims(face_gray_emo, axis=0)\n",
+    "            preprocessed_faces_emo.append(face_gray_emo)\n",
+    "            \n",
+    "        # make a prediction for Age and Gender\n",
+    "        results = model.predict(np.array(preprocessed_faces_ag))\n",
+    "        predicted_genders = results[0]\n",
+    "        ages = np.arange(0, 101).reshape(101, 1)\n",
+    "        predicted_ages = results[1].dot(ages).flatten()\n",
+    "\n",
+    "        # make a prediction for Emotion \n",
+    "        emo_labels = []\n",
+    "        for (i, face) in enumerate(faces):\n",
+    "            preds = classifier.predict(preprocessed_faces_emo[i])[0]\n",
+    "            emo_labels.append(emotion_classes[preds.argmax()])\n",
+    "        \n",
+    "        # draw results, for Age and Gender\n",
+    "        for (i, face) in enumerate(faces):\n",
+    "            label = \"{}, {}, {}\".format(int(predicted_ages[i]),\n",
+    "                                        \"F\" if predicted_genders[i][0] > 0.6 else \"M\",\n",
+    "                                        emo_labels[i])\n",
+    "            \n",
+    "        #Overlay our detected emotion on our pic\n",
+    "        for (i, face) in enumerate(faces):\n",
+    "            label_position = (rects[i][0] + int((rects[i][1]/2)), abs(rects[i][2] - 10))\n",
+    "            cv2.putText(image, label, label_position , cv2.FONT_HERSHEY_PLAIN,1, (0,255,0), 2)\n",
+    "\n",
+    "    cv2.imshow(\"Emotion Detector\", image)\n",
+    "    if cv2.waitKey(1) == 13: #13 is the Enter Key\n",
+    "        break\n",
+    "\n",
+    "cap.release()\n",
+    "cv2.destroyAllWindows()      "
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "cap.release()\n",
+    "cv2.destroyAllWindows()      "
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Testing our Emotion, Age and Gender Detector - On Images"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "metadata": {},
+   "outputs": [
+    {
+     "ename": "FileNotFoundError",
+     "evalue": "[WinError 3] The system cannot find the path specified: './images/'",
+     "output_type": "error",
+     "traceback": [
+      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
+      "\u001b[1;31mFileNotFoundError\u001b[0m                         Traceback (most recent call last)",
+      "\u001b[1;32m<ipython-input-5-d6f5a6d6ebc5>\u001b[0m in \u001b[0;36m<module>\u001b[1;34m\u001b[0m\n\u001b[0;32m     42\u001b[0m \u001b[0mmodel\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mload_weights\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mweight_file\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m     43\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m---> 44\u001b[1;33m \u001b[0mimage_names\u001b[0m \u001b[1;33m=\u001b[0m \u001b[1;33m[\u001b[0m\u001b[0mf\u001b[0m \u001b[1;32mfor\u001b[0m \u001b[0mf\u001b[0m \u001b[1;32min\u001b[0m \u001b[0mlistdir\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mimage_path\u001b[0m\u001b[1;33m)\u001b[0m \u001b[1;32mif\u001b[0m \u001b[0misfile\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mjoin\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mimage_path\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mf\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m]\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m     45\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m     46\u001b[0m \u001b[1;32mfor\u001b[0m \u001b[0mimage_name\u001b[0m \u001b[1;32min\u001b[0m \u001b[0mimage_names\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
+      "\u001b[1;31mFileNotFoundError\u001b[0m: [WinError 3] The system cannot find the path specified: './images/'"
+     ]
+    }
+   ],
+   "source": [
+    "from os import listdir\n",
+    "from os.path import isfile, join\n",
+    "import os\n",
+    "import cv2\n",
+    "\n",
+    "# Define Image Path Here\n",
+    "image_path = \"./images/\"\n",
+    "\n",
+    "modhash = 'fbe63257a054c1c5466cfd7bf14646d6'\n",
+    "emotion_classes = {0: 'Angry', 1: 'Fear', 2: 'Happy', 3: 'Neutral', 4: 'Sad', 5: 'Surprise'}\n",
+    "\n",
+    "def face_detector(img):\n",
+    "    # Convert image to grayscale for faster detection\n",
+    "    gray = cv2.cvtColor(img.copy(),cv2.COLOR_BGR2GRAY)\n",
+    "    faces = face_classifier.detectMultiScale(gray, 1.3, 5)\n",
+    "    if faces is ():\n",
+    "        return False ,(0,0,0,0), np.zeros((1,48,48,3), np.uint8), img\n",
+    "    \n",
+    "    allfaces = []   \n",
+    "    rects = []\n",
+    "    for (x,y,w,h) in faces:\n",
+    "        cv2.rectangle(img,(x,y),(x+w,y+h),(255,0,0),2)\n",
+    "        roi = img[y:y+h, x:x+w]\n",
+    "        allfaces.append(roi)\n",
+    "        rects.append((x,w,y,h))\n",
+    "    return True, rects, allfaces, img\n",
+    "\n",
+    "# Define our model parameters\n",
+    "depth = 16\n",
+    "k = 8\n",
+    "weight_file = None\n",
+    "margin = 0.4\n",
+    "image_dir = None\n",
+    "\n",
+    "# Get our weight file \n",
+    "if not weight_file:\n",
+    "    weight_file = get_file(\"weights.28-3.73.hdf5\", pretrained_model, cache_subdir=\"pretrained_models\",\n",
+    "                           file_hash=modhash, cache_dir=Path(sys.argv[0]).resolve().parent)\n",
+    "# load model and weights\n",
+    "img_size = 64\n",
+    "model = WideResNet(img_size, depth=depth, k=k)()\n",
+    "model.load_weights(weight_file)\n",
+    "\n",
+    "image_names = [f for f in listdir(image_path) if isfile(join(image_path, f))]\n",
+    "\n",
+    "for image_name in image_names:\n",
+    "    frame = cv2.imread(\"./images/\" + image_name)\n",
+    "    ret, rects, faces, image = face_detector(frame)\n",
+    "    preprocessed_faces_ag = []\n",
+    "    preprocessed_faces_emo = []\n",
+    "    \n",
+    "    if ret:\n",
+    "        for (i,face) in enumerate(faces):\n",
+    "            face_ag = cv2.resize(face, (64, 64), interpolation = cv2.INTER_AREA)\n",
+    "            preprocessed_faces_ag.append(face_ag)\n",
+    "\n",
+    "            face_gray_emo = cv2.cvtColor(face, cv2.COLOR_BGR2GRAY)\n",
+    "            face_gray_emo = cv2.resize(face_gray_emo, (48, 48), interpolation = cv2.INTER_AREA)\n",
+    "            face_gray_emo = face_gray_emo.astype(\"float\") / 255.0\n",
+    "            face_gray_emo = img_to_array(face_gray_emo)\n",
+    "            face_gray_emo = np.expand_dims(face_gray_emo, axis=0)\n",
+    "            preprocessed_faces_emo.append(face_gray_emo)\n",
+    "            \n",
+    "        # make a prediction for Age and Gender\n",
+    "        results = model.predict(np.array(preprocessed_faces_ag))\n",
+    "        predicted_genders = results[0]\n",
+    "        ages = np.arange(0, 101).reshape(101, 1)\n",
+    "        predicted_ages = results[1].dot(ages).flatten()\n",
+    "\n",
+    "        # make a prediction for Emotion \n",
+    "        emo_labels = []\n",
+    "        for (i, face) in enumerate(faces):\n",
+    "            preds = classifier.predict(preprocessed_faces_emo[i])[0]\n",
+    "            emo_labels.append(emotion_classes[preds.argmax()])\n",
+    "        \n",
+    "        # draw results, for Age and Gender\n",
+    "        for (i, face) in enumerate(faces):\n",
+    "            label = \"{}, {}, {}\".format(int(predicted_ages[i]),\n",
+    "                                        \"F\" if predicted_genders[i][0] > 0.4 else \"M\",\n",
+    "                                        emo_labels[i])\n",
+    "            \n",
+    "        #Overlay our detected emotion on our pic\n",
+    "        for (i, face) in enumerate(faces):\n",
+    "            label_position = (rects[i][0] + int((rects[i][1]/2)), abs(rects[i][2] - 10))\n",
+    "            cv2.putText(image, label, label_position , cv2.FONT_HERSHEY_PLAIN,1, (0,255,0), 2)\n",
+    "\n",
+    "    cv2.imshow(\"Emotion Detector\", image)\n",
+    "    cv2.waitKey(0)\n",
+    "\n",
+    "cv2.destroyAllWindows()      "
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Using Dlib's Face Detection"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 7,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from os import listdir\n",
+    "from os.path import isfile, join\n",
+    "import os\n",
+    "import cv2\n",
+    "\n",
+    "# Define Image Path Here\n",
+    "image_path = \"./images/\"\n",
+    "\n",
+    "modhash = 'fbe63257a054c1c5466cfd7bf14646d6'\n",
+    "emotion_classes = {0: 'Angry', 1: 'Fear', 2: 'Happy', 3: 'Neutral', 4: 'Sad', 5: 'Surprise'}\n",
+    "\n",
+    "def draw_label(image, point, label, font=cv2.FONT_HERSHEY_SIMPLEX,\n",
+    "               font_scale=0.8, thickness=1):\n",
+    "    size = cv2.getTextSize(label, font, font_scale, thickness)[0]\n",
+    "    x, y = point\n",
+    "    cv2.rectangle(image, (x, y - size[1]), (x + size[0], y), (255, 0, 0), cv2.FILLED)\n",
+    "    cv2.putText(image, label, point, font, font_scale, (255, 255, 255), thickness, lineType=cv2.LINE_AA)\n",
+    "    \n",
+    "\n",
+    "# Define our model parameters\n",
+    "depth = 16\n",
+    "k = 8\n",
+    "weight_file = None\n",
+    "margin = 0.4\n",
+    "image_dir = None\n",
+    "\n",
+    "# Get our weight file \n",
+    "if not weight_file:\n",
+    "    weight_file = get_file(\"weights.28-3.73.hdf5\", pretrained_model, cache_subdir=\"pretrained_models\",\n",
+    "                           file_hash=modhash, cache_dir=Path(sys.argv[0]).resolve().parent)\n",
+    "# load model and weights\n",
+    "img_size = 64\n",
+    "model = WideResNet(img_size, depth=depth, k=k)()\n",
+    "model.load_weights(weight_file)\n",
+    "\n",
+    "detector = dlib.get_frontal_face_detector()\n",
+    "\n",
+    "image_names = [f for f in listdir(image_path) if isfile(join(image_path, f))]\n",
+    "\n",
+    "for image_name in image_names:\n",
+    "    frame = cv2.imread(\"./images/\" + image_name)\n",
+    "    preprocessed_faces_emo = []           \n",
+    " \n",
+    "    input_img = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)\n",
+    "    img_h, img_w, _ = np.shape(input_img)\n",
+    "    detected = detector(frame, 1)\n",
+    "    faces = np.empty((len(detected), img_size, img_size, 3))\n",
+    "    \n",
+    "    preprocessed_faces_emo = []\n",
+    "    if len(detected) > 0:\n",
+    "        for i, d in enumerate(detected):\n",
+    "            x1, y1, x2, y2, w, h = d.left(), d.top(), d.right() + 1, d.bottom() + 1, d.width(), d.height()\n",
+    "            xw1 = max(int(x1 - margin * w), 0)\n",
+    "            yw1 = max(int(y1 - margin * h), 0)\n",
+    "            xw2 = min(int(x2 + margin * w), img_w - 1)\n",
+    "            yw2 = min(int(y2 + margin * h), img_h - 1)\n",
+    "            cv2.rectangle(frame, (x1, y1), (x2, y2), (255, 0, 0), 2)\n",
+    "            # cv2.rectangle(img, (xw1, yw1), (xw2, yw2), (255, 0, 0), 2)\n",
+    "            faces[i, :, :, :] = cv2.resize(frame[yw1:yw2 + 1, xw1:xw2 + 1, :], (img_size, img_size))\n",
+    "            face =  frame[yw1:yw2 + 1, xw1:xw2 + 1, :]\n",
+    "            face_gray_emo = cv2.cvtColor(face, cv2.COLOR_BGR2GRAY)\n",
+    "            face_gray_emo = cv2.resize(face_gray_emo, (48, 48), interpolation = cv2.INTER_AREA)\n",
+    "            face_gray_emo = face_gray_emo.astype(\"float\") / 255.0\n",
+    "            face_gray_emo = img_to_array(face_gray_emo)\n",
+    "            face_gray_emo = np.expand_dims(face_gray_emo, axis=0)\n",
+    "            preprocessed_faces_emo.append(face_gray_emo)\n",
+    "\n",
+    "        # make a prediction for Age and Gender\n",
+    "        results = model.predict(np.array(faces))\n",
+    "        predicted_genders = results[0]\n",
+    "        ages = np.arange(0, 101).reshape(101, 1)\n",
+    "        predicted_ages = results[1].dot(ages).flatten()\n",
+    "\n",
+    "        # make a prediction for Emotion \n",
+    "        emo_labels = []\n",
+    "        for i, d in enumerate(detected):\n",
+    "            preds = classifier.predict(preprocessed_faces_emo[i])[0]\n",
+    "            emo_labels.append(emotion_classes[preds.argmax()])\n",
+    "        \n",
+    "        # draw results\n",
+    "        for i, d in enumerate(detected):\n",
+    "            label = \"{}, {}, {}\".format(int(predicted_ages[i]),\n",
+    "                                        \"F\" if predicted_genders[i][0] > 0.4 else \"M\", emo_labels[i])\n",
+    "            draw_label(frame, (d.left(), d.top()), label)\n",
+    "\n",
+    "    cv2.imshow(\"Emotion Detector\", frame)\n",
+    "    cv2.waitKey(0)\n",
+    "\n",
+    "cv2.destroyAllWindows()      "
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### And now using dlib's detector with our webcam"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 8,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from os import listdir\n",
+    "from os.path import isfile, join\n",
+    "import os\n",
+    "import cv2\n",
+    "\n",
+    "# Define Image Path Here\n",
+    "image_path = \"./images/\"\n",
+    "\n",
+    "modhash = 'fbe63257a054c1c5466cfd7bf14646d6'\n",
+    "emotion_classes = {0: 'Angry', 1: 'Fear', 2: 'Happy', 3: 'Neutral', 4: 'Sad', 5: 'Surprise'}\n",
+    "\n",
+    "def draw_label(image, point, label, font=cv2.FONT_HERSHEY_SIMPLEX,\n",
+    "               font_scale=0.8, thickness=1):\n",
+    "    size = cv2.getTextSize(label, font, font_scale, thickness)[0]\n",
+    "    x, y = point\n",
+    "    cv2.rectangle(image, (x, y - size[1]), (x + size[0], y), (255, 0, 0), cv2.FILLED)\n",
+    "    cv2.putText(image, label, point, font, font_scale, (255, 255, 255), thickness, lineType=cv2.LINE_AA)\n",
+    "    \n",
+    "\n",
+    "# Define our model parameters\n",
+    "depth = 16\n",
+    "k = 8\n",
+    "weight_file = None\n",
+    "margin = 0.4\n",
+    "image_dir = None\n",
+    "\n",
+    "# Get our weight file \n",
+    "if not weight_file:\n",
+    "    weight_file = get_file(\"weights.28-3.73.hdf5\", pretrained_model, cache_subdir=\"pretrained_models\",\n",
+    "                           file_hash=modhash, cache_dir=Path(sys.argv[0]).resolve().parent)\n",
+    "# load model and weights\n",
+    "img_size = 64\n",
+    "model = WideResNet(img_size, depth=depth, k=k)()\n",
+    "model.load_weights(weight_file)\n",
+    "\n",
+    "detector = dlib.get_frontal_face_detector()\n",
+    "\n",
+    "# Initialize Webcam\n",
+    "cap = cv2.VideoCapture(0)\n",
+    "\n",
+    "while True:\n",
+    "    ret, frame = cap.read()\n",
+    "    preprocessed_faces_emo = []           \n",
+    " \n",
+    "    input_img = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)\n",
+    "    img_h, img_w, _ = np.shape(input_img)\n",
+    "    detected = detector(frame, 1)\n",
+    "    faces = np.empty((len(detected), img_size, img_size, 3))\n",
+    "    \n",
+    "    preprocessed_faces_emo = []\n",
+    "    if len(detected) > 0:\n",
+    "        for i, d in enumerate(detected):\n",
+    "            x1, y1, x2, y2, w, h = d.left(), d.top(), d.right() + 1, d.bottom() + 1, d.width(), d.height()\n",
+    "            xw1 = max(int(x1 - margin * w), 0)\n",
+    "            yw1 = max(int(y1 - margin * h), 0)\n",
+    "            xw2 = min(int(x2 + margin * w), img_w - 1)\n",
+    "            yw2 = min(int(y2 + margin * h), img_h - 1)\n",
+    "            cv2.rectangle(frame, (x1, y1), (x2, y2), (255, 0, 0), 2)\n",
+    "            # cv2.rectangle(img, (xw1, yw1), (xw2, yw2), (255, 0, 0), 2)\n",
+    "            faces[i, :, :, :] = cv2.resize(frame[yw1:yw2 + 1, xw1:xw2 + 1, :], (img_size, img_size))\n",
+    "            face =  frame[yw1:yw2 + 1, xw1:xw2 + 1, :]\n",
+    "            face_gray_emo = cv2.cvtColor(face, cv2.COLOR_BGR2GRAY)\n",
+    "            face_gray_emo = cv2.resize(face_gray_emo, (48, 48), interpolation = cv2.INTER_AREA)\n",
+    "            face_gray_emo = face_gray_emo.astype(\"float\") / 255.0\n",
+    "            face_gray_emo = img_to_array(face_gray_emo)\n",
+    "            face_gray_emo = np.expand_dims(face_gray_emo, axis=0)\n",
+    "            preprocessed_faces_emo.append(face_gray_emo)\n",
+    "\n",
+    "        # make a prediction for Age and Gender\n",
+    "        results = model.predict(np.array(faces))\n",
+    "        predicted_genders = results[0]\n",
+    "        ages = np.arange(0, 101).reshape(101, 1)\n",
+    "        predicted_ages = results[1].dot(ages).flatten()\n",
+    "\n",
+    "        # make a prediction for Emotion \n",
+    "        emo_labels = []\n",
+    "        for i, d in enumerate(detected):\n",
+    "            preds = classifier.predict(preprocessed_faces_emo[i])[0]\n",
+    "            emo_labels.append(emotion_classes[preds.argmax()])\n",
+    "        \n",
+    "        # draw results\n",
+    "        for i, d in enumerate(detected):\n",
+    "            label = \"{}, {}, {}\".format(int(predicted_ages[i]),\n",
+    "                                        \"F\" if predicted_genders[i][0] > 0.4 else \"M\", emo_labels[i])\n",
+    "            draw_label(frame, (d.left(), d.top()), label)\n",
+    "\n",
+    "    cv2.imshow(\"Emotion Detector\", frame)\n",
+    "    if cv2.waitKey(1) == 13: #13 is the Enter Key\n",
+    "        break\n",
+    "\n",
+    "cap.release()\n",
+    "cv2.destroyAllWindows()      "
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.7.4"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}

18 . Deep Survaliance - Build a Face Detector with Emotion, Age and Gender Recognition/Face Detection - Friends Characters.ipynb

@@ -0,0 +1,526 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Basic Deep Learning Face Recogntion\n",
+    "## Building a Friends TV Show Character Identifier"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Let's train our model\n",
+    "I've created a dataset with the faces of 4 Friends characters taken from a handful of different scenes."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 33,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Found 2663 images belonging to 4 classes.\n",
+      "Found 955 images belonging to 4 classes.\n"
+     ]
+    }
+   ],
+   "source": [
+    "from __future__ import print_function\n",
+    "import keras\n",
+    "from keras.preprocessing.image import ImageDataGenerator\n",
+    "from keras.models import Sequential\n",
+    "from keras.layers import Dense, Dropout, Activation, Flatten, BatchNormalization\n",
+    "from keras.layers import Conv2D, MaxPooling2D\n",
+    "from keras.preprocessing.image import ImageDataGenerator\n",
+    "import os\n",
+    "\n",
+    "num_classes = 4\n",
+    "img_rows, img_cols = 48, 48\n",
+    "batch_size = 16\n",
+    "\n",
+    "train_data_dir = './faces/train'\n",
+    "validation_data_dir = './faces/validation'\n",
+    "\n",
+    "# Let's use some data augmentaiton \n",
+    "train_datagen = ImageDataGenerator(\n",
+    "      rescale=1./255,\n",
+    "      rotation_range=30,\n",
+    "      shear_range=0.3,\n",
+    "      zoom_range=0.3,\n",
+    "      width_shift_range=0.4,\n",
+    "      height_shift_range=0.4,\n",
+    "      horizontal_flip=True,\n",
+    "      fill_mode='nearest')\n",
+    " \n",
+    "validation_datagen = ImageDataGenerator(rescale=1./255)\n",
+    " \n",
+    "train_generator = train_datagen.flow_from_directory(\n",
+    "        train_data_dir,\n",
+    "        target_size=(img_rows, img_cols),\n",
+    "        batch_size=batch_size,\n",
+    "        class_mode='categorical',\n",
+    "        shuffle=True)\n",
+    " \n",
+    "validation_generator = validation_datagen.flow_from_directory(\n",
+    "        validation_data_dir,\n",
+    "        target_size=(img_rows, img_cols),\n",
+    "        batch_size=batch_size,\n",
+    "        class_mode='categorical',\n",
+    "        shuffle=True)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 37,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "#Our Keras imports\n",
+    "from keras.models import Sequential\n",
+    "from keras.layers.normalization import BatchNormalization\n",
+    "from keras.layers.convolutional import Conv2D, MaxPooling2D\n",
+    "from keras.layers.advanced_activations import ELU\n",
+    "from keras.layers.core import Activation, Flatten, Dropout, Dense"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Creating a simple VGG based model for Face Recognition"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 35,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "_________________________________________________________________\n",
+      "Layer (type)                 Output Shape              Param #   \n",
+      "=================================================================\n",
+      "conv2d_25 (Conv2D)           (None, 48, 48, 32)        896       \n",
+      "_________________________________________________________________\n",
+      "activation_34 (Activation)   (None, 48, 48, 32)        0         \n",
+      "_________________________________________________________________\n",
+      "batch_normalization_31 (Batc (None, 48, 48, 32)        128       \n",
+      "_________________________________________________________________\n",
+      "conv2d_26 (Conv2D)           (None, 48, 48, 32)        9248      \n",
+      "_________________________________________________________________\n",
+      "activation_35 (Activation)   (None, 48, 48, 32)        0         \n",
+      "_________________________________________________________________\n",
+      "batch_normalization_32 (Batc (None, 48, 48, 32)        128       \n",
+      "_________________________________________________________________\n",
+      "max_pooling2d_13 (MaxPooling (None, 24, 24, 32)        0         \n",
+      "_________________________________________________________________\n",
+      "dropout_19 (Dropout)         (None, 24, 24, 32)        0         \n",
+      "_________________________________________________________________\n",
+      "conv2d_27 (Conv2D)           (None, 24, 24, 64)        18496     \n",
+      "_________________________________________________________________\n",
+      "activation_36 (Activation)   (None, 24, 24, 64)        0         \n",
+      "_________________________________________________________________\n",
+      "batch_normalization_33 (Batc (None, 24, 24, 64)        256       \n",
+      "_________________________________________________________________\n",
+      "conv2d_28 (Conv2D)           (None, 24, 24, 64)        36928     \n",
+      "_________________________________________________________________\n",
+      "activation_37 (Activation)   (None, 24, 24, 64)        0         \n",
+      "_________________________________________________________________\n",
+      "batch_normalization_34 (Batc (None, 24, 24, 64)        256       \n",
+      "_________________________________________________________________\n",
+      "max_pooling2d_14 (MaxPooling (None, 12, 12, 64)        0         \n",
+      "_________________________________________________________________\n",
+      "dropout_20 (Dropout)         (None, 12, 12, 64)        0         \n",
+      "_________________________________________________________________\n",
+      "conv2d_29 (Conv2D)           (None, 12, 12, 128)       73856     \n",
+      "_________________________________________________________________\n",
+      "activation_38 (Activation)   (None, 12, 12, 128)       0         \n",
+      "_________________________________________________________________\n",
+      "batch_normalization_35 (Batc (None, 12, 12, 128)       512       \n",
+      "_________________________________________________________________\n",
+      "conv2d_30 (Conv2D)           (None, 12, 12, 128)       147584    \n",
+      "_________________________________________________________________\n",
+      "activation_39 (Activation)   (None, 12, 12, 128)       0         \n",
+      "_________________________________________________________________\n",
+      "batch_normalization_36 (Batc (None, 12, 12, 128)       512       \n",
+      "_________________________________________________________________\n",
+      "max_pooling2d_15 (MaxPooling (None, 6, 6, 128)         0         \n",
+      "_________________________________________________________________\n",
+      "dropout_21 (Dropout)         (None, 6, 6, 128)         0         \n",
+      "_________________________________________________________________\n",
+      "conv2d_31 (Conv2D)           (None, 6, 6, 256)         295168    \n",
+      "_________________________________________________________________\n",
+      "activation_40 (Activation)   (None, 6, 6, 256)         0         \n",
+      "_________________________________________________________________\n",
+      "batch_normalization_37 (Batc (None, 6, 6, 256)         1024      \n",
+      "_________________________________________________________________\n",
+      "conv2d_32 (Conv2D)           (None, 6, 6, 256)         590080    \n",
+      "_________________________________________________________________\n",
+      "activation_41 (Activation)   (None, 6, 6, 256)         0         \n",
+      "_________________________________________________________________\n",
+      "batch_normalization_38 (Batc (None, 6, 6, 256)         1024      \n",
+      "_________________________________________________________________\n",
+      "max_pooling2d_16 (MaxPooling (None, 3, 3, 256)         0         \n",
+      "_________________________________________________________________\n",
+      "dropout_22 (Dropout)         (None, 3, 3, 256)         0         \n",
+      "_________________________________________________________________\n",
+      "flatten_4 (Flatten)          (None, 2304)              0         \n",
+      "_________________________________________________________________\n",
+      "dense_10 (Dense)             (None, 64)                147520    \n",
+      "_________________________________________________________________\n",
+      "activation_42 (Activation)   (None, 64)                0         \n",
+      "_________________________________________________________________\n",
+      "batch_normalization_39 (Batc (None, 64)                256       \n",
+      "_________________________________________________________________\n",
+      "dropout_23 (Dropout)         (None, 64)                0         \n",
+      "_________________________________________________________________\n",
+      "dense_11 (Dense)             (None, 64)                4160      \n",
+      "_________________________________________________________________\n",
+      "activation_43 (Activation)   (None, 64)                0         \n",
+      "_________________________________________________________________\n",
+      "batch_normalization_40 (Batc (None, 64)                256       \n",
+      "_________________________________________________________________\n",
+      "dropout_24 (Dropout)         (None, 64)                0         \n",
+      "_________________________________________________________________\n",
+      "dense_12 (Dense)             (None, 4)                 260       \n",
+      "_________________________________________________________________\n",
+      "activation_44 (Activation)   (None, 4)                 0         \n",
+      "=================================================================\n",
+      "Total params: 1,328,548\n",
+      "Trainable params: 1,326,372\n",
+      "Non-trainable params: 2,176\n",
+      "_________________________________________________________________\n",
+      "None\n"
+     ]
+    }
+   ],
+   "source": [
+    "model = Sequential()\n",
+    "\n",
+    "model.add(Conv2D(32, (3, 3), padding = 'same', kernel_initializer=\"he_normal\",\n",
+    "                 input_shape = (img_rows, img_cols, 3)))\n",
+    "model.add(Activation('elu'))\n",
+    "model.add(BatchNormalization())\n",
+    "model.add(Conv2D(32, (3, 3), padding = \"same\", kernel_initializer=\"he_normal\", \n",
+    "                 input_shape = (img_rows, img_cols, 3)))\n",
+    "model.add(Activation('elu'))\n",
+    "model.add(BatchNormalization())\n",
+    "model.add(MaxPooling2D(pool_size=(2, 2)))\n",
+    "model.add(Dropout(0.2))\n",
+    "\n",
+    "# Block #2: second CONV => RELU => CONV => RELU => POOL\n",
+    "# layer set\n",
+    "model.add(Conv2D(64, (3, 3), padding=\"same\", kernel_initializer=\"he_normal\"))\n",
+    "model.add(Activation('elu'))\n",
+    "model.add(BatchNormalization())\n",
+    "model.add(Conv2D(64, (3, 3), padding=\"same\", kernel_initializer=\"he_normal\"))\n",
+    "model.add(Activation('elu'))\n",
+    "model.add(BatchNormalization())\n",
+    "model.add(MaxPooling2D(pool_size=(2, 2)))\n",
+    "model.add(Dropout(0.2))\n",
+    "\n",
+    "# Block #3: third CONV => RELU => CONV => RELU => POOL\n",
+    "# layer set\n",
+    "model.add(Conv2D(128, (3, 3), padding=\"same\", kernel_initializer=\"he_normal\"))\n",
+    "model.add(Activation('elu'))\n",
+    "model.add(BatchNormalization())\n",
+    "model.add(Conv2D(128, (3, 3), padding=\"same\", kernel_initializer=\"he_normal\"))\n",
+    "model.add(Activation('elu'))\n",
+    "model.add(BatchNormalization())\n",
+    "model.add(MaxPooling2D(pool_size=(2, 2)))\n",
+    "model.add(Dropout(0.2))\n",
+    "\n",
+    "# Block #4: third CONV => RELU => CONV => RELU => POOL\n",
+    "# layer set\n",
+    "model.add(Conv2D(256, (3, 3), padding=\"same\", kernel_initializer=\"he_normal\"))\n",
+    "model.add(Activation('elu'))\n",
+    "model.add(BatchNormalization())\n",
+    "model.add(Conv2D(256, (3, 3), padding=\"same\", kernel_initializer=\"he_normal\"))\n",
+    "model.add(Activation('elu'))\n",
+    "model.add(BatchNormalization())\n",
+    "model.add(MaxPooling2D(pool_size=(2, 2)))\n",
+    "model.add(Dropout(0.2))\n",
+    "\n",
+    "# Block #5: first set of FC => RELU layers\n",
+    "model.add(Flatten())\n",
+    "model.add(Dense(64, kernel_initializer=\"he_normal\"))\n",
+    "model.add(Activation('elu'))\n",
+    "model.add(BatchNormalization())\n",
+    "model.add(Dropout(0.5))\n",
+    "\n",
+    "# Block #6: second set of FC => RELU layers\n",
+    "model.add(Dense(64, kernel_initializer=\"he_normal\"))\n",
+    "model.add(Activation('elu'))\n",
+    "model.add(BatchNormalization())\n",
+    "model.add(Dropout(0.5))\n",
+    "\n",
+    "# Block #7: softmax classifier\n",
+    "model.add(Dense(num_classes, kernel_initializer=\"he_normal\"))\n",
+    "model.add(Activation(\"softmax\"))\n",
+    "\n",
+    "print(model.summary())"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Training our Model"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 36,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 1/10\n",
+      "166/166 [==============================] - 76s 457ms/step - loss: 1.1153 - acc: 0.5700 - val_loss: 1.4428 - val_acc: 0.4841\n",
+      "\n",
+      "Epoch 00001: val_loss improved from inf to 1.44279, saving model to /home/deeplearningcv/DeepLearningCV/Trained Models/face_recognition_friends_vgg.h5\n",
+      "Epoch 2/10\n",
+      "166/166 [==============================] - 67s 403ms/step - loss: 0.7034 - acc: 0.7343 - val_loss: 3.7705 - val_acc: 0.2705\n",
+      "\n",
+      "Epoch 00002: val_loss did not improve from 1.44279\n",
+      "Epoch 3/10\n",
+      "166/166 [==============================] - 62s 373ms/step - loss: 0.6037 - acc: 0.7690 - val_loss: 0.9403 - val_acc: 0.6912\n",
+      "\n",
+      "Epoch 00003: val_loss improved from 1.44279 to 0.94025, saving model to /home/deeplearningcv/DeepLearningCV/Trained Models/face_recognition_friends_vgg.h5\n",
+      "Epoch 4/10\n",
+      "166/166 [==============================] - 62s 373ms/step - loss: 0.5432 - acc: 0.7988 - val_loss: 1.3018 - val_acc: 0.5548\n",
+      "\n",
+      "Epoch 00004: val_loss did not improve from 0.94025\n",
+      "Epoch 5/10\n",
+      "166/166 [==============================] - 69s 414ms/step - loss: 0.4715 - acc: 0.8301 - val_loss: 3.8879 - val_acc: 0.1534\n",
+      "\n",
+      "Epoch 00005: val_loss did not improve from 0.94025\n",
+      "Epoch 6/10\n",
+      "166/166 [==============================] - 77s 467ms/step - loss: 0.4233 - acc: 0.8524 - val_loss: 0.6878 - val_acc: 0.7093\n",
+      "\n",
+      "Epoch 00006: val_loss improved from 0.94025 to 0.68784, saving model to /home/deeplearningcv/DeepLearningCV/Trained Models/face_recognition_friends_vgg.h5\n",
+      "Epoch 7/10\n",
+      "166/166 [==============================] - 71s 429ms/step - loss: 0.4130 - acc: 0.8636 - val_loss: 3.3402 - val_acc: 0.2971\n",
+      "\n",
+      "Epoch 00007: val_loss did not improve from 0.68784\n",
+      "Epoch 8/10\n",
+      "166/166 [==============================] - 79s 477ms/step - loss: 0.3821 - acc: 0.8748 - val_loss: 2.6729 - val_acc: 0.6283\n",
+      "\n",
+      "Epoch 00008: val_loss did not improve from 0.68784\n",
+      "Epoch 9/10\n",
+      "166/166 [==============================] - 86s 519ms/step - loss: 0.3622 - acc: 0.8709 - val_loss: 1.5067 - val_acc: 0.5197\n",
+      "Restoring model weights from the end of the best epoch\n",
+      "\n",
+      "Epoch 00009: val_loss did not improve from 0.68784\n",
+      "\n",
+      "Epoch 00009: ReduceLROnPlateau reducing learning rate to 0.0019999999552965165.\n",
+      "Epoch 00009: early stopping\n"
+     ]
+    }
+   ],
+   "source": [
+    "from keras.optimizers import RMSprop, SGD, Adam\n",
+    "from keras.callbacks import ModelCheckpoint, EarlyStopping, ReduceLROnPlateau\n",
+    "\n",
+    "                     \n",
+    "checkpoint = ModelCheckpoint(\"/home/deeplearningcv/DeepLearningCV/Trained Models/face_recognition_friends_vgg.h5\",\n",
+    "                             monitor=\"val_loss\",\n",
+    "                             mode=\"min\",\n",
+    "                             save_best_only = True,\n",
+    "                             verbose=1)\n",
+    "\n",
+    "earlystop = EarlyStopping(monitor = 'val_loss', \n",
+    "                          min_delta = 0, \n",
+    "                          patience = 3,\n",
+    "                          verbose = 1,\n",
+    "                          restore_best_weights = True)\n",
+    "\n",
+    "reduce_lr = ReduceLROnPlateau(monitor = 'val_loss', factor = 0.2, patience = 3, verbose = 1, min_delta = 0.0001)\n",
+    "\n",
+    "# we put our call backs into a callback list\n",
+    "callbacks = [earlystop, checkpoint, reduce_lr]\n",
+    "\n",
+    "# We use a very small learning rate \n",
+    "model.compile(loss = 'categorical_crossentropy',\n",
+    "              optimizer = Adam(lr=0.01),\n",
+    "              metrics = ['accuracy'])\n",
+    "\n",
+    "nb_train_samples = 2663\n",
+    "nb_validation_samples = 955\n",
+    "epochs = 10\n",
+    "\n",
+    "history = model.fit_generator(\n",
+    "    train_generator,\n",
+    "    steps_per_epoch = nb_train_samples // batch_size,\n",
+    "    epochs = epochs,\n",
+    "    callbacks = callbacks,\n",
+    "    validation_data = validation_generator,\n",
+    "    validation_steps = nb_validation_samples // batch_size)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "#### Getting our Class Labels"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 39,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "{0: 'Chandler', 1: 'Joey', 2: 'Pheobe', 3: 'Rachel'}"
+      ]
+     },
+     "execution_count": 39,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "class_labels = validation_generator.class_indices\n",
+    "class_labels = {v: k for k, v in class_labels.items()}\n",
+    "classes = list(class_labels.values())\n",
+    "class_labels"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Load our model\n",
+    "from keras.models import load_model\n",
+    "\n",
+    "classifier = load_model('/home/deeplearningcv/DeepLearningCV/Trained Models/face_recognition_friends_vgg.h5')"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Testing our model on some real video"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 43,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from os import listdir\n",
+    "from os.path import isfile, join\n",
+    "import os\n",
+    "import cv2\n",
+    "import numpy as np\n",
+    "\n",
+    "\n",
+    "face_classes = {0: 'Chandler', 1: 'Joey', 2: 'Pheobe', 3: 'Rachel'}\n",
+    "\n",
+    "def draw_label(image, point, label, font=cv2.FONT_HERSHEY_SIMPLEX,\n",
+    "               font_scale=0.8, thickness=1):\n",
+    "    size = cv2.getTextSize(label, font, font_scale, thickness)[0]\n",
+    "    x, y = point\n",
+    "    cv2.rectangle(image, (x, y - size[1]), (x + size[0], y), (255, 0, 0), cv2.FILLED)\n",
+    "    cv2.putText(image, label, point, font, font_scale, (255, 255, 255), thickness, lineType=cv2.LINE_AA)\n",
+    "    \n",
+    "margin = 0.2\n",
+    "# load model and weights\n",
+    "img_size = 64\n",
+    "\n",
+    "detector = dlib.get_frontal_face_detector()\n",
+    "\n",
+    "cap = cv2.VideoCapture('testfriends.mp4')\n",
+    "\n",
+    "while True:\n",
+    "    ret, frame = cap.read()\n",
+    "    frame = cv2.resize(frame, None, fx=0.5, fy=0.5, interpolation = cv2.INTER_LINEAR)\n",
+    "    preprocessed_faces = []           \n",
+    " \n",
+    "    input_img = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)\n",
+    "    img_h, img_w, _ = np.shape(input_img)\n",
+    "    detected = detector(frame, 1)\n",
+    "    faces = np.empty((len(detected), img_size, img_size, 3))\n",
+    "    \n",
+    "    preprocessed_faces_emo = []\n",
+    "    if len(detected) > 0:\n",
+    "        for i, d in enumerate(detected):\n",
+    "            x1, y1, x2, y2, w, h = d.left(), d.top(), d.right() + 1, d.bottom() + 1, d.width(), d.height()\n",
+    "            xw1 = max(int(x1 - margin * w), 0)\n",
+    "            yw1 = max(int(y1 - margin * h), 0)\n",
+    "            xw2 = min(int(x2 + margin * w), img_w - 1)\n",
+    "            yw2 = min(int(y2 + margin * h), img_h - 1)\n",
+    "            cv2.rectangle(frame, (x1, y1), (x2, y2), (255, 0, 0), 2)\n",
+    "            # cv2.rectangle(img, (xw1, yw1), (xw2, yw2), (255, 0, 0), 2)\n",
+    "            #faces[i, :, :, :] = cv2.resize(frame[yw1:yw2 + 1, xw1:xw2 + 1, :], (img_size, img_size))\n",
+    "            face =  frame[yw1:yw2 + 1, xw1:xw2 + 1, :]\n",
+    "            face = cv2.resize(face, (48, 48), interpolation = cv2.INTER_AREA)\n",
+    "            face = face.astype(\"float\") / 255.0\n",
+    "            face = img_to_array(face)\n",
+    "            face = np.expand_dims(face, axis=0)\n",
+    "            preprocessed_faces.append(face)\n",
+    "\n",
+    "        # make a prediction for Emotion \n",
+    "        face_labels = []\n",
+    "        for i, d in enumerate(detected):\n",
+    "            preds = classifier.predict(preprocessed_faces[i])[0]\n",
+    "            face_labels.append(face_classes[preds.argmax()])\n",
+    "        \n",
+    "        # draw results\n",
+    "        for i, d in enumerate(detected):\n",
+    "            label = \"{}\".format(face_labels[i])\n",
+    "            draw_label(frame, (d.left(), d.top()), label)\n",
+    "\n",
+    "    cv2.imshow(\"Friend Character Identifier\", frame)\n",
+    "    if cv2.waitKey(1) == 13: #13 is the Enter Key\n",
+    "        break\n",
+    "\n",
+    "cap.release()\n",
+    "cv2.destroyAllWindows()      "
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.6.6"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}

18 . Deep Survaliance - Build a Face Detector with Emotion, Age and Gender Recognition/Face Extraction from Video.ipynb

@@ -0,0 +1,93 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Extracting the faces from a video"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from os import listdir\n",
+    "from os.path import isfile, join\n",
+    "import os\n",
+    "import cv2\n",
+    "import dlib\n",
+    "import numpy as np\n",
+    "\n",
+    "# Define Image Path Here\n",
+    "image_path = \"./images/\"\n",
+    "\n",
+    "def draw_label(image, point, label, font=cv2.FONT_HERSHEY_SIMPLEX,\n",
+    "               font_scale=0.8, thickness=1):\n",
+    "    size = cv2.getTextSize(label, font, font_scale, thickness)[0]\n",
+    "    x, y = point\n",
+    "    cv2.rectangle(image, (x, y - size[1]), (x + size[0], y), (255, 0, 0), cv2.FILLED)\n",
+    "    cv2.putText(image, label, point, font, font_scale, (255, 255, 255), thickness, lineType=cv2.LINE_AA)\n",
+    "    \n",
+    "detector = dlib.get_frontal_face_detector()\n",
+    "\n",
+    "# Initialize Webcam\n",
+    "cap = cv2.VideoCapture('testfriends.mp4')\n",
+    "img_size = 64\n",
+    "margin = 0.2\n",
+    "frame_count = 0\n",
+    "\n",
+    "while True:\n",
+    "    ret, frame = cap.read()\n",
+    "    frame_count += 1\n",
+    "    print(frame_count)   \n",
+    " \n",
+    "    input_img = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)\n",
+    "    img_h, img_w, _ = np.shape(input_img)\n",
+    "    detected = detector(frame, 1)\n",
+    "    faces = []\n",
+    "    \n",
+    "    if len(detected) > 0:\n",
+    "        for i, d in enumerate(detected):\n",
+    "            x1, y1, x2, y2, w, h = d.left(), d.top(), d.right() + 1, d.bottom() + 1, d.width(), d.height()\n",
+    "            xw1 = max(int(x1 - margin * w), 0)\n",
+    "            yw1 = max(int(y1 - margin * h), 0)\n",
+    "            xw2 = min(int(x2 + margin * w), img_w - 1)\n",
+    "            yw2 = min(int(y2 + margin * h), img_h - 1)\n",
+    "            face =  frame[yw1:yw2 + 1, xw1:xw2 + 1, :]\n",
+    "            file_name = \"./faces/\"+str(frame_count)+\"_\"+str(i)+\".jpg\"\n",
+    "            cv2.imwrite(file_name, face)\n",
+    "            cv2.rectangle(frame, (x1, y1), (x2, y2), (255, 0, 0), 2)\n",
+    "\n",
+    "    cv2.imshow(\"Face Detector\", frame)\n",
+    "    if cv2.waitKey(1) == 13: #13 is the Enter Key\n",
+    "        break\n",
+    "\n",
+    "cap.release()\n",
+    "cv2.destroyAllWindows()      "
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.6.6"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}

18 . Deep Survaliance - Build a Face Detector with Emotion, Age and Gender Recognition/wide_resnet.py

@@ -0,0 +1,152 @@
+# This code is imported from the following project: https://github.com/asmith26/wide_resnets_keras
+
+import logging
+import sys
+import numpy as np
+from tensorflow.keras.models import Model
+from tensorflow.keras.layers import Input, Activation, add, Dense, Flatten, Dropout
+from tensorflow.keras.layers import Conv2D, AveragePooling2D
+from tensorflow.keras.layers import BatchNormalization
+from tensorflow.keras.regularizers import l2
+from tensorflow.keras import backend as K
+
+sys.setrecursionlimit(2 ** 20)
+np.random.seed(2 ** 10)
+
+
+class WideResNet:
+    def __init__(self, image_size, depth=16, k=8):
+        self._depth = depth
+        self._k = k
+        self._dropout_probability = 0
+        self._weight_decay = 0.0005
+        self._use_bias = False
+        self._weight_init = "he_normal"
+
+        if K.image_data_format() == "channels_first":
+            logging.debug("image_dim_ordering = 'th'")
+            self._channel_axis = 1
+            self._input_shape = (3, image_size, image_size)
+        else:
+            logging.debug("image_dim_ordering = 'tf'")
+            self._channel_axis = -1
+            self._input_shape = (image_size, image_size, 3)
+
+    # Wide residual network http://arxiv.org/abs/1605.07146
+    def _wide_basic(self, n_input_plane, n_output_plane, stride):
+        def f(net):
+            # format of conv_params:
+            #               [ [kernel_size=("kernel width", "kernel height"),
+            #               strides="(stride_vertical,stride_horizontal)",
+            #               padding="same" or "valid"] ]
+            # B(3,3): orignal <<basic>> block
+            conv_params = [[3, 3, stride, "same"],
+                           [3, 3, (1, 1), "same"]]
+
+            n_bottleneck_plane = n_output_plane
+
+            # Residual block
+            for i, v in enumerate(conv_params):
+                if i == 0:
+                    if n_input_plane != n_output_plane:
+                        net = BatchNormalization(axis=self._channel_axis)(net)
+                        net = Activation("relu")(net)
+                        convs = net
+                    else:
+                        convs = BatchNormalization(axis=self._channel_axis)(net)
+                        convs = Activation("relu")(convs)
+
+                    convs = Conv2D(n_bottleneck_plane, kernel_size=(v[0], v[1]),
+                                          strides=v[2],
+                                          padding=v[3],
+                                          kernel_initializer=self._weight_init,
+                                          kernel_regularizer=l2(self._weight_decay),
+                                          use_bias=self._use_bias)(convs)
+                else:
+                    convs = BatchNormalization(axis=self._channel_axis)(convs)
+                    convs = Activation("relu")(convs)
+                    if self._dropout_probability > 0:
+                        convs = Dropout(self._dropout_probability)(convs)
+                    convs = Conv2D(n_bottleneck_plane, kernel_size=(v[0], v[1]),
+                                          strides=v[2],
+                                          padding=v[3],
+                                          kernel_initializer=self._weight_init,
+                                          kernel_regularizer=l2(self._weight_decay),
+                                          use_bias=self._use_bias)(convs)
+
+            # Shortcut Connection: identity function or 1x1 convolutional
+            #  (depends on difference between input & output shape - this
+            #   corresponds to whether we are using the first block in each
+            #   group; see _layer() ).
+            if n_input_plane != n_output_plane:
+                shortcut = Conv2D(n_output_plane, kernel_size=(1, 1),
+                                         strides=stride,
+                                         padding="same",
+                                         kernel_initializer=self._weight_init,
+                                         kernel_regularizer=l2(self._weight_decay),
+                                         use_bias=self._use_bias)(net)
+            else:
+                shortcut = net
+
+            return add([convs, shortcut])
+
+        return f
+
+
+    # "Stacking Residual Units on the same stage"
+    def _layer(self, block, n_input_plane, n_output_plane, count, stride):
+        def f(net):
+            net = block(n_input_plane, n_output_plane, stride)(net)
+            for i in range(2, int(count + 1)):
+                net = block(n_output_plane, n_output_plane, stride=(1, 1))(net)
+            return net
+
+        return f
+
+#    def create_model(self):
+    def __call__(self):
+        logging.debug("Creating model...")
+
+        assert ((self._depth - 4) % 6 == 0)
+        n = (self._depth - 4) / 6
+
+        inputs = Input(shape=self._input_shape)
+
+        n_stages = [16, 16 * self._k, 32 * self._k, 64 * self._k]
+
+        conv1 = Conv2D(filters=n_stages[0], kernel_size=(3, 3),
+                              strides=(1, 1),
+                              padding="same",
+                              kernel_initializer=self._weight_init,
+                              kernel_regularizer=l2(self._weight_decay),
+                              use_bias=self._use_bias)(inputs)  # "One conv at the beginning (spatial size: 32x32)"
+
+        # Add wide residual blocks
+        block_fn = self._wide_basic
+        conv2 = self._layer(block_fn, n_input_plane=n_stages[0], n_output_plane=n_stages[1], count=n, stride=(1, 1))(conv1)
+        conv3 = self._layer(block_fn, n_input_plane=n_stages[1], n_output_plane=n_stages[2], count=n, stride=(2, 2))(conv2)
+        conv4 = self._layer(block_fn, n_input_plane=n_stages[2], n_output_plane=n_stages[3], count=n, stride=(2, 2))(conv3)
+        batch_norm = BatchNormalization(axis=self._channel_axis)(conv4)
+        relu = Activation("relu")(batch_norm)
+
+        # Classifier block
+        pool = AveragePooling2D(pool_size=(8, 8), strides=(1, 1), padding="same")(relu)
+        flatten = Flatten()(pool)
+        predictions_g = Dense(units=2, kernel_initializer=self._weight_init, use_bias=self._use_bias,
+                              kernel_regularizer=l2(self._weight_decay), activation="softmax",
+                              name="pred_gender")(flatten)
+        predictions_a = Dense(units=101, kernel_initializer=self._weight_init, use_bias=self._use_bias,
+                              kernel_regularizer=l2(self._weight_decay), activation="softmax",
+                              name="pred_age")(flatten)
+        model = Model(inputs=inputs, outputs=[predictions_g, predictions_a])
+
+        return model
+
+
+def main():
+    model = WideResNet(64)()
+    model.summary()
+
+
+if __name__ == '__main__':
+    main()

18. Facial Applications - Emotion, Age & Gender Recognition/1. Chapter Introduction.srt

@@ -0,0 +1,47 @@
+1
+00:00:00,390 --> 00:00:07,520
+OK so welcome to Chapter 18 this episode of deep Haviland's because I'm no longer teaching you new Tieri
+
+2
+00:00:07,530 --> 00:00:13,710
+here Purcey what I'm going to do now is going to be like a history practice project where we build basically
+
+3
+00:00:13,710 --> 00:00:20,280
+an emotion age and gender recognition using face detection at first to find faces in an image or video
+
+4
+00:00:20,760 --> 00:00:26,850
+and then we pick out what emotion is on the face or being shown by the face the estimated age or predicted
+
+5
+00:00:26,880 --> 00:00:29,240
+age of the person and to predict the gender.
+
+6
+00:00:29,370 --> 00:00:33,890
+So let's take a look at a section and see how it split it up so fiercely.
+
+7
+00:00:34,020 --> 00:00:38,040
+We're going to build a simple emotion or facial expression to the actor.
+
+8
+00:00:38,130 --> 00:00:42,780
+And secondly we're going to bill and we're going to build and age and gender directed director.
+
+9
+00:00:43,140 --> 00:00:45,600
+And then the last chapter we're going to combine them both.
+
+10
+00:00:45,840 --> 00:00:48,780
+And it's going to be called Deep surveillance.
+
+11
+00:00:49,080 --> 00:00:50,130
+So stay tuned.
+
+12
+00:00:50,130 --> 00:00:51,510
+It's going to be a very cool projec.

18. Facial Applications - Emotion, Age & Gender Recognition/2. Build an Emotion, Facial Expression Detector.srt

@@ -0,0 +1,1239 @@
+1
+00:00:00,570 --> 00:00:06,450
+Hi and welcome to Chapter 18 point to where we get to build our supercool a motion detector.
+
+2
+00:00:06,480 --> 00:00:11,600
+So let's move on into a virtual machine and go to a Python book and let's see how it's done.
+
+3
+00:00:12,050 --> 00:00:19,800
+OK so now we're here in our Python browser and let's go to Chapter 18 deep civilians and eighteen point
+
+4
+00:00:19,800 --> 00:00:23,340
+to is building an emotion that acts with little little Viji.
+
+5
+00:00:23,400 --> 00:00:24,920
+So let's open him up.
+
+6
+00:00:26,910 --> 00:00:27,960
+And there we go.
+
+7
+00:00:27,960 --> 00:00:28,590
+Should be loaded.
+
+8
+00:00:28,590 --> 00:00:29,340
+Here we go.
+
+9
+00:00:29,340 --> 00:00:30,050
+All right.
+
+10
+00:00:30,330 --> 00:00:36,040
+So now the first thing we're going to do before I even go into this not book is Let's go back to our
+
+11
+00:00:36,060 --> 00:00:39,100
+father explo here and you see these two directories.
+
+12
+00:00:39,140 --> 00:00:42,510
+Let me explain to you and picture of me for testing.
+
+13
+00:00:42,510 --> 00:00:45,810
+Let me explain to you what these three directories are first for this age and gender one.
+
+14
+00:00:45,870 --> 00:00:48,540
+This is our age and gender which is in the next chapter.
+
+15
+00:00:48,660 --> 00:00:50,820
+We're not going to touch just this just yet.
+
+16
+00:00:50,820 --> 00:00:52,670
+So let's leave this alone here.
+
+17
+00:00:52,910 --> 00:00:54,500
+Now this here is food.
+
+18
+00:00:54,510 --> 00:00:55,330
+2013.
+
+19
+00:00:55,340 --> 00:00:57,760
+That's a dataset that we're going to try it on.
+
+20
+00:00:58,080 --> 00:00:59,580
+So let's take a look at this dataset.
+
+21
+00:00:59,580 --> 00:01:01,500
+So we have two directories.
+
+22
+00:01:01,770 --> 00:01:04,190
+Some of the CSU files that we aren't going to use.
+
+23
+00:01:04,200 --> 00:01:05,630
+We're going to look at these here.
+
+24
+00:01:05,760 --> 00:01:08,990
+I would set up so there's a trade and validation as usual.
+
+25
+00:01:09,000 --> 00:01:12,480
+And now we see these are the emotions here.
+
+26
+00:01:13,310 --> 00:01:19,380
+There is something I'm going to do which I haven't told you guys yet is that we're going to basically
+
+27
+00:01:19,650 --> 00:01:26,400
+delete or discard the discussed directory because I'm going to show you the plots afterward but discussed
+
+28
+00:01:26,460 --> 00:01:29,320
+only has four hundred and fifty five images.
+
+29
+00:01:29,400 --> 00:01:36,390
+All right now let's take a look at fear there has a lot more images for television.
+
+30
+00:01:36,660 --> 00:01:44,290
+So you can already see an angry happy neutralising look at Happy How many faces on this artery.
+
+31
+00:01:44,330 --> 00:01:45,600
+All right.
+
+32
+00:01:45,610 --> 00:01:47,330
+It is 7000.
+
+33
+00:01:47,330 --> 00:01:48,590
+So it's quite a lot.
+
+34
+00:01:48,590 --> 00:01:53,200
+So these are this of go back to this.
+
+35
+00:01:53,380 --> 00:01:56,590
+This is a very imbalanced data set here.
+
+36
+00:01:56,590 --> 00:02:05,020
+So these two things we can do one we can move the disgustful the images here into fear because honestly
+
+37
+00:02:05,020 --> 00:02:06,140
+are a bit similar.
+
+38
+00:02:06,160 --> 00:02:10,090
+It's bit hard even for me to pick out what is fair and what is discussed.
+
+39
+00:02:10,180 --> 00:02:12,170
+But for now let's just delete it.
+
+40
+00:02:12,220 --> 00:02:17,070
+So we press Delete here and let's go back to validation and we press Delete here.
+
+41
+00:02:17,170 --> 00:02:17,690
+OK.
+
+42
+00:02:17,830 --> 00:02:24,520
+So now we just left at six classes instead of seven and six much more evenly balanced classes even surprise
+
+43
+00:02:24,580 --> 00:02:25,150
+surprise.
+
+44
+00:02:25,180 --> 00:02:31,690
+The second one yet has 400 something images but that's fine because surprise does look a lot different
+
+45
+00:02:32,050 --> 00:02:33,620
+to disgust.
+
+46
+00:02:33,700 --> 00:02:39,130
+So let's go back to here and cascades which I mentioned before.
+
+47
+00:02:39,140 --> 00:02:40,640
+No I haven't mentioned that yet.
+
+48
+00:02:40,660 --> 00:02:43,710
+That's coming up in our object detection chapter.
+
+49
+00:02:43,990 --> 00:02:46,500
+But how cascades are basically fiest detection.
+
+50
+00:02:46,510 --> 00:02:48,620
+Well it's a type of object detector.
+
+51
+00:02:48,970 --> 00:02:51,690
+And we're going to use We're actually not going to use detectors.
+
+52
+00:02:51,700 --> 00:02:52,690
+We can do that.
+
+53
+00:02:52,750 --> 00:02:54,170
+I should just leave it in Fanaa.
+
+54
+00:02:54,250 --> 00:02:58,170
+We're going to use these here for BCB or face the action unit.
+
+55
+00:02:58,330 --> 00:03:00,260
+So let's go back to this notebook here.
+
+56
+00:03:01,230 --> 00:03:01,770
+All right.
+
+57
+00:03:01,770 --> 00:03:05,210
+So this is how we load the data set.
+
+58
+00:03:05,220 --> 00:03:06,650
+This is basically a number of classes.
+
+59
+00:03:06,660 --> 00:03:07,380
+It was seven.
+
+60
+00:03:07,410 --> 00:03:08,690
+Now it's six.
+
+61
+00:03:08,730 --> 00:03:14,260
+Now we're actually going to have the number of rows and as 48th to him in size and they're all greyscale.
+
+62
+00:03:14,380 --> 00:03:20,730
+I mentioned that you may have mentioned it in the previous section I hope but it is all grayscale images.
+
+63
+00:03:20,740 --> 00:03:24,880
+Just take a look and see to verify all.
+
+64
+00:03:25,110 --> 00:03:30,810
+So now we also are going to do some data augmentation as we have become routine.
+
+65
+00:03:30,810 --> 00:03:33,840
+Right now these are the parameters of used here.
+
+66
+00:03:33,930 --> 00:03:36,800
+Feel free to play with us as well.
+
+67
+00:03:36,840 --> 00:03:41,720
+It doesn't make a huge difference sometimes but you never know it.
+
+68
+00:03:41,940 --> 00:03:44,180
+No normal rescaling normalization.
+
+69
+00:03:44,340 --> 00:03:46,420
+Now we just put our data here.
+
+70
+00:03:46,420 --> 00:03:51,270
+Notice we have a new pragmatical column mode in boats that I don't believe we used before because it
+
+71
+00:03:51,270 --> 00:03:52,910
+defaults to color.
+
+72
+00:03:53,400 --> 00:03:55,000
+No we're specifying it's greyscale.
+
+73
+00:03:55,020 --> 00:03:56,170
+So just take note of that.
+
+74
+00:03:56,170 --> 00:04:03,030
+So when you want to do a grayscale basically operation of training you have to specified here in your
+
+75
+00:04:03,280 --> 00:04:06,500
+day to Gen from factory function.
+
+76
+00:04:06,540 --> 00:04:11,800
+So now let's load this and it's going to tell us how many images and how many classes.
+
+77
+00:04:11,800 --> 00:04:13,330
+Six and six.
+
+78
+00:04:13,330 --> 00:04:14,550
+Excellent.
+
+79
+00:04:14,560 --> 00:04:14,850
+All right.
+
+80
+00:04:14,870 --> 00:04:20,750
+So now we move on to let me just make this a little more clear.
+
+81
+00:04:20,750 --> 00:04:32,110
+You can put it here this is basically imports Leesville Karris imports and put another thing here so
+
+82
+00:04:32,110 --> 00:04:36,060
+we can say this is Chris little.
+
+83
+00:04:36,550 --> 00:04:38,180
+He Jiechi model
+
+84
+00:04:41,390 --> 00:04:46,950
+so now as you've seen in the previous chapter of its Simpsons this is the what we're going to use.
+
+85
+00:04:47,240 --> 00:04:52,470
+So let's just run this and here we go.
+
+86
+00:04:52,650 --> 00:04:54,980
+Number of parameters.
+
+87
+00:04:55,020 --> 00:04:56,810
+This does look a little different much.
+
+88
+00:04:56,820 --> 00:04:57,580
+Oh no.
+
+89
+00:04:57,600 --> 00:04:58,720
+Because it's black and white.
+
+90
+00:04:58,760 --> 00:05:03,600
+Grayscale images that is wide enough parameters or less silly.
+
+91
+00:05:03,960 --> 00:05:04,360
+OK.
+
+92
+00:05:04,530 --> 00:05:05,560
+Happens a lot.
+
+93
+00:05:05,660 --> 00:05:06,840
+It's fine.
+
+94
+00:05:06,900 --> 00:05:12,940
+Now let us just label this as well treating our model.
+
+95
+00:05:12,960 --> 00:05:13,700
+There we go.
+
+96
+00:05:14,010 --> 00:05:18,540
+So now we have basically some callbacks sets here.
+
+97
+00:05:18,690 --> 00:05:22,000
+I'm going to pause here because I realized I made some changes to this code.
+
+98
+00:05:22,020 --> 00:05:23,570
+Actually no it's fine.
+
+99
+00:05:23,570 --> 00:05:24,130
+I'm stupid.
+
+100
+00:05:24,180 --> 00:05:25,390
+It is fine.
+
+101
+00:05:26,110 --> 00:05:27,270
+So let's continue.
+
+102
+00:05:27,270 --> 00:05:28,380
+Number of training samples.
+
+103
+00:05:28,380 --> 00:05:33,720
+Let's just double check this because I'm not entirely sure it just reflects the classes we deleted.
+
+104
+00:05:33,750 --> 00:05:40,060
+So 6:32 tree let's scroll up and let's see if it does 6:40 we did OK.
+
+105
+00:05:40,080 --> 00:05:40,580
+Good.
+
+106
+00:05:41,630 --> 00:05:43,920
+I'm doing things right for a change.
+
+107
+00:05:43,940 --> 00:05:44,410
+OK.
+
+108
+00:05:44,600 --> 00:05:44,940
+Good.
+
+109
+00:05:45,020 --> 00:05:52,940
+And now standard procedure how we fit using our data generators and all callbacks to find here and model
+
+110
+00:05:52,940 --> 00:05:53,750
+compile.
+
+111
+00:05:53,780 --> 00:05:59,210
+So I'm not actually going to run this is going to show you what I've run prior to the chapter and this
+
+112
+00:05:59,240 --> 00:06:01,160
+was with six classes.
+
+113
+00:06:01,160 --> 00:06:01,520
+All right.
+
+114
+00:06:01,580 --> 00:06:04,250
+And now all the way down.
+
+115
+00:06:04,290 --> 00:06:06,370
+You see we have 47 percent accuracy.
+
+116
+00:06:06,560 --> 00:06:08,110
+Now that is not that good.
+
+117
+00:06:08,220 --> 00:06:08,860
+All right.
+
+118
+00:06:09,080 --> 00:06:11,790
+So tele box is actually not bad.
+
+119
+00:06:11,960 --> 00:06:17,240
+I've seen people use similar models and actually sometimes much more complicated models someone who's
+
+120
+00:06:17,240 --> 00:06:19,380
+a full DTG 9000 on this.
+
+121
+00:06:19,580 --> 00:06:23,940
+And it is very difficult to get past 70 percent accuracy.
+
+122
+00:06:24,110 --> 00:06:30,680
+I am 100 percent sure if I try this for maybe 280 ebox I'll get probably about 60 percent accuracy.
+
+123
+00:06:30,770 --> 00:06:32,030
+So you can give it a try.
+
+124
+00:06:32,030 --> 00:06:36,260
+All right I'll probably do it and update it at a later date.
+
+125
+00:06:36,530 --> 00:06:39,020
+With this new model for you guys.
+
+126
+00:06:39,020 --> 00:06:40,870
+So that's fine.
+
+127
+00:06:40,880 --> 00:06:44,300
+So now let's look at the confusion matrix from these results.
+
+128
+00:06:44,310 --> 00:06:46,100
+It is 47 percent.
+
+129
+00:06:46,640 --> 00:06:47,020
+OK.
+
+130
+00:06:47,150 --> 00:06:50,720
+So normally what do you what do you think of this.
+
+131
+00:06:50,900 --> 00:06:52,370
+How would you analyze this.
+
+132
+00:06:52,610 --> 00:06:56,800
+Now I would say it's not that good especially not good at picking up fear.
+
+133
+00:06:57,080 --> 00:06:58,940
+You really got much fear correct.
+
+134
+00:06:58,940 --> 00:06:59,620
+All right.
+
+135
+00:06:59,780 --> 00:07:01,610
+Now what does it say but happy to.
+
+136
+00:07:01,970 --> 00:07:03,200
+Now this is these are comics by the way.
+
+137
+00:07:03,200 --> 00:07:06,920
+But even still it's definitely good at getting happy.
+
+138
+00:07:06,920 --> 00:07:07,840
+All right.
+
+139
+00:07:08,210 --> 00:07:13,080
+But the others aren't that great yet as you do see some mismatches here.
+
+140
+00:07:13,250 --> 00:07:15,640
+You do see fares not being picked up that well.
+
+141
+00:07:15,740 --> 00:07:16,860
+I mean it has been picked up.
+
+142
+00:07:16,950 --> 00:07:19,790
+Is this a different color from Stephanie from here to here.
+
+143
+00:07:20,170 --> 00:07:23,580
+If we need a good answer to probably tell but there's a difference.
+
+144
+00:07:23,660 --> 00:07:30,020
+But generally you can tell here fares actually being confused with neutral and angry a lot as well.
+
+145
+00:07:30,020 --> 00:07:38,330
+So this analysis says Our model is decent but not great which is obvious given its 47 percent accuracy
+
+146
+00:07:38,330 --> 00:07:38,850
+here.
+
+147
+00:07:39,320 --> 00:07:46,250
+And you can look at the one school is here definitely can see exactly as I said Happy is it's good at
+
+148
+00:07:46,250 --> 00:07:49,710
+finding happy hearted finding fair decent.
+
+149
+00:07:49,730 --> 00:07:56,360
+Everything else except neutral although in my experience at least for my face it picked up neutral fairly
+
+150
+00:07:56,360 --> 00:07:57,380
+well.
+
+151
+00:07:57,380 --> 00:07:57,710
+All right.
+
+152
+00:07:57,710 --> 00:07:58,770
+So let's just look.
+
+153
+00:07:58,790 --> 00:08:00,910
+I believe this was saved.
+
+154
+00:08:01,040 --> 00:08:02,190
+Let me just make sure.
+
+155
+00:08:03,350 --> 00:08:05,320
+It would be truly great.
+
+156
+00:08:05,360 --> 00:08:05,940
+OK.
+
+157
+00:08:06,290 --> 00:08:15,270
+So now that's look at our model as usual this takes about an annoyingly long seconds.
+
+158
+00:08:15,790 --> 00:08:16,210
+OK.
+
+159
+00:08:16,330 --> 00:08:18,550
+Quick quick and it's time Ali.
+
+160
+00:08:18,550 --> 00:08:23,950
+All right so let's get A-class labels again because I believe they ran it before when it was 7 classes
+
+161
+00:08:23,950 --> 00:08:24,560
+here.
+
+162
+00:08:25,150 --> 00:08:29,930
+And now let's look at some images not sure what's in this directory.
+
+163
+00:08:29,950 --> 00:08:30,330
+OK.
+
+164
+00:08:30,460 --> 00:08:35,500
+So let's see how it went we predicted angry and true class was fair.
+
+165
+00:08:35,850 --> 00:08:37,240
+So OK.
+
+166
+00:08:37,270 --> 00:08:39,380
+Reasonable fanfare.
+
+167
+00:08:39,430 --> 00:08:39,910
+Bingo.
+
+168
+00:08:39,920 --> 00:08:41,400
+Got it spot on.
+
+169
+00:08:41,410 --> 00:08:41,890
+Fair.
+
+170
+00:08:41,900 --> 00:08:42,580
+And it was neutral.
+
+171
+00:08:42,580 --> 00:08:45,190
+No I wouldn't say this is a neutral expression.
+
+172
+00:08:45,240 --> 00:08:51,580
+Don't know who labeled this datasets but I mean it's probably not Fader's probably some weird expression
+
+173
+00:08:51,730 --> 00:08:53,330
+she's making.
+
+174
+00:08:53,980 --> 00:08:55,630
+We predict an angry and it's neutral.
+
+175
+00:08:55,650 --> 00:09:00,280
+Now I would say you're probably neutral but it does look a bit angry doesn't he.
+
+176
+00:09:01,310 --> 00:09:01,570
+OK.
+
+177
+00:09:01,590 --> 00:09:05,870
+So this one picked up we detected fear but it was actually angry close.
+
+178
+00:09:06,360 --> 00:09:08,990
+This one we predicted angry but actually was fair.
+
+179
+00:09:09,120 --> 00:09:12,670
+We parted angry again pretty fair and neutral.
+
+180
+00:09:12,670 --> 00:09:13,570
+It's true.
+
+181
+00:09:13,890 --> 00:09:14,270
+OK.
+
+182
+00:09:14,310 --> 00:09:19,710
+So now we saw this and now let's actually try it on a picture of me.
+
+183
+00:09:19,800 --> 00:09:20,510
+OK.
+
+184
+00:09:20,850 --> 00:09:24,810
+So one thing I should have mentioned before actually it was an undisclosed isn't disco.
+
+185
+00:09:25,050 --> 00:09:27,930
+This is how we use OKOK cascade justifies.
+
+186
+00:09:27,960 --> 00:09:32,490
+Now we're using something different as an open Zeevi function.
+
+187
+00:09:32,820 --> 00:09:39,390
+So it be pointed if it classified we want to use that was to go back to this this one.
+
+188
+00:09:39,490 --> 00:09:40,050
+All right.
+
+189
+00:09:40,210 --> 00:09:48,370
+We have Eifel buddy and caucus's fires which I'm probably going to see in space are tiny it was going
+
+190
+00:09:48,370 --> 00:09:51,340
+to lead them if they were taking up all of space but they don't.
+
+191
+00:09:51,670 --> 00:09:52,250
+OK.
+
+192
+00:09:52,720 --> 00:09:54,700
+So this is a phase detector module.
+
+193
+00:09:54,760 --> 00:09:56,860
+Now what this module does.
+
+194
+00:09:56,860 --> 00:09:59,990
+You can see we've created as obstacles fiest classify.
+
+195
+00:10:00,150 --> 00:10:05,110
+Now when we get an image here this is an image from my webcam.
+
+196
+00:10:05,110 --> 00:10:06,940
+This is a function here that we're looking at by the way.
+
+197
+00:10:07,060 --> 00:10:07,550
+OK.
+
+198
+00:10:07,870 --> 00:10:16,030
+So we get this function we convert it into a greyscale image and then we pass this phunk face this gray
+
+199
+00:10:16,420 --> 00:10:19,700
+scale image of the webcam input into this.
+
+200
+00:10:19,700 --> 00:10:24,280
+This is what does the face detection does detect multi-skilled function.
+
+201
+00:10:24,280 --> 00:10:27,010
+These are some parameters to tweak to tweak the sensitivity as well.
+
+202
+00:10:27,040 --> 00:10:31,870
+And how many times like if you want to find small pieces a lot of cases you tweak some of the Skilling
+
+203
+00:10:31,870 --> 00:10:32,930
+parameters.
+
+204
+00:10:33,010 --> 00:10:38,130
+So what it returns do is basically an array of faces.
+
+205
+00:10:38,140 --> 00:10:42,310
+So basically if you have no faces the fact that it is written in some blank data because I'm using it
+
+206
+00:10:42,340 --> 00:10:44,770
+in this function for some other stuff.
+
+207
+00:10:45,040 --> 00:10:48,790
+But if it finds faces basically it returns this.
+
+208
+00:10:48,790 --> 00:10:55,820
+These are basically did the location of the face the X Y which is the top left basically x.
+
+209
+00:10:55,900 --> 00:10:58,920
+Let's assume let's assume this is a box.
+
+210
+00:10:58,930 --> 00:11:03,690
+All right so x y is going to be say let's say this.
+
+211
+00:11:03,750 --> 00:11:05,540
+It's in this box I'm doing right here.
+
+212
+00:11:05,560 --> 00:11:07,820
+Hope you can make it out is the face.
+
+213
+00:11:07,820 --> 00:11:13,610
+So the x y starts here on the top left corner of the face and the width and the height is which is this
+
+214
+00:11:13,610 --> 00:11:18,850
+way and hightest like the down measurement definitely it's still off the face.
+
+215
+00:11:18,850 --> 00:11:24,830
+So that's how we use no discipline Sivy fun function here to draw a rectangle around this.
+
+216
+00:11:25,030 --> 00:11:31,120
+And then we take description of the image here and basically we just crop it to get this fish out of
+
+217
+00:11:31,120 --> 00:11:31,690
+it.
+
+218
+00:11:31,860 --> 00:11:37,980
+And what I do is just said run this of all the faces in this file resize it correctly to what declassifies
+
+219
+00:11:38,380 --> 00:11:47,110
+is has been trained to detect 48 48 and X return an array of all the faces or the rectangle dimensions
+
+220
+00:11:47,470 --> 00:11:53,110
+and the original image muchall way but I was maybe doing it doing something with it afterward.
+
+221
+00:11:53,410 --> 00:11:56,560
+And yes I think I was just putting the label on it afterwards.
+
+222
+00:11:57,010 --> 00:11:58,150
+So there we go.
+
+223
+00:11:58,150 --> 00:12:01,590
+So we love my image to run this cool function and hope it works.
+
+224
+00:12:01,600 --> 00:12:03,400
+After that explanation.
+
+225
+00:12:03,880 --> 00:12:04,270
+Yes.
+
+226
+00:12:04,270 --> 00:12:04,580
+OK.
+
+227
+00:12:04,600 --> 00:12:07,460
+So thanks I'm happy I was happy actually.
+
+228
+00:12:07,600 --> 00:12:09,580
+This was on my birthday two months ago.
+
+229
+00:12:09,880 --> 00:12:16,480
+I was in Madeira Portugal which is a wonderful wonderful island that you really should visit and not
+
+230
+00:12:16,840 --> 00:12:19,520
+get paid to say that's just highly recommended.
+
+231
+00:12:19,760 --> 00:12:22,710
+When we met my wife and I went whale watching that day.
+
+232
+00:12:23,050 --> 00:12:24,740
+So yes I was happy.
+
+233
+00:12:24,790 --> 00:12:29,070
+So pretty pretty decent so you can load your images here.
+
+234
+00:12:29,080 --> 00:12:33,420
+One thing to note you can probably reduce the size of this Texas started maybe out of left corner here.
+
+235
+00:12:33,940 --> 00:12:37,240
+And kids faces more on the right hand side of image.
+
+236
+00:12:37,270 --> 00:12:40,390
+Texas not going to go outside of the image.
+
+237
+00:12:40,390 --> 00:12:43,550
+So that's one thing you can do for your homework listen.
+
+238
+00:12:43,870 --> 00:12:46,610
+So now let's try this on our web cam.
+
+239
+00:12:46,690 --> 00:12:47,250
+OK.
+
+240
+00:12:47,560 --> 00:12:49,480
+So let's try this now.
+
+241
+00:12:54,200 --> 00:12:54,440
+OK.
+
+242
+00:12:54,450 --> 00:12:58,830
+So you may have noticed a slight break in the code and that was because when I ran this I realized my
+
+243
+00:12:58,830 --> 00:13:02,310
+T-shirt had a stain which did not look good on camera.
+
+244
+00:13:02,310 --> 00:13:04,600
+So again I'm not dressed up.
+
+245
+00:13:04,650 --> 00:13:08,260
+I'm just here at home alone recording this.
+
+246
+00:13:08,550 --> 00:13:11,780
+So let me just run this.
+
+247
+00:13:11,950 --> 00:13:12,790
+And here we go.
+
+248
+00:13:12,790 --> 00:13:20,270
+So we see my face being detected for my emotion my facial emotion expression my microphone right here.
+
+249
+00:13:20,270 --> 00:13:23,590
+I hope they don't make any noise so they come.
+
+250
+00:13:23,590 --> 00:13:28,760
+So right now it wasn't that alternating between happy and neutral quite a bit.
+
+251
+00:13:30,480 --> 00:13:33,110
+Surprise they call it I worked.
+
+252
+00:13:33,120 --> 00:13:33,400
+All right.
+
+253
+00:13:33,420 --> 00:13:33,910
+Nice.
+
+254
+00:13:33,920 --> 00:13:36,230
+So it is working fairly well.
+
+255
+00:13:36,660 --> 00:13:38,880
+So you can experiment with this training.
+
+256
+00:13:39,030 --> 00:13:42,450
+One thing you should know you see this bounding box here.
+
+257
+00:13:42,600 --> 00:13:48,600
+This is actually bad because just take a quick look at a dataset here.
+
+258
+00:13:48,900 --> 00:13:56,550
+Something you can do for us in which I neglected to do for you guys but these faces tightly cropped
+
+259
+00:13:56,700 --> 00:13:57,600
+if you take a look at them.
+
+260
+00:13:57,690 --> 00:14:01,820
+Let's just open this quickly they mean tighter cropped isn't they.
+
+261
+00:14:01,880 --> 00:14:04,670
+They don't have that much space inside.
+
+262
+00:14:04,830 --> 00:14:13,180
+So generally because I've noticed this is this is not that tightly cropped hair on the webcam.
+
+263
+00:14:13,430 --> 00:14:16,250
+So let me just go back to this code.
+
+264
+00:14:16,370 --> 00:14:21,520
+So if you wanted to change that you actually CAN THIS IS ACTUALLY actually the opposite of cropping.
+
+265
+00:14:21,530 --> 00:14:29,860
+So I can actually change this to 20 20 which was because the default settings for the would Cascades
+
+266
+00:14:29,960 --> 00:14:31,030
+quite tight.
+
+267
+00:14:31,280 --> 00:14:37,690
+I actually previously I did some spacing some left right up down spacing so let's run this now.
+
+268
+00:14:37,730 --> 00:14:38,880
+See what happens.
+
+269
+00:14:40,320 --> 00:14:40,650
+OK.
+
+270
+00:14:40,670 --> 00:14:45,200
+So as you can see it is a bit it is a bit better.
+
+271
+00:14:45,380 --> 00:14:47,170
+Maybe we can reduce to bits even more.
+
+272
+00:14:47,240 --> 00:14:52,310
+And we can eliminate it all to get a much less evil.
+
+273
+00:14:52,390 --> 00:14:54,450
+This is stupid.
+
+274
+00:14:54,470 --> 00:14:58,850
+And so the w w just not do it at the height spacing at least
+
+275
+00:15:02,620 --> 00:15:07,170
+so I would say this is probably a small tightly cropped on my face.
+
+276
+00:15:07,270 --> 00:15:12,220
+And if you wanted to go even for that let's just not do any height adjustment here.
+
+277
+00:15:13,400 --> 00:15:14,840
+And see what that gives us.
+
+278
+00:15:14,850 --> 00:15:18,390
+No.
+
+279
+00:15:19,170 --> 00:15:19,530
+It is.
+
+280
+00:15:19,530 --> 00:15:22,050
+It is definitely more stable and better.
+
+281
+00:15:22,050 --> 00:15:25,830
+I would say OK cool.
+
+282
+00:15:25,900 --> 00:15:27,140
+So this works fairly well.
+
+283
+00:15:27,190 --> 00:15:27,670
+OK.
+
+284
+00:15:27,840 --> 00:15:28,500
+I'm happy with this.
+
+285
+00:15:28,510 --> 00:15:29,880
+I hope you're happy with us.
+
+286
+00:15:29,890 --> 00:15:36,100
+This is a 47 percent accurate model and it's doing quite well but decently well at least.
+
+287
+00:15:36,100 --> 00:15:41,630
+So what you can do as a lesson for you guys is trained us for more ebox.
+
+288
+00:15:41,650 --> 00:15:49,690
+Also try a different augmentations try different optimize if you want to use them with a fairly large
+
+289
+00:15:49,690 --> 00:15:50,360
+dating rate.
+
+290
+00:15:50,380 --> 00:15:53,040
+Just realize you can reduce that even more.
+
+291
+00:15:53,620 --> 00:15:58,510
+And you know adjust your template to options here and what you can do as well.
+
+292
+00:15:58,630 --> 00:16:06,150
+You can add more filters individually started 32 64 128 256 hub what you saw at 64.
+
+293
+00:16:06,370 --> 00:16:14,200
+Eliminating the tool to get us 64 128 256 and 512 and you can even add more densely as here may not
+
+294
+00:16:14,950 --> 00:16:16,850
+be necessary but try it.
+
+295
+00:16:17,350 --> 00:16:19,750
+Play with your dhrupad values as well.
+
+296
+00:16:19,810 --> 00:16:24,950
+He'd even add a whole new convolutional layer here instead of stopping at 256.
+
+297
+00:16:25,060 --> 00:16:34,620
+You can just go here and add one with 512 I'll do this for you but you can do it on your own.
+
+298
+00:16:34,920 --> 00:16:38,830
+Maybe change activation functions maybe change initializers as I say.
+
+299
+00:16:38,880 --> 00:16:44,390
+Although I wouldn't change to be fair I wouldn't change the Batumi position activation and iching normal.
+
+300
+00:16:44,610 --> 00:16:49,740
+But you can if you want but I wouldn't I think those are the best for Viji type model like we're using
+
+301
+00:16:49,740 --> 00:16:50,700
+here.
+
+302
+00:16:50,700 --> 00:16:53,980
+And last Leslee try for more ebox.
+
+303
+00:16:54,000 --> 00:16:55,010
+All right.
+
+304
+00:16:55,050 --> 00:16:59,200
+A good value for this a satisfactory value would be 70 percent.
+
+305
+00:16:59,520 --> 00:17:04,960
+So give it a go again and try different data augmentations as well.
+
+306
+00:17:05,220 --> 00:17:10,770
+Give it a go and see how much good if accurate model you can get.
+
+307
+00:17:10,770 --> 00:17:12,660
+OK so that's it for this lesson.
+
+308
+00:17:12,990 --> 00:17:20,510
+What we're going to do next is do an run an age and gender detector I guess you could call it classify
+
+309
+00:17:21,270 --> 00:17:29,200
+and combined them afterward into one super Fishell deep surveillance classify classify.
+
+310
+00:17:29,710 --> 00:17:30,450
+OK thank you.

18. Facial Applications - Emotion, Age & Gender Recognition/2.1 Download Dataset.html

@@ -0,0 +1 @@
+<script type="text/javascript">window.location = "https://drive.google.com/file/d/1317edb3koW63Zzjxt5b-9xOK1ElQ07xk/view?usp=sharing";</script>

18. Facial Applications - Emotion, Age & Gender Recognition/3. Build EmotionAgeGender Recognition in our Deep Surveillance Monitor.srt

@@ -0,0 +1,1547 @@
+1
+00:00:00,690 --> 00:00:06,150
+Hi and welcome to chapter eighteen point three where we're going to build a deep surveillance facial
+
+2
+00:00:06,150 --> 00:00:10,980
+monitoring system that combines emotion age and gender recognition.
+
+3
+00:00:11,010 --> 00:00:15,910
+So now let's go into an update on that book and our virtual machine and start building this.
+
+4
+00:00:16,200 --> 00:00:16,530
+OK.
+
+5
+00:00:16,560 --> 00:00:18,170
+So we're in chapter 18.
+
+6
+00:00:18,240 --> 00:00:20,380
+No footnote but book for this.
+
+7
+00:00:20,640 --> 00:00:28,650
+But is no 18.00 Reiffel notebook file whereas it is actually in here into to make it to quickly into
+
+8
+00:00:28,650 --> 00:00:31,110
+the age gender estimation for the hill.
+
+9
+00:00:31,110 --> 00:00:31,460
+All right.
+
+10
+00:00:31,470 --> 00:00:32,820
+So click it.
+
+11
+00:00:32,940 --> 00:00:39,870
+And what this is this is basically a guitar project I pulled it for one of the best one of the better
+
+12
+00:00:39,870 --> 00:00:42,740
+ones I should say for age and gender detection.
+
+13
+00:00:43,020 --> 00:00:47,820
+And the reason why I don't get it is for this code and what we're going to do we're going to use his
+
+14
+00:00:47,820 --> 00:00:52,500
+Pretorian model is mainly because it's this data set and it's.
+
+15
+00:00:52,760 --> 00:00:53,750
+It's not loaded here.
+
+16
+00:00:53,890 --> 00:00:58,320
+I hope it always will be taken up way too much space probably isn't.
+
+17
+00:00:58,320 --> 00:01:03,390
+But it's still said that he create He loaded he turned it on is quite large and I actually did have
+
+18
+00:01:03,390 --> 00:01:11,670
+a project prior to this where we were treating it on that dataset for age and stuff stop it's OK for
+
+19
+00:01:11,670 --> 00:01:14,280
+age and that's why.
+
+20
+00:01:14,620 --> 00:01:14,920
+OK.
+
+21
+00:01:14,970 --> 00:01:18,240
+In fact the dataset is here is not good at all.
+
+22
+00:01:18,240 --> 00:01:18,490
+All right.
+
+23
+00:01:18,510 --> 00:01:21,380
+So I'm going to leave that before I give it to you guys.
+
+24
+00:01:21,390 --> 00:01:23,070
+All right back to this.
+
+25
+00:01:23,130 --> 00:01:23,690
+OK.
+
+26
+00:01:24,030 --> 00:01:28,680
+So either way when you're trying this model you actually can train his on his model here.
+
+27
+00:01:28,820 --> 00:01:29,170
+All right.
+
+28
+00:01:29,190 --> 00:01:30,560
+He has the code to train.
+
+29
+00:01:30,630 --> 00:01:35,180
+However it is not quick and it is basically a standard training procedure.
+
+30
+00:01:35,190 --> 00:01:39,490
+What we have done before but it is not worth the effort to do in a CPE use system.
+
+31
+00:01:39,510 --> 00:01:45,780
+So I said you know what we will just actually just use our Pretorian models and we will just Pretorian
+
+32
+00:01:45,780 --> 00:01:52,860
+model and it's a good exercise to basically executing someone else's model is actually it may seem like
+
+33
+00:01:52,860 --> 00:01:55,620
+cheating sometimes but you can learn a lot.
+
+34
+00:01:55,620 --> 00:01:56,180
+All right.
+
+35
+00:01:56,220 --> 00:01:58,120
+So I'll quickly.
+
+36
+00:01:58,150 --> 00:01:59,970
+This is his project here.
+
+37
+00:02:00,180 --> 00:02:01,270
+Let's bring this up.
+
+38
+00:02:01,290 --> 00:02:02,780
+All right.
+
+39
+00:02:03,250 --> 00:02:07,890
+First what are we going to do is basically because he has some of the books he should actually label
+
+40
+00:02:08,100 --> 00:02:09,290
+what our minds are.
+
+41
+00:02:09,540 --> 00:02:11,350
+So student quickly.
+
+42
+00:02:11,400 --> 00:02:11,660
+All right.
+
+43
+00:02:11,670 --> 00:02:18,030
+So this is going to be eighteen point tree it's called us.
+
+44
+00:02:18,770 --> 00:02:20,660
+And second one we're going to run right after.
+
+45
+00:02:20,690 --> 00:02:26,360
+So we combine them both because firstly what we're going to run is his age and gender detector and Despard
+
+46
+00:02:26,400 --> 00:02:28,600
+be eighteen point tree.
+
+47
+00:02:28,710 --> 00:02:30,700
+B look at.
+
+48
+00:02:30,750 --> 00:02:32,080
+So now let's bring this up.
+
+49
+00:02:32,180 --> 00:02:33,420
+Stick a look at this.
+
+50
+00:02:35,020 --> 00:02:35,370
+Right.
+
+51
+00:02:35,380 --> 00:02:36,960
+So this is what I think I wanted.
+
+52
+00:02:36,990 --> 00:02:39,440
+This was his get help for his project.
+
+53
+00:02:39,460 --> 00:02:40,240
+OK.
+
+54
+00:02:40,690 --> 00:02:43,020
+Now I believe this was him.
+
+55
+00:02:43,060 --> 00:02:47,860
+If I'm not mistaken there was a slight issue with his project that didn't run off the bat and I had
+
+56
+00:02:47,860 --> 00:02:49,400
+to make some changes in the file.
+
+57
+00:02:49,720 --> 00:02:51,540
+Luckily I did make the changes in the file.
+
+58
+00:02:51,540 --> 00:02:56,620
+So if you you don't have to put this code from scratch his code is already in this field that we saw
+
+59
+00:02:56,620 --> 00:02:57,680
+here.
+
+60
+00:02:57,700 --> 00:03:03,070
+So basically he what's good about this still is that he does give some instructions some if you wanted
+
+61
+00:03:03,070 --> 00:03:08,690
+to actually train the model on you and he can trance on different data sets as well.
+
+62
+00:03:08,710 --> 00:03:11,460
+This is that is that he included in his file.
+
+63
+00:03:11,860 --> 00:03:17,000
+You can train Athill or you can use some of his demo file is the file we are actually going to.
+
+64
+00:03:17,280 --> 00:03:19,040
+That's the way it's open.
+
+65
+00:03:19,060 --> 00:03:21,690
+And you can see it here.
+
+66
+00:03:21,810 --> 00:03:27,160
+We don't want to doing it because I already have downloaded it and basically have been trapped under
+
+67
+00:03:27,200 --> 00:03:28,980
+I MVB basically this.
+
+68
+00:03:29,080 --> 00:03:31,910
+I am D-B data here talks about here.
+
+69
+00:03:31,910 --> 00:03:37,520
+Basically someone's script I am divvies Web site you know Internet Movie Database website extracted
+
+70
+00:03:37,520 --> 00:03:43,520
+defaces and actually labeled the ages and gender which might have been a quite tedious task but they
+
+71
+00:03:43,520 --> 00:03:44,150
+did it.
+
+72
+00:03:44,180 --> 00:03:44,730
+OK.
+
+73
+00:03:45,080 --> 00:03:47,350
+So these are his results here.
+
+74
+00:03:47,420 --> 00:03:52,730
+His Pretorian model is lost is quite low for age and gender.
+
+75
+00:03:52,730 --> 00:03:53,860
+Not that low in the end.
+
+76
+00:03:53,870 --> 00:03:55,490
+But it's fine.
+
+77
+00:03:55,670 --> 00:04:03,700
+And so he probably stopped at some pre-trained really something and got the Marlow's out before accuracy
+
+78
+00:04:04,100 --> 00:04:05,720
+and the stuff went to hell.
+
+79
+00:04:05,720 --> 00:04:07,370
+Actually I'm looking at around accuracy.
+
+80
+00:04:07,490 --> 00:04:10,600
+This is a one blue green and blue.
+
+81
+00:04:10,690 --> 00:04:13,040
+Like I said fine either way.
+
+82
+00:04:13,090 --> 00:04:15,080
+Definitely we're sitting here
+
+83
+00:04:18,070 --> 00:04:22,800
+actually is not fitting that is actually on his training data set.
+
+84
+00:04:22,840 --> 00:04:25,250
+So this kind of results to be fair.
+
+85
+00:04:25,260 --> 00:04:25,890
+All right.
+
+86
+00:04:26,110 --> 00:04:28,110
+Either way it worked fine.
+
+87
+00:04:28,120 --> 00:04:34,690
+And as an exercise for you guys if you're very if you're interested in seeing how this works his code
+
+88
+00:04:34,690 --> 00:04:35,510
+is here.
+
+89
+00:04:35,630 --> 00:04:37,360
+It's fairly well documented as well.
+
+90
+00:04:37,390 --> 00:04:42,460
+So you can take a look and try doing some cool stuff with this what we're doing now is using his preaching
+
+91
+00:04:42,460 --> 00:04:42,860
+model.
+
+92
+00:04:42,890 --> 00:04:51,010
+So basically extracted the code from his project code is code and doing it here and I it in the book.
+
+93
+00:04:51,010 --> 00:04:56,110
+So let's step through this code quickly it's not that different than the previous code just some little
+
+94
+00:04:56,110 --> 00:04:57,810
+things you should note.
+
+95
+00:04:57,880 --> 00:05:04,220
+Firstly this is how we load his model and we have to specify a hash when we look at his model.
+
+96
+00:05:04,360 --> 00:05:08,530
+So go back to our face detector results.
+
+97
+00:05:08,530 --> 00:05:09,950
+This is what gives us our faces.
+
+98
+00:05:09,970 --> 00:05:14,220
+And this is what displays our results actually.
+
+99
+00:05:14,470 --> 00:05:15,340
+Let me just check something.
+
+100
+00:05:15,340 --> 00:05:17,610
+I don't believe I'm using that function any more.
+
+101
+00:05:17,980 --> 00:05:20,420
+And I am right I am not using it anymore.
+
+102
+00:05:20,430 --> 00:05:23,700
+So that's to it on.
+
+103
+00:05:23,760 --> 00:05:25,050
+Totally unnecessary.
+
+104
+00:05:25,050 --> 00:05:26,400
+All right.
+
+105
+00:05:26,400 --> 00:05:29,020
+So now these are the moral premises here.
+
+106
+00:05:29,220 --> 00:05:29,900
+OK.
+
+107
+00:05:30,270 --> 00:05:31,360
+Just say no.
+
+108
+00:05:31,380 --> 00:05:34,040
+Can I leave this alone ok dept OK.
+
+109
+00:05:34,100 --> 00:05:39,230
+With none is all the specific specifications for loading his model.
+
+110
+00:05:39,270 --> 00:05:39,820
+OK.
+
+111
+00:05:39,990 --> 00:05:41,180
+His pre-trained weights.
+
+112
+00:05:41,190 --> 00:05:46,850
+So we do that so we have his model we loaded it we got the weights and everything from his model and
+
+113
+00:05:46,870 --> 00:05:50,770
+each the EF 5 format.
+
+114
+00:05:51,080 --> 00:05:54,450
+So no set up stuff.
+
+115
+00:05:54,630 --> 00:05:55,820
+We get his model.
+
+116
+00:05:55,920 --> 00:06:00,250
+Fitbit does or does not go with his model here and model that load.
+
+117
+00:06:00,270 --> 00:06:02,600
+So we get his model now officially.
+
+118
+00:06:02,600 --> 00:06:03,420
+All right.
+
+119
+00:06:03,810 --> 00:06:06,500
+So we initialize a webcam here.
+
+120
+00:06:06,510 --> 00:06:10,940
+Standard stuff we said is if you've seen before we get this.
+
+121
+00:06:11,070 --> 00:06:15,840
+Now what I've done I've made some changes to this code because the previous code didn't tell you.
+
+122
+00:06:15,900 --> 00:06:21,780
+But if a second person appeared in the diagram it would only show one if you guys are the time it doesn't
+
+123
+00:06:21,780 --> 00:06:23,210
+show more than one face.
+
+124
+00:06:23,360 --> 00:06:25,380
+Oh it may have him trash.
+
+125
+00:06:25,410 --> 00:06:27,380
+I'm not even sure you can do it.
+
+126
+00:06:27,390 --> 00:06:28,360
+Bring a friend.
+
+127
+00:06:28,730 --> 00:06:29,040
+OK.
+
+128
+00:06:29,130 --> 00:06:31,250
+I have no friends with me right right now the wife is out.
+
+129
+00:06:31,320 --> 00:06:32,370
+So I can't test that.
+
+130
+00:06:32,640 --> 00:06:33,970
+But that's OK.
+
+131
+00:06:33,990 --> 00:06:37,910
+So this is how actually got it to work with multiple faces.
+
+132
+00:06:37,920 --> 00:06:40,340
+So we do this here.
+
+133
+00:06:41,130 --> 00:06:47,430
+Basically we append the faces we extract here and then what we do we use his model that we loaded here
+
+134
+00:06:48,050 --> 00:06:51,200
+to make a prediction and this prediction gives us two things.
+
+135
+00:06:51,220 --> 00:06:54,450
+Gender first very male female.
+
+136
+00:06:54,540 --> 00:06:59,640
+I know there are many agendas right now but for now is history in the male and female because the genders
+
+137
+00:07:00,070 --> 00:07:03,270
+will still look like male or female either way.
+
+138
+00:07:03,270 --> 00:07:04,070
+So that's fine.
+
+139
+00:07:04,340 --> 00:07:04,930
+All right.
+
+140
+00:07:05,920 --> 00:07:08,600
+And so we have predicted ages here as well.
+
+141
+00:07:08,890 --> 00:07:12,580
+And this is all in a bit of a funny shape here.
+
+142
+00:07:12,820 --> 00:07:14,380
+So he does some reshipping.
+
+143
+00:07:14,400 --> 00:07:14,730
+All right.
+
+144
+00:07:14,770 --> 00:07:15,640
+And flattens it.
+
+145
+00:07:15,680 --> 00:07:17,380
+And then we get the results here.
+
+146
+00:07:17,380 --> 00:07:20,430
+So we get the predicted ages and this is good.
+
+147
+00:07:20,520 --> 00:07:20,900
+All right.
+
+148
+00:07:20,920 --> 00:07:21,470
+So we get it.
+
+149
+00:07:21,470 --> 00:07:28,270
+No no what we're going to do since we have multiple we possibly have multiple cases in this file is
+
+150
+00:07:28,270 --> 00:07:34,850
+that we go through defaces and go through the results and display the results later on here.
+
+151
+00:07:35,380 --> 00:07:36,170
+OK.
+
+152
+00:07:36,700 --> 00:07:38,250
+And that is it.
+
+153
+00:07:38,260 --> 00:07:40,110
+This is what runs forward.
+
+154
+00:07:40,120 --> 00:07:41,290
+Age and gender.
+
+155
+00:07:41,620 --> 00:07:42,930
+So let's give it a try.
+
+156
+00:07:47,660 --> 00:07:51,400
+Slutting probably a learning model.
+
+157
+00:07:51,410 --> 00:07:52,280
+There we go.
+
+158
+00:07:52,310 --> 00:07:53,950
+So that is pretty awesome.
+
+159
+00:07:54,260 --> 00:08:00,950
+All right what I'm going to do going to bring up a picture of my funny face and try to simulate this
+
+160
+00:08:01,010 --> 00:08:05,960
+with another person or a pretend person we're going to use
+
+161
+00:08:10,180 --> 00:08:12,540
+I guess is actually hard and I taught
+
+162
+00:08:17,740 --> 00:08:18,030
+it.
+
+163
+00:08:18,120 --> 00:08:24,160
+So my friend here she has some good face pictures.
+
+164
+00:08:24,270 --> 00:08:27,200
+I hope she's OK with me using this right.
+
+165
+00:08:27,600 --> 00:08:30,700
+OK so I'm going to bring this up here.
+
+166
+00:08:36,270 --> 00:08:36,560
+OK.
+
+167
+00:08:36,600 --> 00:08:41,530
+So this is me this is my friend is not detecting.
+
+168
+00:08:41,550 --> 00:08:42,200
+Oh there we go.
+
+169
+00:08:43,220 --> 00:08:44,630
+Thinks she's a guy.
+
+170
+00:08:44,680 --> 00:08:45,790
+I have no idea.
+
+171
+00:08:45,820 --> 00:08:47,470
+She doesn't need any of this.
+
+172
+00:08:47,660 --> 00:08:49,210
+And so I think she's a guy.
+
+173
+00:08:49,350 --> 00:08:52,360
+Now there's a problem here and I hope you can see it.
+
+174
+00:08:52,480 --> 00:08:55,630
+And I did actually see it in the code and didn't mention it to you guys because I was like wondering
+
+175
+00:08:56,260 --> 00:08:59,690
+how it doesn't work the way it does supposed to doesn't it.
+
+176
+00:08:59,920 --> 00:09:06,790
+And that's because exactly as I said it will detect the faces here but it only draws results for one
+
+177
+00:09:06,790 --> 00:09:09,760
+prediction here because it's not a pending results results.
+
+178
+00:09:09,760 --> 00:09:12,700
+In theory this is something I fix later on.
+
+179
+00:09:12,700 --> 00:09:15,240
+By the way so for now though just be aware.
+
+180
+00:09:15,250 --> 00:09:21,800
+At this age and gender detector only technically detects one face at a time not two.
+
+181
+00:09:22,240 --> 00:09:26,020
+And basically this is something I also noted here.
+
+182
+00:09:26,320 --> 00:09:27,850
+If you get this early.
+
+183
+00:09:27,970 --> 00:09:30,900
+This is the area you'll see here faces not being detected.
+
+184
+00:09:31,120 --> 00:09:33,430
+It's because your webcam has not been turned on.
+
+185
+00:09:33,430 --> 00:09:41,260
+So basically if this happened this happens just go to devices and webcams and tick tick tick off.
+
+186
+00:09:41,320 --> 00:09:41,890
+OK.
+
+187
+00:09:42,520 --> 00:09:43,250
+So that is that.
+
+188
+00:09:43,270 --> 00:09:48,790
+And if whatever reason this program crashes when you're messing with the code and the webcam isn't released
+
+189
+00:09:49,060 --> 00:09:55,120
+that will happen quite often I'm sure just run this line separately to reclaim your webcam so that you
+
+190
+00:09:55,120 --> 00:09:58,220
+can re-initialize it again later on.
+
+191
+00:09:58,240 --> 00:09:58,570
+OK.
+
+192
+00:09:58,660 --> 00:10:03,410
+So no that was age and gender and a Eating point tree.
+
+193
+00:10:03,640 --> 00:10:04,330
+Now let's do.
+
+194
+00:10:04,330 --> 00:10:06,630
+Age and gender with emotions.
+
+195
+00:10:06,640 --> 00:10:08,430
+That's the cool part.
+
+196
+00:10:08,680 --> 00:10:10,800
+That's deep surveillance partner.
+
+197
+00:10:10,810 --> 00:10:18,100
+So again let's load all of this 10 seconds wasted of my life again.
+
+198
+00:10:19,810 --> 00:10:24,610
+Yoga seemed faster this time maybe me complain.
+
+199
+00:10:25,130 --> 00:10:25,460
+OK.
+
+200
+00:10:25,550 --> 00:10:30,290
+So no this is testing emotion agent and using a webcam.
+
+201
+00:10:30,430 --> 00:10:37,350
+Now I'm not going to go through this code in super detail because it is a bit messy.
+
+202
+00:10:37,880 --> 00:10:43,640
+However what I'm going to tell you is that I have manipulated this code to support two faces.
+
+203
+00:10:43,640 --> 00:10:44,590
+All right.
+
+204
+00:10:44,720 --> 00:10:46,800
+So let's give this ago.
+
+205
+00:10:47,270 --> 00:10:49,880
+Bring up my friend's Instagram picture again
+
+206
+00:10:54,990 --> 00:10:55,330
+OK.
+
+207
+00:10:55,360 --> 00:10:57,490
+So we got an error.
+
+208
+00:10:57,610 --> 00:10:59,050
+Color is not defined.
+
+209
+00:10:59,210 --> 00:11:02,290
+Like I'm going to pause this and see what is happening here.
+
+210
+00:11:02,770 --> 00:11:04,430
+I know exactly what's happening.
+
+211
+00:11:04,480 --> 00:11:07,040
+I was trying to do some color manipulation before.
+
+212
+00:11:07,260 --> 00:11:07,540
+OK.
+
+213
+00:11:07,610 --> 00:11:12,140
+CBG are Spudis get back green OK.
+
+214
+00:11:12,240 --> 00:11:13,430
+It is going to.
+
+215
+00:11:13,700 --> 00:11:18,780
+Oh dammit what happened is that it claimed my webcam.
+
+216
+00:11:18,940 --> 00:11:21,900
+It's not going to get an image.
+
+217
+00:11:21,910 --> 00:11:27,850
+So what we can do as I mentioned before and I did not put it in this file accessible area just put a
+
+218
+00:11:27,850 --> 00:11:34,690
+cell below and reclean my webcam so I can run this finally again
+
+219
+00:11:37,860 --> 00:11:38,720
+I attempted it.
+
+220
+00:11:38,750 --> 00:11:39,300
+There we go.
+
+221
+00:11:40,460 --> 00:11:42,150
+So yeah I'm not actually sad.
+
+222
+00:11:42,260 --> 00:11:44,710
+I don't know why it's tinks I'm sad no neutral.
+
+223
+00:11:45,060 --> 00:11:50,230
+And bring up the multiple face Gladys's Merde is confusing.
+
+224
+00:11:52,290 --> 00:11:55,380
+At still in.
+
+225
+00:11:55,400 --> 00:11:58,780
+Why I doing a in front of me.
+
+226
+00:12:00,900 --> 00:12:01,850
+All right.
+
+227
+00:12:01,910 --> 00:12:03,040
+Come on detect
+
+228
+00:12:12,320 --> 00:12:13,630
+get a brief second.
+
+229
+00:12:13,700 --> 00:12:15,680
+But let me just bring this.
+
+230
+00:12:15,760 --> 00:12:18,800
+This is much easier when you actually have a real person next to you.
+
+231
+00:12:21,690 --> 00:12:22,220
+OK.
+
+232
+00:12:22,420 --> 00:12:24,430
+I'm going to pause this and get a better picture.
+
+233
+00:12:24,430 --> 00:12:25,270
+No offense.
+
+234
+00:12:25,430 --> 00:12:27,920
+This picture has not been affected.
+
+235
+00:12:28,480 --> 00:12:28,850
+OK.
+
+236
+00:12:28,990 --> 00:12:37,920
+So hold one second because the recording that really put it on as fast forward this bit while I just
+
+237
+00:12:37,920 --> 00:12:44,240
+waste time like.
+
+238
+00:12:44,410 --> 00:12:44,700
+All right.
+
+239
+00:12:44,710 --> 00:12:52,290
+So I found a picture of my wife I may use from a Facebook profile maybe I should use like stock images.
+
+240
+00:12:52,290 --> 00:12:55,610
+It seems like these images don't have the best luck but we'll never know.
+
+241
+00:12:55,610 --> 00:12:55,820
+All right.
+
+242
+00:12:55,820 --> 00:12:59,060
+So let's put it in front here.
+
+243
+00:12:59,240 --> 00:13:08,560
+I just saw a detective come back from back in says I actually thought it looks because now I can see
+
+244
+00:13:08,560 --> 00:13:09,320
+my screen.
+
+245
+00:13:10,210 --> 00:13:14,590
+And I can see my face and it's not detecting my face.
+
+246
+00:13:14,590 --> 00:13:16,370
+Maybe I should just bring this back slightly.
+
+247
+00:13:16,380 --> 00:13:17,320
+There we go.
+
+248
+00:13:17,830 --> 00:13:18,230
+OK.
+
+249
+00:13:18,370 --> 00:13:24,670
+So I know my wife is going to be happy with this but I think she's a guy who is 24 years old.
+
+250
+00:13:24,940 --> 00:13:31,160
+Probably because it's from a cell phone and the skill is definitely going to be off because of it.
+
+251
+00:13:31,450 --> 00:13:33,930
+I'm 28 when I'm 24.
+
+252
+00:13:34,330 --> 00:13:35,400
+Damn you.
+
+253
+00:13:35,500 --> 00:13:37,230
+Sorry to deduct it.
+
+254
+00:13:37,570 --> 00:13:37,830
+OK.
+
+255
+00:13:37,870 --> 00:13:40,620
+When it starts kind of to kind of going to hold.
+
+256
+00:13:40,660 --> 00:13:45,940
+All right so what this means here is that this code works.
+
+257
+00:13:45,940 --> 00:13:52,420
+It's not exactly 100 percent accurate with gender detection as we can see but that probably has to do
+
+258
+00:13:52,420 --> 00:13:53,590
+with my little fake friends.
+
+259
+00:13:53,590 --> 00:13:55,210
+I'm using my cell phone.
+
+260
+00:13:55,240 --> 00:13:57,010
+These people are REAL by the way.
+
+261
+00:13:57,380 --> 00:14:02,790
+But either way this should not happen that way but it works it works.
+
+262
+00:14:02,800 --> 00:14:06,810
+It's good that is directing multiple pictures multiple people.
+
+263
+00:14:06,820 --> 00:14:07,120
+All right.
+
+264
+00:14:07,120 --> 00:14:09,500
+So this is good.
+
+265
+00:14:09,550 --> 00:14:13,680
+Let's move on to the next one images so let's run this
+
+266
+00:14:19,570 --> 00:14:23,060
+OK so now let's run this test on images.
+
+267
+00:14:23,070 --> 00:14:23,560
+OK.
+
+268
+00:14:23,560 --> 00:14:27,300
+You did not load anything in my bed
+
+269
+00:14:30,260 --> 00:14:31,460
+not that or
+
+270
+00:14:38,140 --> 00:14:38,620
+OK.
+
+271
+00:14:38,630 --> 00:14:44,480
+So let's test this on some test images and you can place those images in the images folder.
+
+272
+00:14:44,930 --> 00:14:45,550
+Will show you here.
+
+273
+00:14:45,590 --> 00:14:45,910
+OK.
+
+274
+00:14:45,980 --> 00:14:49,840
+So let's look at Donald Trump oh this is quite funny.
+
+275
+00:14:49,950 --> 00:14:57,440
+I think it's a female 59 looks much older in my opinion but fifty nine and said It's me again.
+
+276
+00:14:57,460 --> 00:15:01,050
+I did some extra years to me because I'm actually actually 24.
+
+277
+00:15:01,140 --> 00:15:02,310
+But fair enough.
+
+278
+00:15:03,940 --> 00:15:04,330
+All right.
+
+279
+00:15:04,330 --> 00:15:06,560
+Queen Elizabeth and something funny is happening.
+
+280
+00:15:06,640 --> 00:15:10,930
+It actually thinks this bouquet of roses is a guy who took the ideas off.
+
+281
+00:15:11,170 --> 00:15:16,360
+Clearly this is a bad example of Mitt not working too well.
+
+282
+00:15:16,840 --> 00:15:17,710
+But there's a reason for that.
+
+283
+00:15:17,720 --> 00:15:18,870
+I'll tell you afterwards.
+
+284
+00:15:18,910 --> 00:15:19,800
+OK.
+
+285
+00:15:20,470 --> 00:15:21,270
+Barack Obama.
+
+286
+00:15:21,460 --> 00:15:24,260
+Male angry to the one not the best.
+
+287
+00:15:24,400 --> 00:15:25,480
+Is my wife.
+
+288
+00:15:25,480 --> 00:15:30,870
+Female tity she actually was tastiness Richardsons quite good and she was definitely happy here.
+
+289
+00:15:31,100 --> 00:15:31,590
+All right.
+
+290
+00:15:31,660 --> 00:15:35,350
+She was actually this was the two days of the pictures were two days apart.
+
+291
+00:15:35,380 --> 00:15:37,860
+So she aged a year in those two days.
+
+292
+00:15:38,260 --> 00:15:38,560
+OK.
+
+293
+00:15:38,590 --> 00:15:40,040
+So that's cool.
+
+294
+00:15:40,240 --> 00:15:41,750
+Now I said I'll tell you why.
+
+295
+00:15:42,010 --> 00:15:49,350
+That's because when we were doing this here we were basically Horncastle pacifies don't crop enough
+
+296
+00:15:49,350 --> 00:15:55,090
+of the face out even with the default settings here which are removed because you may have seen it in
+
+297
+00:15:55,100 --> 00:16:01,710
+a court because it just cut and paste some of these images discotheques or videos before we actually
+
+298
+00:16:01,710 --> 00:16:04,070
+had some cropping being done.
+
+299
+00:16:04,140 --> 00:16:07,080
+And I took it out because it was less accurate.
+
+300
+00:16:07,410 --> 00:16:10,350
+But now anyway so you libs facial recognition.
+
+301
+00:16:10,350 --> 00:16:12,380
+Let's give this a try.
+
+302
+00:16:12,390 --> 00:16:13,470
+Let's quickly run this
+
+303
+00:16:17,750 --> 00:16:19,830
+and I'll tell you about dealing with starting in the back row here.
+
+304
+00:16:19,850 --> 00:16:21,180
+I'll tell you about dealer dealer.
+
+305
+00:16:21,220 --> 00:16:27,460
+It is it's a machine learning basically package that was built in C++ and you can use it into Python
+
+306
+00:16:27,800 --> 00:16:29,210
+and it does a bunch of cool stuff.
+
+307
+00:16:29,210 --> 00:16:34,430
+And what I'm using it here for is for phase detection but it actually can do facial recognition.
+
+308
+00:16:34,460 --> 00:16:39,480
+In my open C-v course in the past I haven't included in discourse.
+
+309
+00:16:39,590 --> 00:16:47,170
+You actually do use Nonpoint recognition on faces for some cool projects like your detection and the
+
+310
+00:16:47,540 --> 00:16:50,400
+swaps as well more advanced with swaps.
+
+311
+00:16:50,390 --> 00:16:50,680
+All right.
+
+312
+00:16:50,740 --> 00:16:53,660
+So I'm sorry I'm retarded.
+
+313
+00:16:53,780 --> 00:16:55,690
+We are using the live on images here.
+
+314
+00:16:55,700 --> 00:17:00,110
+I really should label this that I think in the last section used to live on the webcam.
+
+315
+00:17:00,140 --> 00:17:01,120
+Yes.
+
+316
+00:17:01,250 --> 00:17:01,970
+OK.
+
+317
+00:17:02,120 --> 00:17:02,980
+So again.
+
+318
+00:17:03,100 --> 00:17:04,280
+So it actually is.
+
+319
+00:17:04,340 --> 00:17:08,820
+Now a trump he actually aged two years less fend off me.
+
+320
+00:17:08,980 --> 00:17:09,420
+Oh.
+
+321
+00:17:09,430 --> 00:17:10,790
+You know when you're younger.
+
+322
+00:17:10,790 --> 00:17:11,900
+That's good.
+
+323
+00:17:12,830 --> 00:17:14,130
+Fifty four female OK.
+
+324
+00:17:14,200 --> 00:17:14,980
+Fair enough.
+
+325
+00:17:14,980 --> 00:17:16,420
+She has a lot.
+
+326
+00:17:16,610 --> 00:17:18,030
+In that picture.
+
+327
+00:17:18,580 --> 00:17:20,200
+That's fine.
+
+328
+00:17:20,290 --> 00:17:21,500
+Tony correct.
+
+329
+00:17:21,970 --> 00:17:27,220
+Barack Obama 45 I think he was in his late 40s and this picture may be very very accurate actually is
+
+330
+00:17:27,220 --> 00:17:28,760
+quite close.
+
+331
+00:17:29,180 --> 00:17:35,590
+And my wife she would not be happy with each year but it's fair enough again because she was Tuti in
+
+332
+00:17:35,590 --> 00:17:36,670
+those pictures.
+
+333
+00:17:37,000 --> 00:17:37,950
+OK but fair enough.
+
+334
+00:17:37,950 --> 00:17:39,860
+So you can experiment with some different things.
+
+335
+00:17:39,860 --> 00:17:41,500
+You know it's not going to be perfect.
+
+336
+00:17:41,500 --> 00:17:45,440
+Age is actually a very hard thing to guess.
+
+337
+00:17:45,460 --> 00:17:52,870
+I myself sometimes have thought someone was to be in it today 25 or 26 and I learned later they were
+
+338
+00:17:52,870 --> 00:17:59,130
+like 50 which was embarrassing but they were very happy about my mis take.
+
+339
+00:17:59,140 --> 00:18:04,170
+So anyway we can run this with the lab using the webcam so I'm going on this.
+
+340
+00:18:04,300 --> 00:18:09,660
+And I've changed my T-shirt by the way back to the one with stain unfortunately.
+
+341
+00:18:09,910 --> 00:18:12,340
+Hope it isn't picked up in this camera.
+
+342
+00:18:12,340 --> 00:18:14,070
+No it does not.
+
+343
+00:18:14,380 --> 00:18:15,620
+And the bedroom doesn't open.
+
+344
+00:18:15,640 --> 00:18:16,660
+Yeah it's fine.
+
+345
+00:18:16,960 --> 00:18:18,280
+So again this is quite cool.
+
+346
+00:18:18,280 --> 00:18:22,570
+You can definitely see dealer has a slew of film right here OK.
+
+347
+00:18:22,940 --> 00:18:27,290
+And literally did my hair thing five seconds 10 seconds before.
+
+348
+00:18:27,610 --> 00:18:27,970
+OK.
+
+349
+00:18:28,120 --> 00:18:29,820
+So close.
+
+350
+00:18:30,340 --> 00:18:35,020
+So the advantage of the lid though is that it is better at picking up faces.
+
+351
+00:18:35,050 --> 00:18:40,600
+One thing you should have noted noted here is that it didn't mystic Queen Elizabeth's bouquet for a
+
+352
+00:18:40,600 --> 00:18:43,610
+face like what a hawk husky classified did.
+
+353
+00:18:43,870 --> 00:18:47,170
+So you can generally see that as better and more robust.
+
+354
+00:18:47,180 --> 00:18:53,650
+Remember for Cutie Caskie classifies it or this was a face that it was a guy which I would think it
+
+355
+00:18:53,640 --> 00:18:54,240
+was a female.
+
+356
+00:18:54,310 --> 00:18:57,660
+Either way if it was a face that is.
+
+357
+00:18:58,270 --> 00:19:00,340
+But that's fine.
+
+358
+00:19:00,340 --> 00:19:04,650
+So what I'm saying Bakassi cast philosophise a definitely faster.
+
+359
+00:19:04,920 --> 00:19:09,850
+So if speed is your concern or if your hardware if it's an embedded system you can use Hawkhurst justifies
+
+360
+00:19:10,300 --> 00:19:16,920
+and maybe tweak some of the parameters like this scaling parameters especially this definitely helps.
+
+361
+00:19:16,960 --> 00:19:23,770
+But these settings are generally well understood as the gold standard Woodhall Cascades like this is
+
+362
+00:19:23,780 --> 00:19:27,330
+the best depending on your application.
+
+363
+00:19:27,350 --> 00:19:29,020
+But generally this is the best.
+
+364
+00:19:29,500 --> 00:19:35,590
+So a dealer may be more suited for some applications if its accuracy especially is of importance.
+
+365
+00:19:35,590 --> 00:19:37,540
+OK so that's it for this chapter.
+
+366
+00:19:37,540 --> 00:19:42,160
+I hope you enjoyed it and I hope you build something cool out of this and I hope you expand upon it
+
+367
+00:19:42,160 --> 00:19:42,490
+too.
+
+368
+00:19:42,510 --> 00:19:47,980
+You can actually take this code now and train and train over these models or add in some new stuff as
+
+369
+00:19:47,980 --> 00:19:48,370
+well.
+
+370
+00:19:48,490 --> 00:19:50,010
+And ethnicity as well.
+
+371
+00:19:50,020 --> 00:19:57,380
+You can actually now take some of this data sets here like this one for one and maybe you can if you
+
+372
+00:19:57,380 --> 00:20:02,410
+can find like a label that you can manually label the agenda from this as well to get even more accurate
+
+373
+00:20:02,470 --> 00:20:04,060
+gender recognition.
+
+374
+00:20:04,060 --> 00:20:05,310
+So this is pretty cool.
+
+375
+00:20:05,350 --> 00:20:12,850
+And the challenge and in bringing both of these together which I probably failed to mention is that
+
+376
+00:20:13,530 --> 00:20:16,260
+they're both taking different sized faces to boot.
+
+377
+00:20:16,270 --> 00:20:18,940
+One is ticking and ticking in a color image.
+
+378
+00:20:19,120 --> 00:20:20,790
+That's the age and gender sticking.
+
+379
+00:20:20,820 --> 00:20:26,730
+Taking in a color image that's 64 by 64 whereas the other one which is all in motion detectors ticking
+
+380
+00:20:26,800 --> 00:20:30,410
+in a smaller and gray scale image.
+
+381
+00:20:30,460 --> 00:20:34,780
+So you have to sort of do some a little more processing here in our pipeline.
+
+382
+00:20:35,020 --> 00:20:41,180
+And basically then you have to get the results and make sure they're lined up like the same face is
+
+383
+00:20:41,200 --> 00:20:47,590
+tied together with his agent and the emotion which is fairly easy anyway and then plus and take it correctly
+
+384
+00:20:47,590 --> 00:20:51,480
+and then making sure the labels follow the face around the image.
+
+385
+00:20:51,520 --> 00:20:54,390
+So it was a fun piece of code to build.
+
+386
+00:20:54,400 --> 00:20:55,270
+To be fair.
+
+387
+00:20:55,470 --> 00:20:55,760
+But.

18. Facial Applications - Emotion, Age & Gender Recognition/3.1 Download weights file.html

@@ -0,0 +1 @@
+<script type="text/javascript">window.location = "https://drive.google.com/file/d/17kyPQfUyk2un-d-XYFT54R0q9vvSxbuJ/view?usp=sharing";</script>

18. Facial Applications - Emotion, Age & Gender Recognition/3.2 Code and files required for project.html

@@ -0,0 +1 @@
+<script type="text/javascript">window.location = "https://drive.google.com/file/d/1lI_gZM9QuxjyRIKGvOm63d1td5lgRz4i/view?usp=sharing";</script>

19. Medical Imaging - Image Segmentation with U-Net/1. Chapter Overview on Image Segmentation & Medical Imaging in U-Net.srt

@@ -0,0 +1,31 @@
+1
+00:00:00,400 --> 00:00:00,750
+OK.
+
+2
+00:00:00,810 --> 00:00:06,330
+So let's move on to image segmentation and I'll talk a bit about medical imaging and the unit which
+
+3
+00:00:06,330 --> 00:00:10,000
+is a very cool CNN that does image segmentation.
+
+4
+00:00:10,060 --> 00:00:15,560
+So this section is built up into four parts Firstly discuss what is segmentation Exactly.
+
+5
+00:00:15,820 --> 00:00:19,300
+And I provide some examples of applications in medical imaging.
+
+6
+00:00:19,480 --> 00:00:25,720
+Dennis are talking about units and how it applies to image segmentation and CNN's Then I did find some
+
+7
+00:00:25,720 --> 00:00:30,310
+units so we are going to need to know about which is the intersection of union metric and then we do
+
+8
+00:00:30,310 --> 00:00:35,580
+a final project in the shop too where we find a nuclear nuclei and I frequent images.

19. Medical Imaging - Image Segmentation with U-Net/2. What is Segmentation And Applications in Medical Imaging.srt

@@ -0,0 +1,215 @@
+1
+00:00:00,960 --> 00:00:07,350
+Hi and welcome to chapter or section nineteen point one where we talk about segmentation and its applications
+
+2
+00:00:07,350 --> 00:00:09,450
+in medical imaging.
+
+3
+00:00:09,520 --> 00:00:12,280
+So what exactly is image segmentation.
+
+4
+00:00:12,280 --> 00:00:18,040
+So the goal of segmentation is to separate different parts of an image into sensible coherent parts.
+
+5
+00:00:18,040 --> 00:00:19,740
+And what do I mean by that.
+
+6
+00:00:19,810 --> 00:00:22,500
+Basically you see this cat and this dog.
+
+7
+00:00:22,540 --> 00:00:26,490
+And then there's a background definitely in the back to a bed or couch.
+
+8
+00:00:26,680 --> 00:00:28,600
+And then there are some kittens in the back.
+
+9
+00:00:28,600 --> 00:00:34,400
+What we want in a window actually what we want is basically to do pixel level predictions.
+
+10
+00:00:34,480 --> 00:00:39,640
+So we want to actually know exactly what pixels here belong to the dog what belongs to the cat and what
+
+11
+00:00:39,640 --> 00:00:41,510
+belong to the objects in the background.
+
+12
+00:00:41,860 --> 00:00:45,620
+So as you can see it's a pretty challenging task because before.
+
+13
+00:00:45,850 --> 00:00:50,140
+Well what we were doing was basically a prediction of the entire image.
+
+14
+00:00:50,140 --> 00:00:55,360
+And basically if we fed this declassify it would probably give a high probability of what cat and the
+
+15
+00:00:55,360 --> 00:00:57,080
+dog being in the picture.
+
+16
+00:00:57,430 --> 00:01:00,540
+But now what we want to do is actually segment the image now.
+
+17
+00:01:00,610 --> 00:01:02,400
+So how do we go about doing this.
+
+18
+00:01:02,530 --> 00:01:06,800
+And before you even explain that let's talk about two types of segmentation.
+
+19
+00:01:08,510 --> 00:01:14,510
+So the first type is called semantic segmentation and basically this is just pixel level predictions
+
+20
+00:01:15,260 --> 00:01:16,870
+based on defined classes.
+
+21
+00:01:16,880 --> 00:01:19,740
+So we have example Rood's persons cause entry.
+
+22
+00:01:19,910 --> 00:01:26,720
+So it's simple enough to understand not simple to do but we can pretty much see how it's done here and
+
+23
+00:01:26,720 --> 00:01:32,900
+you can imagine this has a lot of application in self-driving cars because now you need to know what
+
+24
+00:01:32,900 --> 00:01:37,470
+is a road what is a building what are like ampoules people those sorts of things.
+
+25
+00:01:37,730 --> 00:01:40,200
+So it's going to be quite useful for that application.
+
+26
+00:01:41,730 --> 00:01:42,420
+Type 2.
+
+27
+00:01:42,460 --> 00:01:44,610
+So no we're doing two different things.
+
+28
+00:01:44,640 --> 00:01:49,890
+We're doing pixel level predictions which is exactly what we did before but now we're actually doing
+
+29
+00:01:50,010 --> 00:01:54,340
+object detection and actually object identification as well.
+
+30
+00:01:54,480 --> 00:01:57,290
+So we know Person 1 to call 1 and 2.
+
+31
+00:01:57,330 --> 00:02:03,050
+So this is a more advanced level of segmentation.
+
+32
+00:02:03,120 --> 00:02:05,870
+So let's talk a bit about applications and medical imaging.
+
+33
+00:02:06,090 --> 00:02:12,630
+So as you know a lot of medical imaging necessitates finding and accurately labeling basically things
+
+34
+00:02:12,630 --> 00:02:19,080
+we find in these scans because if you've seen a lot of these medical imaging scans are very very hard
+
+35
+00:02:19,080 --> 00:02:19,720
+to interpret.
+
+36
+00:02:19,740 --> 00:02:24,870
+And I think you need some very skilled professionals analyzing and accurately assessing what they see
+
+37
+00:02:24,990 --> 00:02:31,340
+in those pictures generated from the different scans and basically Often this task is actually there.
+
+38
+00:02:31,370 --> 00:02:34,810
+There is a lot of advance software being used in these tests.
+
+39
+00:02:35,010 --> 00:02:40,920
+However they still require a human to actually go through it and maybe label things properly so that
+
+40
+00:02:41,010 --> 00:02:42,770
+you know the machine isn't alone.
+
+41
+00:02:42,780 --> 00:02:43,360
+All right.
+
+42
+00:02:43,410 --> 00:02:48,420
+However this is definitely definitely a task where you can improve it because humans are definitely
+
+43
+00:02:48,420 --> 00:02:51,750
+prone to error and there are a number of applications mainly to him.
+
+44
+00:02:51,820 --> 00:02:58,380
+And I don't mean that are being done by convolutional your own that's an advanced neural nets.
+
+45
+00:02:58,620 --> 00:03:04,910
+So as I said is a huge initiative to use can be division and planning to automate many of these tests.
+
+46
+00:03:04,980 --> 00:03:11,040
+So it's a lot of tests that can be improved with computer vision not just her but surgery which is actually
+
+47
+00:03:11,040 --> 00:03:15,230
+going to be a main application in the future for the division.
+
+48
+00:03:15,240 --> 00:03:21,450
+However right now the trend seems to be in a lot of these medical scans things like CAT scans X-rays
+
+49
+00:03:21,600 --> 00:03:25,060
+ultrasounds PET scans and an MRI.
+
+50
+00:03:25,410 --> 00:03:29,590
+And so many different types of diseases to look for it.
+
+51
+00:03:29,610 --> 00:03:37,290
+This is an ideal area of startups to take advantage of and the use cases for this are endless from cancer
+
+52
+00:03:37,290 --> 00:03:42,510
+detection disease monitoring Alzheimer's and many many other ailments.
+
+53
+00:03:42,540 --> 00:03:48,270
+So computer vision can definitely revolutionize the medical industry and improve patient care and get
+
+54
+00:03:48,270 --> 00:03:52,460
+much faster diagnostics and even be used to find cures much faster.

19. Medical Imaging - Image Segmentation with U-Net/3. U-Net Image Segmentation with CNNs.srt

@@ -0,0 +1,203 @@
+1
+00:00:00,370 --> 00:00:00,930
+OK.
+
+2
+00:00:00,960 --> 00:00:02,840
+So let's talk about usenet.
+
+3
+00:00:02,940 --> 00:00:06,060
+Usenet is a CNN that can actually do image segmentation.
+
+4
+00:00:06,060 --> 00:00:08,500
+So let's see how it works.
+
+5
+00:00:08,550 --> 00:00:15,570
+Usenet was created in 2015 and it basically was a CNN specifically developed for the biomedical image
+
+6
+00:00:15,630 --> 00:00:20,290
+segmentation task which is what we're going to use it for in our project at the end of this chapter.
+
+7
+00:00:20,580 --> 00:00:26,140
+You know it has now become very popular for entend included decoded type networks for semantics of mentation.
+
+8
+00:00:26,340 --> 00:00:28,150
+And it has a very unique architecture.
+
+9
+00:00:28,220 --> 00:00:33,570
+Called updown architecture which has a contracting part an expensive part and you'll see it's in the
+
+10
+00:00:33,570 --> 00:00:39,120
+diagram here where you can see this is how it starts the input image comes in and basically it goes
+
+11
+00:00:39,120 --> 00:00:41,030
+down to the contracting part here.
+
+12
+00:00:41,460 --> 00:00:47,490
+And then there's this middle area here that's called the lead story to the expensive part.
+
+13
+00:00:47,490 --> 00:00:47,990
+OK.
+
+14
+00:00:48,420 --> 00:00:50,110
+And basically the opposite here.
+
+15
+00:00:50,430 --> 00:00:53,140
+So first things first take a look at us.
+
+16
+00:00:53,280 --> 00:00:59,760
+We have an input image here and it's outputting an image or basically a segmentation map that's effectively
+
+17
+00:00:59,760 --> 00:01:03,270
+an image because we're going to use it to segment the image.
+
+18
+00:01:03,300 --> 00:01:09,430
+So this is a model label here the contracting part of an expensive part.
+
+19
+00:01:09,520 --> 00:01:14,880
+So yes I wanted it caught wanted to talk to you about was a bottle looking area here.
+
+20
+00:01:15,080 --> 00:01:15,710
+OK.
+
+21
+00:01:16,040 --> 00:01:22,090
+So this is how unit structure works is a down sample a bottleneck and then assemble the downsampling
+
+22
+00:01:22,130 --> 00:01:24,660
+part and unit consist of four blocks.
+
+23
+00:01:24,690 --> 00:01:30,120
+There's these tree by tree convolutional Lia's with all these really special motion and troppo to use.
+
+24
+00:01:30,170 --> 00:01:32,600
+And this is basically Fort Lee is here.
+
+25
+00:01:32,840 --> 00:01:39,440
+So it is these two kindly as here two by two max pooling and then the feature maps double as we go down
+
+26
+00:01:40,100 --> 00:01:41,620
+which is not unusual.
+
+27
+00:01:41,630 --> 00:01:48,370
+We've seen it happen in Viji as well as starting up 64 and then going to 128 56 and 512.
+
+28
+00:01:48,470 --> 00:01:55,450
+If you go back to diagram you can see it here 64 128 256 and 512 then it's bottlenecked here.
+
+29
+00:01:55,580 --> 00:01:57,740
+And then we basically do some dung sampling.
+
+30
+00:01:57,740 --> 00:02:03,540
+Again going back here to get output segmentation map sort of bottleneck.
+
+31
+00:02:03,540 --> 00:02:05,020
+Let's talk a bit about this.
+
+32
+00:02:05,030 --> 00:02:09,680
+This consists of two convolutional is what again Bagenal opposition and dropout.
+
+33
+00:02:09,740 --> 00:02:10,290
+OK.
+
+34
+00:02:10,580 --> 00:02:13,740
+Nothing majorly special here just how it works.
+
+35
+00:02:13,750 --> 00:02:16,120
+It doesn't follow a traditional CNN architecture.
+
+36
+00:02:16,240 --> 00:02:17,970
+What is a continuous growing.
+
+37
+00:02:18,110 --> 00:02:24,230
+There's this continuous growing here and then is this bottleneck and this is downsampling here.
+
+38
+00:02:24,490 --> 00:02:29,930
+So the upsampling part now the upsampling part basically consists of a similar pattern.
+
+39
+00:02:30,020 --> 00:02:34,840
+However instead of a convolutional Liya there's something called the deconvolution layer.
+
+40
+00:02:35,150 --> 00:02:39,530
+And then it's concatenated with the future map of the corresponding contracting part.
+
+41
+00:02:39,530 --> 00:02:44,210
+This is basically the major area here that allows you not to work.
+
+42
+00:02:44,210 --> 00:02:49,690
+And basically then it has to convery is here to create the output up here.
+
+43
+00:02:49,880 --> 00:02:55,490
+And if you're wondering what a deconvolution layer is because I mentioned it in this proceeding slide
+
+44
+00:02:56,660 --> 00:03:00,910
+basically it it basically reverses the effects of the convolutional.
+
+45
+00:03:01,190 --> 00:03:06,560
+So just imagine what a convolution of the convolution layer basically apply some transform that image
+
+46
+00:03:06,560 --> 00:03:11,340
+will a deconvolution layer Leo does the same however basically the opposite.
+
+47
+00:03:11,340 --> 00:03:15,840
+So it produces basically the output of a reverse convolutional.
+
+48
+00:03:16,370 --> 00:03:18,020
+Hopefully that makes sense to you.
+
+49
+00:03:18,230 --> 00:03:22,670
+What we're going to do now is going to define some more metrics you need to consider when making an
+
+50
+00:03:22,670 --> 00:03:23,760
+image segmentation.
+
+51
+00:03:23,900 --> 00:03:24,350
+CNN.

19. Medical Imaging - Image Segmentation with U-Net/4. The Intersection over Union (IoU) Metric.srt

@@ -0,0 +1,267 @@
+1
+00:00:00,610 --> 00:00:01,180
+OK.
+
+2
+00:00:01,260 --> 00:00:03,490
+So that brings us to the next chapter.
+
+3
+00:00:05,900 --> 00:00:06,240
+OK.
+
+4
+00:00:06,240 --> 00:00:08,580
+So welcome to chapter nineteen point three.
+
+5
+00:00:08,700 --> 00:00:13,970
+Well we talk about the intersection of the Union now that means song.
+
+6
+00:00:15,480 --> 00:00:20,970
+Hi and welcome to Chapter 19 point tree where we now talk about the intersection of union metric which
+
+7
+00:00:20,970 --> 00:00:24,910
+is an important metric you need to know when we're actually training.
+
+8
+00:00:25,370 --> 00:00:26,550
+So image segmentation.
+
+9
+00:00:26,550 --> 00:00:29,160
+CNN So let's talk about this for us.
+
+10
+00:00:29,190 --> 00:00:29,600
+OK.
+
+11
+00:00:29,970 --> 00:00:32,980
+So let's assume we're doing some object detection here.
+
+12
+00:00:33,240 --> 00:00:37,960
+And basically the green boxes are true bowling box over this nice car.
+
+13
+00:00:38,340 --> 00:00:41,250
+So this is like a human labeled image of what it is.
+
+14
+00:00:41,460 --> 00:00:43,060
+And now our classifier.
+
+15
+00:00:43,140 --> 00:00:46,950
+Oh predicter gave us just bounding boxes red one.
+
+16
+00:00:46,990 --> 00:00:48,930
+So generally it's correct.
+
+17
+00:00:48,930 --> 00:00:50,510
+I wouldn't say it's wrong wrong at all.
+
+18
+00:00:50,520 --> 00:00:55,560
+However you do see that it could have been brought in a lot closer here and maybe brought in here and
+
+19
+00:00:55,560 --> 00:00:56,470
+a bit lower here.
+
+20
+00:00:56,850 --> 00:00:59,590
+So it's a good box but not the best box.
+
+21
+00:01:00,090 --> 00:01:06,020
+So how much of the correct area is covered by a protected bonded box that is this area here.
+
+22
+00:01:06,200 --> 00:01:13,040
+So honestly it seems like about 90 percent of all true blocks is covered by our predicted box.
+
+23
+00:01:13,050 --> 00:01:15,290
+So how do we measure this in a metric.
+
+24
+00:01:15,360 --> 00:01:16,070
+OK.
+
+25
+00:01:16,380 --> 00:01:19,570
+What is a good metric for this what if this was a bounding box.
+
+26
+00:01:19,570 --> 00:01:23,040
+Here it still covers 90 percent of our true books.
+
+27
+00:01:23,070 --> 00:01:27,210
+However this is a much poorer bounding box than this one.
+
+28
+00:01:28,720 --> 00:01:33,070
+This is where the union intersection of the union comes in.
+
+29
+00:01:33,070 --> 00:01:37,590
+So are you going to call it that for short is basically the size of a union.
+
+30
+00:01:37,750 --> 00:01:41,760
+That's the shaded area here over the size of our predicted box.
+
+31
+00:01:41,830 --> 00:01:48,390
+So you can see the size of our predicted box here is much bigger than the size of this predicted box.
+
+32
+00:01:48,400 --> 00:01:50,010
+So what's what does that mean.
+
+33
+00:01:50,050 --> 00:01:56,020
+It means that I you with this box is going to be maybe like point five competitive or you have this
+
+34
+00:01:56,020 --> 00:02:04,060
+box which is going to be probably two point nine so generally do it typically and you have a point five
+
+35
+00:02:04,060 --> 00:02:08,750
+is considered acceptable mainly because optic conduction is very hard to get right.
+
+36
+00:02:08,920 --> 00:02:15,180
+So we do have it's fairly lenient Trishul and obviously the higher the oil you the better the prediction.
+
+37
+00:02:15,490 --> 00:02:21,690
+And essentially I use a measure of overlap how good Basically the overlap is.
+
+38
+00:02:22,090 --> 00:02:27,640
+So before I begin to show you how we actually implemented you you increase and use it as a wonderful
+
+39
+00:02:27,670 --> 00:02:33,190
+metrics we want to so during training I'll tell you why we need this image segmentation.
+
+40
+00:02:33,190 --> 00:02:40,630
+Remember in image segmentation we're basically measuring overlap of like a masked image over the original
+
+41
+00:02:40,630 --> 00:02:41,400
+image.
+
+42
+00:02:41,410 --> 00:02:45,680
+So suppose we're developing a mask that covers this image with just a call here.
+
+43
+00:02:45,760 --> 00:02:47,240
+Forget about the bounding box video.
+
+44
+00:02:47,290 --> 00:02:48,890
+We just want to measure this mask.
+
+45
+00:02:49,300 --> 00:02:50,590
+And what if our.
+
+46
+00:02:50,650 --> 00:02:54,480
+So imagine we have a mask pure yellow mask covering the scar.
+
+47
+00:02:54,850 --> 00:03:00,370
+And imagine we have a predicted mass that is a segmentation algorithm that produced something that covers
+
+48
+00:03:00,430 --> 00:03:02,090
+a blob like this here.
+
+49
+00:03:02,440 --> 00:03:08,290
+How do we measure the effectiveness or basically the accuracy of this mass given that this was the mask
+
+50
+00:03:08,470 --> 00:03:15,370
+correct mask here for this and that is why we are so it is not just useful object reduction is used
+
+51
+00:03:15,370 --> 00:03:18,420
+for mass image masking which is segmentation.
+
+52
+00:03:18,430 --> 00:03:25,010
+So now let's see how we do this in Chris it's pretty easy to find these custom metric functions.
+
+53
+00:03:25,010 --> 00:03:28,810
+In Paris you just have to write a simple function.
+
+54
+00:03:28,830 --> 00:03:37,430
+This takes in a true way year predictive labels we compute and I use school and these labels say I'm
+
+55
+00:03:37,440 --> 00:03:39,720
+just going to be labels are actually going to be mass.
+
+56
+00:03:39,930 --> 00:03:44,550
+And what we do here in when we compile a model we define are one metrics.
+
+57
+00:03:44,550 --> 00:03:46,800
+Previously we used to use accuracy.
+
+58
+00:03:46,800 --> 00:03:50,440
+Now we just use it my own metric which is going to be this function here.
+
+59
+00:03:50,720 --> 00:03:51,270
+All right.
+
+60
+00:03:51,330 --> 00:03:55,100
+And then we train and when we're training we actually see the report here.
+
+61
+00:03:55,290 --> 00:04:00,210
+So we see that my own metric which is dysfunction here technically that should be the same name here
+
+62
+00:04:00,220 --> 00:04:02,420
+just remember that I should have done that for you guys.
+
+63
+00:04:02,430 --> 00:04:03,670
+You don't get confuse.
+
+64
+00:04:04,080 --> 00:04:05,870
+But it's actually going up at a school.
+
+65
+00:04:05,940 --> 00:04:11,380
+So instead of monitoring a loss and accuracy we're now monitoring a loss and a custom function.
+
+66
+00:04:12,000 --> 00:04:12,960
+So that's pretty cool.
+
+67
+00:04:12,960 --> 00:04:15,330
+So let's move on to the next chapter.

19. Medical Imaging - Image Segmentation with U-Net/5. Finding the Nuclei in Divergent Images.srt

@@ -0,0 +1,875 @@
+1
+00:00:00,660 --> 00:00:05,540
+It and welcome back to chapter nineteen point four where we're going to implement is project and project
+
+2
+00:00:05,540 --> 00:00:08,880
+is called Finding the nuclei in divergent images.
+
+3
+00:00:09,050 --> 00:00:10,980
+So let's go on to see a bit about it.
+
+4
+00:00:11,000 --> 00:00:14,820
+So this was part of toggles Science Bowl of 2018.
+
+5
+00:00:15,050 --> 00:00:20,390
+Basically the challenge was to spotting Nicholai to get speed up Achillas that was tackling used and
+
+6
+00:00:20,390 --> 00:00:25,670
+what we wanted to do or what they wanted to wanted to do was automate nucular detection.
+
+7
+00:00:25,670 --> 00:00:28,560
+So we basically perform you'll see the images soon.
+
+8
+00:00:28,840 --> 00:00:33,480
+We're basically performing performing Nicolay detection in these images.
+
+9
+00:00:33,800 --> 00:00:36,360
+And this is basically the writeup they used here.
+
+10
+00:00:36,410 --> 00:00:42,140
+They're designed to cells Nikolai's a starting point for most analysis because most of the human bodies
+
+11
+00:00:42,540 --> 00:00:48,920
+to tity trillion of ocelots cells can t and Nicholas full of DNA the genetic code that programs each
+
+12
+00:00:48,920 --> 00:00:49,780
+cell.
+
+13
+00:00:49,820 --> 00:00:55,880
+So identifying the nuclei allows rescissions to identify each individual cell in a sample and by measuring
+
+14
+00:00:55,910 --> 00:01:01,640
+how many cells are how cells were active various treatments the researchers can now understand the underlying
+
+15
+00:01:01,670 --> 00:01:03,690
+biological processes at work.
+
+16
+00:01:03,770 --> 00:01:09,200
+So you can see this project actually has tremendous application in the medical field.
+
+17
+00:01:09,560 --> 00:01:14,320
+And this was the flyer as well as the tagline I mentioned to you before that Kaggle used advertizes
+
+18
+00:01:14,340 --> 00:01:15,290
+contests.
+
+19
+00:01:15,470 --> 00:01:19,890
+So you can see it is definitely in need a practical need to get this done right.
+
+20
+00:01:21,870 --> 00:01:24,320
+So these were the images in our data set here.
+
+21
+00:01:24,660 --> 00:01:25,910
+Look at the first row here.
+
+22
+00:01:25,980 --> 00:01:29,830
+These four images we are we were given images like this.
+
+23
+00:01:29,850 --> 00:01:34,700
+These dots represent nuclei is and is basically different shades of gray.
+
+24
+00:01:34,730 --> 00:01:36,950
+Like these we're all in the set.
+
+25
+00:01:36,960 --> 00:01:41,900
+We actually have full color images here as well as some basic grayscale images here.
+
+26
+00:01:42,210 --> 00:01:47,760
+So these definitely made it could be you can use some open see the trembling functions and get these
+
+27
+00:01:47,760 --> 00:01:50,780
+mass with a human labeled mass here.
+
+28
+00:01:51,090 --> 00:01:56,550
+However when it comes to these here you definitely need some sort of intelligence to actually extract
+
+29
+00:01:56,550 --> 00:01:59,020
+this and label Nikolai's here.
+
+30
+00:01:59,250 --> 00:02:04,320
+As you can see doing this manually as a human is going to take some time definitely.
+
+31
+00:02:04,350 --> 00:02:08,780
+And then getting the counts of overcomplex images it's going to be an exhaustive tests.
+
+32
+00:02:08,970 --> 00:02:14,250
+So this is the role of two true original images.
+
+33
+00:02:14,260 --> 00:02:15,570
+The true mass.
+
+34
+00:02:15,570 --> 00:02:18,480
+And again more true images and more true mass.
+
+35
+00:02:18,480 --> 00:02:24,960
+So we're going to try and classify to take this image and put this image any image here and produce
+
+36
+00:02:24,960 --> 00:02:26,940
+a mask that looks like this.
+
+37
+00:02:26,940 --> 00:02:33,910
+So our approach is basically to use unit which is a special CNN designed exactly for image segmentation.
+
+38
+00:02:33,910 --> 00:02:35,280
+Tests like this.
+
+39
+00:02:35,280 --> 00:02:36,340
+So let's get started.
+
+40
+00:02:41,170 --> 00:02:41,560
+OK.
+
+41
+00:02:41,630 --> 00:02:47,630
+So we're back to a virtual machine and we're going to use you now in our medical imaging imaging segmentation
+
+42
+00:02:47,630 --> 00:02:51,800
+project which is finding the nuclei in divergent images.
+
+43
+00:02:51,800 --> 00:02:52,210
+OK.
+
+44
+00:02:52,290 --> 00:02:56,710
+So we want to make you want to make sure you downloaded data set correctly.
+
+45
+00:02:56,840 --> 00:02:59,180
+And I want to actually show you something in this data set.
+
+46
+00:02:59,210 --> 00:03:03,980
+It is different to the type of the assets we used before because now it has mass.
+
+47
+00:03:04,010 --> 00:03:07,580
+So let's open this dataset here.
+
+48
+00:03:07,580 --> 00:03:09,590
+So hopefully you've extracted it here.
+
+49
+00:03:10,280 --> 00:03:13,900
+And as you can see yes it has the same train and validation for this.
+
+50
+00:03:14,210 --> 00:03:17,780
+But look at this it's no longer images now it's actually footless.
+
+51
+00:03:17,960 --> 00:03:21,010
+So we have an image here.
+
+52
+00:03:21,050 --> 00:03:25,290
+This is a test image which we're supposed to produce a mask from.
+
+53
+00:03:25,370 --> 00:03:28,070
+And now what does this handful of mask.
+
+54
+00:03:28,370 --> 00:03:32,320
+And these are multiple files and if you look at this there are multiple images.
+
+55
+00:03:32,450 --> 00:03:34,780
+Each one is a Nicholai label.
+
+56
+00:03:35,000 --> 00:03:37,190
+So what we're looking at.
+
+57
+00:03:37,340 --> 00:03:44,550
+If you go back to our presentation here spring is actually can be like this what we're looking at right
+
+58
+00:03:44,550 --> 00:03:49,470
+now is basically this masking is basically all the images we just saw.
+
+59
+00:03:49,500 --> 00:03:52,520
+It's minimized us stacked upon each other.
+
+60
+00:03:52,680 --> 00:04:00,690
+So the data is not as easy to basically interpret don't interpret but actually use as we would for all
+
+61
+00:04:00,690 --> 00:04:02,010
+previous tests.
+
+62
+00:04:02,010 --> 00:04:04,150
+We do have to do some processing on this data.
+
+63
+00:04:05,700 --> 00:04:07,380
+So that's a data set here.
+
+64
+00:04:07,560 --> 00:04:12,180
+And now let's go to all part of the book really have it loaded up here.
+
+65
+00:04:12,840 --> 00:04:17,520
+So this code here is basically code that was provided by this guy here.
+
+66
+00:04:17,520 --> 00:04:22,800
+He actually made the most popular little on Kaggle targeted one for this project and there were a number
+
+67
+00:04:22,800 --> 00:04:28,110
+of contestants and this guy had probably the best example of how it works.
+
+68
+00:04:28,110 --> 00:04:30,340
+So those were under his code.
+
+69
+00:04:30,840 --> 00:04:34,500
+So we have images here 128 size he defined.
+
+70
+00:04:34,530 --> 00:04:42,150
+We have all folders wanting to run that and fine ports everything successfully and we set our training
+
+71
+00:04:42,150 --> 00:04:47,360
+parts of test paths and this is the part that is very important.
+
+72
+00:04:47,370 --> 00:04:52,620
+Remember we showed you how the nuclides are basically on one image at a time.
+
+73
+00:04:52,620 --> 00:05:00,690
+So basically one input image has a bunch of sub images that each one having one you can labeled on it.
+
+74
+00:05:00,690 --> 00:05:06,410
+So what he does here what we do here is we basically stack those images together.
+
+75
+00:05:06,810 --> 00:05:11,290
+So effectively you can read the comments and stuff I've left in here.
+
+76
+00:05:11,430 --> 00:05:19,540
+What we do is we basically take these images and combine them into one single image with all these Nikolai's
+
+77
+00:05:19,570 --> 00:05:23,040
+in that image because that's basically the final message we want to produce.
+
+78
+00:05:23,110 --> 00:05:24,770
+So we have to process.
+
+79
+00:05:24,780 --> 00:05:26,370
+I will try to get a bit.
+
+80
+00:05:26,770 --> 00:05:30,760
+So I'm not going to run it because it takes some time to really run it successfully here and you can
+
+81
+00:05:30,760 --> 00:05:34,070
+do so yourself.
+
+82
+00:05:34,170 --> 00:05:38,560
+In fact you will have to do these things yourself because it's run on my machines run on us.
+
+83
+00:05:38,700 --> 00:05:44,130
+These are like basically statements of safe but they don't actually store any of the data in this notebook
+
+84
+00:05:45,120 --> 00:05:46,470
+just the outputs.
+
+85
+00:05:46,470 --> 00:05:48,780
+So let's do some illustrations here.
+
+86
+00:05:48,900 --> 00:05:52,290
+He has some nice cold here that generates this plot and plot.
+
+87
+00:05:52,380 --> 00:05:58,950
+We can see this is image zero here and this is a concatenated are stacked on a mass produced from an
+
+88
+00:05:58,970 --> 00:06:04,320
+input data and he does it for quite a few images here so he can actually see there's a lot of variety
+
+89
+00:06:04,410 --> 00:06:06,590
+in the input images here.
+
+90
+00:06:06,660 --> 00:06:11,030
+There's these grayscale ones like this seem to be the most popular.
+
+91
+00:06:11,040 --> 00:06:12,850
+Then there's his color images here.
+
+92
+00:06:12,930 --> 00:06:17,400
+Then there's these here these look like something from a microscope slide and then these these here
+
+93
+00:06:17,400 --> 00:06:22,850
+are different than they are this type here you can see them putting it here.
+
+94
+00:06:23,220 --> 00:06:25,860
+So there's a lot of different types of images here.
+
+95
+00:06:25,860 --> 00:06:27,800
+It's not one simple answer of the desert.
+
+96
+00:06:27,890 --> 00:06:33,750
+It's a bunch of different types of data all looking at nuclearized and all having mask that are put
+
+97
+00:06:33,750 --> 00:06:39,940
+like this or a label like the shape like this I should say.
+
+98
+00:06:40,030 --> 00:06:42,190
+So this is a function we used before.
+
+99
+00:06:42,350 --> 00:06:46,440
+It's actually what we're going to use is with the example one I used in my slide.
+
+100
+00:06:46,600 --> 00:06:50,090
+So I'm taking it off and reading this MTSO.
+
+101
+00:06:50,400 --> 00:06:54,480
+So this is the actual metric here that he's going to use.
+
+102
+00:06:54,560 --> 00:06:59,040
+We're going to use in our project I'm not going to go through the detail of how it's carefully calculated
+
+103
+00:06:59,430 --> 00:07:03,370
+but it is very similar to the calculation we saw in our slides.
+
+104
+00:07:03,400 --> 00:07:05,620
+It's bit different from us.
+
+105
+00:07:06,240 --> 00:07:11,460
+Alternatively there were a lot of discussions on the Kaggle the message board about if this function
+
+106
+00:07:11,460 --> 00:07:13,200
+was the best metric to use.
+
+107
+00:07:13,200 --> 00:07:16,680
+So here's an alternative when you can use Feel free to use it.
+
+108
+00:07:16,700 --> 00:07:19,340
+And this one uses this one side of it here.
+
+109
+00:07:19,550 --> 00:07:20,030
+OK.
+
+110
+00:07:21,640 --> 00:07:24,730
+So I actually left another one here too.
+
+111
+00:07:25,070 --> 00:07:25,490
+This.
+
+112
+00:07:25,690 --> 00:07:31,870
+This one actually it was basically a consensus said this was the best function and you can see it's
+
+113
+00:07:31,870 --> 00:07:33,340
+quite exhaustive.
+
+114
+00:07:33,340 --> 00:07:34,430
+Pretty technical.
+
+115
+00:07:34,480 --> 00:07:38,380
+Someone did spend a lot of time making this function.
+
+116
+00:07:38,460 --> 00:07:40,630
+So this is the important part here.
+
+117
+00:07:40,680 --> 00:07:45,750
+This is what this was equally important this function but this is of course what I wanted to show you
+
+118
+00:07:46,260 --> 00:07:48,650
+this is how we build our unit model.
+
+119
+00:07:48,900 --> 00:07:50,370
+So two things to note.
+
+120
+00:07:50,490 --> 00:07:51,610
+OK.
+
+121
+00:07:51,630 --> 00:07:58,170
+You're seeing we're actually speak signing a more lovely and sort of thing model that we're assigning
+
+122
+00:07:58,260 --> 00:08:02,560
+is two variables here and then we have this s in brackets here.
+
+123
+00:08:02,880 --> 00:08:05,040
+So what exactly are we doing here now.
+
+124
+00:08:05,280 --> 00:08:10,420
+Well this is simply another way we can build models in Paris.
+
+125
+00:08:10,510 --> 00:08:12,060
+Paris is quite flexible.
+
+126
+00:08:12,060 --> 00:08:16,430
+So what we're doing here we're connecting models by having it here.
+
+127
+00:08:16,460 --> 00:08:21,130
+So instead of using Waddler add we're connecting them here and there's a reason we can't use model that
+
+128
+00:08:21,840 --> 00:08:29,040
+is because of the unit structure which is basically like a bottleneck at the bottom and deconvolution
+
+129
+00:08:29,040 --> 00:08:30,300
+is going up.
+
+130
+00:08:30,300 --> 00:08:32,960
+It's not easy or doesn't facilitate a model.
+
+131
+00:08:33,060 --> 00:08:35,460
+Add method in building this model.
+
+132
+00:08:35,460 --> 00:08:38,840
+We sort of have to do it like this now so we can see.
+
+133
+00:08:38,850 --> 00:08:43,640
+So you want us to find here connected to this lambda function which basically normalizes the inputs.
+
+134
+00:08:43,980 --> 00:08:46,810
+Then we have see one here and see one here.
+
+135
+00:08:46,920 --> 00:08:53,690
+So what this does it basically says this is this is our convolutional layer here would drop all that's
+
+136
+00:08:53,740 --> 00:08:54,730
+tight see one.
+
+137
+00:08:55,110 --> 00:08:56,870
+And this is another completion here.
+
+138
+00:08:56,940 --> 00:08:58,120
+Titus see one again.
+
+139
+00:08:58,140 --> 00:09:00,860
+So this is linking all of these layers together.
+
+140
+00:09:01,200 --> 00:09:03,170
+And then we have Max beling here.
+
+141
+00:09:03,390 --> 00:09:06,470
+Basically it's called the P1 connection.
+
+142
+00:09:06,510 --> 00:09:09,080
+So now this is linked back to these here.
+
+143
+00:09:09,480 --> 00:09:10,890
+So we keep going forward.
+
+144
+00:09:11,490 --> 00:09:17,660
+As you can see the kernel the sizes get larger and larger as we go down and then we do this here.
+
+145
+00:09:17,820 --> 00:09:22,740
+This is how we kind of connect and it will basically bottleneck or blocking the key point here.
+
+146
+00:09:23,680 --> 00:09:30,000
+And basically now we just go up up with the not yet it is called a mandate because it looks like a when
+
+147
+00:09:30,230 --> 00:09:37,460
+those diagrams and we're using a different type of competition here constitute the transpose.
+
+148
+00:09:37,740 --> 00:09:39,020
+That's basically how we do it.
+
+149
+00:09:39,030 --> 00:09:44,220
+Deconvolution Lisieux So it's going up and up and then we have an upper tier.
+
+150
+00:09:44,460 --> 00:09:46,720
+So it is something I want you to note as well.
+
+151
+00:09:46,740 --> 00:09:48,750
+Look at the output of the D-Conn..
+
+152
+00:09:49,020 --> 00:09:54,480
+It's basically a greyscale image 128 by 128 with one dimension.
+
+153
+00:09:54,940 --> 00:09:57,380
+Is a number of parameters not that much.
+
+154
+00:09:57,420 --> 00:10:00,280
+Definitely trainable OCP.
+
+155
+00:10:00,350 --> 00:10:02,280
+So no that's fits our model.
+
+156
+00:10:04,130 --> 00:10:10,680
+So we just do all the basic callbacks here and then we just fit our model.
+
+157
+00:10:10,980 --> 00:10:12,950
+This probably should not be here.
+
+158
+00:10:13,020 --> 00:10:14,180
+Let's show what it was.
+
+159
+00:10:15,460 --> 00:10:16,690
+And here we go.
+
+160
+00:10:16,930 --> 00:10:20,960
+So you can see I'm not going to run this now but I've done it before and doesn't take that long.
+
+161
+00:10:21,250 --> 00:10:22,750
+We're training it here.
+
+162
+00:10:23,070 --> 00:10:24,150
+It's quick.
+
+163
+00:10:24,250 --> 00:10:25,510
+Just over a minute.
+
+164
+00:10:25,670 --> 00:10:30,560
+Poch just quite quick and we can see C-L metric glossier my metric.
+
+165
+00:10:30,580 --> 00:10:36,940
+So let's see what function is that up there was this one here.
+
+166
+00:10:36,970 --> 00:10:38,910
+That was the big one we use.
+
+167
+00:10:38,920 --> 00:10:42,130
+This was the one actually remember didn't Kaggle discussions.
+
+168
+00:10:42,280 --> 00:10:45,470
+That was the best I knew it was appropriate.
+
+169
+00:10:45,510 --> 00:10:50,800
+You not saying that these were wrong well these are bad it's just that this one was actually too much
+
+170
+00:10:50,800 --> 00:10:52,740
+relevance to segmentation to us.
+
+171
+00:10:52,990 --> 00:10:58,900
+Segmentation is very different to OBD-II detection I use which use boxes.
+
+172
+00:10:58,990 --> 00:11:00,810
+So we needed to develop something custom.
+
+173
+00:11:00,850 --> 00:11:06,810
+This was part this was definitely a big part of the Kaggle challenge in this project wasn't just applying
+
+174
+00:11:07,030 --> 00:11:11,860
+to the data it was coming up with a way to actually assess the performance of this model.
+
+175
+00:11:13,120 --> 00:11:21,160
+So we can see a metric changing as it goes and our validation metric going up.
+
+176
+00:11:21,160 --> 00:11:23,020
+So we wanted this to go up actually.
+
+177
+00:11:23,080 --> 00:11:24,850
+So we wanted to get that over time.
+
+178
+00:11:24,850 --> 00:11:31,810
+So I said 10 ebox it was a great way for sex in terms of accuracy that is not a good metric.
+
+179
+00:11:31,810 --> 00:11:37,480
+However this is not this is an I.T. metric and you metrics don't really it exactly to what accuracy
+
+180
+00:11:37,480 --> 00:11:38,340
+means.
+
+181
+00:11:38,350 --> 00:11:45,340
+So 24:6 is actually a pretty good value as seen on Kaggle some guys got up to point seven seven 7 0.8
+
+182
+00:11:45,880 --> 00:11:51,570
+using this metric that training on some GPS use for multiple ebox and tweaking a lot of the training
+
+183
+00:11:51,580 --> 00:11:52,900
+parameters above here.
+
+184
+00:11:53,110 --> 00:11:56,540
+But point Final Point 4 6 is actually pretty good.
+
+185
+00:11:56,740 --> 00:12:01,420
+And you may have noticed actually the code just changed and that was because I actually had some confusion
+
+186
+00:12:01,750 --> 00:12:08,250
+here where I was experimenting with different metrics and I confused myself because you when you look
+
+187
+00:12:08,250 --> 00:12:13,200
+at a model of when you create a model it is specific to this metric here.
+
+188
+00:12:13,450 --> 00:12:18,490
+So when you load them all you actually have to specify what metric you used when treating the model.
+
+189
+00:12:18,520 --> 00:12:26,320
+So we just train using my metric which was a function scroll all the way up the defined right here.
+
+190
+00:12:26,320 --> 00:12:31,240
+This metric basically used all of these functions here so basically calculate it.
+
+191
+00:12:31,270 --> 00:12:36,550
+This was a best one according to this guys and Kaggle which was the most representative representative
+
+192
+00:12:37,060 --> 00:12:38,940
+of what a loss should be.
+
+193
+00:12:39,100 --> 00:12:46,420
+OK so now what I'm going to do we basically use the LoDo model that we just trained or you can use model
+
+194
+00:12:46,420 --> 00:12:49,590
+if you're treating it within the book you don't have to look at it.
+
+195
+00:12:49,630 --> 00:12:50,210
+OK.
+
+196
+00:12:50,560 --> 00:12:54,100
+And basically we just split up split this up.
+
+197
+00:12:54,150 --> 00:13:00,550
+The data are extreme data into 90 percent basically being treating data and the last 10 percent being
+
+198
+00:13:00,550 --> 00:13:01,640
+the validation data.
+
+199
+00:13:02,050 --> 00:13:04,860
+And we just create our mass from this now.
+
+200
+00:13:04,900 --> 00:13:07,730
+So now let's take a look and see how on mask look.
+
+201
+00:13:07,780 --> 00:13:08,360
+OK.
+
+202
+00:13:08,740 --> 00:13:14,610
+So if you just run this function you'll see this was a training image here.
+
+203
+00:13:14,730 --> 00:13:21,350
+This was the mass that we kind of calculated previously from these here and then this is the predictive
+
+204
+00:13:21,350 --> 00:13:21,720
+mask.
+
+205
+00:13:21,740 --> 00:13:25,770
+As you can see it is very very similar to this mass.
+
+206
+00:13:25,790 --> 00:13:28,550
+There are some slight differences at a pixel level.
+
+207
+00:13:28,580 --> 00:13:30,750
+However this is actually quite good.
+
+208
+00:13:30,980 --> 00:13:33,310
+And now we can do some validation data.
+
+209
+00:13:33,620 --> 00:13:35,940
+So this was the actual input image here.
+
+210
+00:13:36,230 --> 00:13:38,850
+And this was a predicted mask of it.
+
+211
+00:13:38,870 --> 00:13:44,790
+So as you can see this is actually doing a pretty good job at image segmentation and it only took maybe
+
+212
+00:13:45,010 --> 00:13:47,540
+15 minutes to train on a super system.
+
+213
+00:13:47,660 --> 00:13:50,240
+So feel free to experiment with this.
+
+214
+00:13:50,240 --> 00:13:52,640
+You could create your own mess.
+
+215
+00:13:52,790 --> 00:13:57,380
+Not sure if you know how well you can do is probably get some software out I'll probably put a link
+
+216
+00:13:57,770 --> 00:14:03,380
+to some software and resources here where you can start annotating and basically creating mass from
+
+217
+00:14:03,410 --> 00:14:04,360
+images like this.
+
+218
+00:14:04,490 --> 00:14:10,430
+So if you want to try some medical imaging class segmentation task or any other sort of segmentation
+
+219
+00:14:10,430 --> 00:14:13,170
+to us you will know exactly how to do it.

19. Medical Imaging - Image Segmentation with U-Net/5.1 Download U-Net.html

@@ -0,0 +1 @@
+<script type="text/javascript">window.location = "https://drive.google.com/file/d/1X5vccywUQSv9VF8nTrldyBtIQqM_N5yS/view?usp=sharing";</script>

20. Principles of Object Detection/1. Chapter Introduction.srt

@@ -0,0 +1,43 @@
+1
+00:00:00,390 --> 00:00:00,780
+OK.
+
+2
+00:00:00,810 --> 00:00:04,960
+So welcome to Chapter 20 where we finally get into Optik detection.
+
+3
+00:00:05,250 --> 00:00:07,900
+And basically this chapter is up into four sections.
+
+4
+00:00:07,920 --> 00:00:14,310
+This is where I introduce the concept basically how Optik it all started and what it evolved to.
+
+5
+00:00:14,310 --> 00:00:20,760
+And then in twenty point to start talking about more modern day CNN based object to directors.
+
+6
+00:00:20,970 --> 00:00:23,920
+So we go from our CNN's to mess here in.
+
+7
+00:00:24,220 --> 00:00:30,450
+And then take a look at single shot detectors SSTV which is one of the most modern abstract algorithms
+
+8
+00:00:30,540 --> 00:00:31,260
+out there.
+
+9
+00:00:31,590 --> 00:00:36,220
+And then it just goes yellow which is a competing object detection algorithm right now.
+
+10
+00:00:36,240 --> 00:00:41,210
+So these two are the latest and greatest state of the art abductor's and we're going to test them.
+
+11
+00:00:41,220 --> 00:00:46,040
+We're going to go through them in detail and and test them and proceed in the following chapters.

20. Principles of Object Detection/2. Object Detection Introduction - Sliding Windows with HOGs.srt

@@ -0,0 +1,303 @@
+1
+00:00:00,470 --> 00:00:01,000
+OK.
+
+2
+00:00:01,050 --> 00:00:02,430
+So let's start at the beginning.
+
+3
+00:00:02,460 --> 00:00:05,520
+Let's talk about object really object vectors.
+
+4
+00:00:05,670 --> 00:00:11,910
+So I'm going to introduce you to the history of it so fiercely detection is one of the holy grails of
+
+5
+00:00:11,910 --> 00:00:17,610
+computer vision because previously what we have been doing is just classifying like an entire image
+
+6
+00:00:17,610 --> 00:00:20,510
+and seeing what objects are what Hassid belong to.
+
+7
+00:00:20,730 --> 00:00:26,490
+But can we take an image like this and label each major component into being a dog car person horse
+
+8
+00:00:26,760 --> 00:00:28,340
+person in the back.
+
+9
+00:00:28,350 --> 00:00:32,230
+Not yet until we have come across up to detection.
+
+10
+00:00:32,640 --> 00:00:40,620
+So object detection is a mix of object classification and localization object action is it is the identification
+
+11
+00:00:40,650 --> 00:00:43,120
+of a bounding box outlining the object.
+
+12
+00:00:43,140 --> 00:00:49,590
+So like in my face here basically is extraction a bony box or on my face and this direction is perhaps
+
+13
+00:00:49,590 --> 00:00:53,760
+one of the most popular object detection algorithms that we all know.
+
+14
+00:00:53,830 --> 00:00:57,220
+We're all quite familiar with from using cameras in our cell phones.
+
+15
+00:00:57,270 --> 00:00:57,780
+OK.
+
+16
+00:00:58,290 --> 00:01:04,150
+So basically it DL tells you instead of telling you this object here is a cat.
+
+17
+00:01:04,170 --> 00:01:09,070
+It actually tells you where is the cat and that is the whole point of object detection.
+
+18
+00:01:10,620 --> 00:01:15,340
+So let's get into the history of it and start with horror Cassiar classifiers.
+
+19
+00:01:15,360 --> 00:01:19,140
+Now there were many public detectors before this.
+
+20
+00:01:19,140 --> 00:01:24,840
+However here is what made it hard to justify this is what made it mainstream and quite popular because
+
+21
+00:01:24,840 --> 00:01:26,340
+it was so fast.
+
+22
+00:01:26,370 --> 00:01:33,420
+So basically this was a this was developed by Viola Jones in the face detection algorithm in 2001 not
+
+23
+00:01:33,420 --> 00:01:35,480
+that long long ago 17 years ago.
+
+24
+00:01:35,520 --> 00:01:40,960
+To be fair and it was superfast and it's actually still use to the number of applications.
+
+25
+00:01:41,280 --> 00:01:43,710
+Basically it's been optimized and tweaked to be even faster.
+
+26
+00:01:43,710 --> 00:01:49,890
+So it basically reduces the CPQ load and it's very very accurate.
+
+27
+00:01:49,890 --> 00:01:52,930
+Basically what it does it's a cascade of classifiers.
+
+28
+00:01:53,190 --> 00:01:56,640
+That's basically how it got it got its name and it uses a horror.
+
+29
+00:01:56,640 --> 00:01:58,590
+Basically let's go into the next slide.
+
+30
+00:01:58,660 --> 00:02:02,760
+Actually I don't have it in this section but it basically uses horror features and harsh features are
+
+31
+00:02:02,760 --> 00:02:06,210
+basically basically like you have rectangles.
+
+32
+00:02:06,250 --> 00:02:07,100
+Overling here.
+
+33
+00:02:07,240 --> 00:02:12,690
+You imagine a white rectangle here and one here and then there are different types of Arcacha pacifies.
+
+34
+00:02:12,810 --> 00:02:15,590
+So basically is just a feature extraction.
+
+35
+00:02:15,690 --> 00:02:22,350
+Basically what we learned before and it's led this box is that over the window over and over continuously
+
+36
+00:02:22,410 --> 00:02:31,950
+looking for a face they're very good but they are pretty hard to train and develop and optimize.
+
+37
+00:02:32,010 --> 00:02:38,010
+So let's move on to histogram with gradients and SVM sliding windows so sliding windows is a method
+
+38
+00:02:38,010 --> 00:02:43,580
+where we extract segments a full image piece by piece in the form of a rectangular extractor box.
+
+39
+00:02:43,590 --> 00:02:48,000
+So I mentioned it in previous slide when I was talking about this box being slid across this image.
+
+40
+00:02:48,330 --> 00:02:53,430
+What it does here in this image is a picture of my wife from the last bodybuilding bikini competition
+
+41
+00:02:53,430 --> 00:02:54,560
+two months ago.
+
+42
+00:02:54,870 --> 00:03:02,550
+And what it does is just imagine this window is being moved here then down here and then down here just
+
+43
+00:03:02,550 --> 00:03:05,670
+like remember how we moved across the image.
+
+44
+00:03:05,680 --> 00:03:07,960
+And CNN's it's exactly the same thing.
+
+45
+00:03:07,970 --> 00:03:14,430
+And we can actually set the same parameters like stride and the size of this box and what this box does
+
+46
+00:03:14,430 --> 00:03:17,640
+here in sliding windows with histogram of gradients.
+
+47
+00:03:17,700 --> 00:03:25,980
+SVM is that it basically extracts the entire hawgs all his brilliance in this box at different scales.
+
+48
+00:03:25,980 --> 00:03:31,620
+So basically it does it with image at one scale and then not a scale smaller scale and then this one
+
+49
+00:03:31,620 --> 00:03:35,480
+here and this one basically has no room to go right to just go straight down.
+
+50
+00:03:35,760 --> 00:03:39,480
+And it tries to match up to how gradients went what it knows.
+
+51
+00:03:39,480 --> 00:03:41,700
+It's supposed to look like to find the object.
+
+52
+00:03:42,000 --> 00:03:47,400
+Now as you can see this could be an effective way but it's not really that resilient.
+
+53
+00:03:47,400 --> 00:03:48,410
+Why.
+
+54
+00:03:48,420 --> 00:03:53,400
+Because imagine we have to do this for every segment of image continuously.
+
+55
+00:03:53,400 --> 00:03:55,680
+It gets exhaustive and computationally expensive
+
+56
+00:03:58,720 --> 00:04:05,370
+so previous action which is basically TISM feature extraction I just mentioned that and why would we
+
+57
+00:04:05,370 --> 00:04:10,740
+want to actually manually find co-features if CNN's actually eliminate that.
+
+58
+00:04:10,740 --> 00:04:16,350
+All right CNN's actually automatically find features by just running all these tests destroying data
+
+59
+00:04:16,680 --> 00:04:20,350
+Trulia algorithm and finding the last matching it with the correct last.
+
+60
+00:04:20,370 --> 00:04:22,770
+So that's what's brilliant about CNN's.
+
+61
+00:04:22,770 --> 00:04:24,760
+It takes that step away from us.
+
+62
+00:04:26,340 --> 00:04:31,970
+So as I said once of problems we're doing this is a sea of scale.
+
+63
+00:04:32,100 --> 00:04:34,920
+Imagine this is a simple image just 20 by 20.
+
+64
+00:04:34,920 --> 00:04:36,870
+So this box can be passed over here.
+
+65
+00:04:36,960 --> 00:04:39,630
+But imagine this was a much bigger continue TV image.
+
+66
+00:04:39,720 --> 00:04:44,130
+How many different times how many different boxes would we extract.
+
+67
+00:04:44,130 --> 00:04:46,460
+How do we know what size box should be.
+
+68
+00:04:46,470 --> 00:04:50,410
+I mean that's where we rescale image but how many different rescaling are we going to do.
+
+69
+00:04:50,440 --> 00:04:54,830
+So as you can see this is not a very effective way of doing object detection.
+
+70
+00:04:56,430 --> 00:05:02,600
+So talk a bit the bullet histogram gradients are not going to go in go into this in detail of taught
+
+71
+00:05:02,600 --> 00:05:05,480
+this in my other op and see the course you can.
+
+72
+00:05:05,480 --> 00:05:07,280
+The video is included free in that section.
+
+73
+00:05:07,290 --> 00:05:09,230
+So that's why I'm going to talk about it much here.
+
+74
+00:05:09,550 --> 00:05:15,290
+But basically the slides are here for you to go through on your own and you can pretty much infer from
+
+75
+00:05:15,290 --> 00:05:17,720
+these steps here what hawgs really are.
+
+76
+00:05:20,110 --> 00:05:22,090
+So now we move on to our CNN's.

20. Principles of Object Detection/3. R-CNN, Fast R-CNN, Faster R-CNN and Mask R-CNN.srt

@@ -0,0 +1,847 @@
+1
+00:00:00,460 --> 00:00:00,840
+OK.
+
+2
+00:00:00,850 --> 00:00:08,260
+So welcome to 20 point to where we talk about all the different types of CNN's going from typical original
+
+3
+00:00:08,520 --> 00:00:10,320
+on CNN all the way to fast.
+
+4
+00:00:10,330 --> 00:00:12,380
+Our CNN and mask are CNN.
+
+5
+00:00:12,640 --> 00:00:15,000
+So let's see what this is about.
+
+6
+00:00:15,010 --> 00:00:20,040
+So what does our our CNN sign for it sends for regions actually not recurrent.
+
+7
+00:00:20,110 --> 00:00:25,210
+That's a different type of neural net and this are seen CNN's were first introduced relatively recently
+
+8
+00:00:25,720 --> 00:00:30,300
+by in 2014 by researchers at the University of College of Berkeley in California.
+
+9
+00:00:30,580 --> 00:00:35,730
+And basically they had dramatically improved performance on D-Pa. VRC challenge.
+
+10
+00:00:35,770 --> 00:00:40,010
+This is the equivalent of image for object detection detection testing.
+
+11
+00:00:40,300 --> 00:00:40,610
+OK.
+
+12
+00:00:40,650 --> 00:00:48,880
+The example of some of the images in there that isn't so our CNN's attempt to solve the exhaustive search
+
+13
+00:00:49,360 --> 00:00:55,090
+previously performed by sliding windows by proposing bounding boxes and passing these extracted bounding
+
+14
+00:00:55,150 --> 00:00:57,380
+boxes to the image ossify.
+
+15
+00:00:57,700 --> 00:01:03,460
+So how do we how do we find these bones in boxes like holiday proposed proposed by using the selective
+
+16
+00:01:03,460 --> 00:01:04,760
+search algorithm.
+
+17
+00:01:04,900 --> 00:01:07,760
+And this is like a simple illustration of what happens here.
+
+18
+00:01:08,020 --> 00:01:09,360
+So we have an image here.
+
+19
+00:01:09,610 --> 00:01:15,820
+We have to propose on the boxes and then we have we basically use these boxes perceptual CNN and try
+
+20
+00:01:15,820 --> 00:01:19,140
+to identify what is in that.
+
+21
+00:01:19,140 --> 00:01:25,290
+So let's talk about the selective search algorithm selective search attempts to segment image into groups
+
+22
+00:01:25,590 --> 00:01:31,680
+by combining similar areas such as colors textures and propose these regions as interesting bounding
+
+23
+00:01:31,680 --> 00:01:32,700
+boxes.
+
+24
+00:01:32,700 --> 00:01:34,680
+So we have this image of some sheep here.
+
+25
+00:01:35,040 --> 00:01:39,810
+And what's left of it is going to do is going to try to like merge different similar pixels similar
+
+26
+00:01:39,810 --> 00:01:42,120
+lighting conditions that sort of stuff.
+
+27
+00:01:42,120 --> 00:01:48,250
+So we can kind of get eventually get something like this and then it's going to draw a box of each segmented
+
+28
+00:01:48,270 --> 00:01:50,630
+region and proposed that.
+
+29
+00:01:50,840 --> 00:01:53,760
+Now it is a lot more tweaking and we can do selective siche.
+
+30
+00:01:53,940 --> 00:01:55,040
+So it all depends.
+
+31
+00:01:55,050 --> 00:01:58,560
+Basically we can get little boxes or a lot of boxes
+
+32
+00:02:01,640 --> 00:02:08,240
+so when selective search has identified these regions of boxes it passes this extracted image to our
+
+33
+00:02:08,240 --> 00:02:11,420
+CNN example one tree and an image in it.
+
+34
+00:02:11,520 --> 00:02:13,890
+OK that would be a very good sign into use.
+
+35
+00:02:14,720 --> 00:02:20,740
+We don't use the CNN directly for classification though although we can we use an SVM to first classify
+
+36
+00:02:20,740 --> 00:02:22,830
+the scene and extracted features.
+
+37
+00:02:22,850 --> 00:02:24,710
+Now I didn't mention that before.
+
+38
+00:02:24,830 --> 00:02:31,990
+But what happens is that instead of actually passing to CNN here using CNN to get to classes what it
+
+39
+00:02:32,000 --> 00:02:36,890
+does it just gives you to CNN features which are basically a feature maps.
+
+40
+00:02:36,890 --> 00:02:37,690
+All right.
+
+41
+00:02:37,790 --> 00:02:42,590
+And then we use an SVM to classified a scene and extract features.
+
+42
+00:02:42,590 --> 00:02:49,550
+This was probably done to speed because if I if I'm not mistaken R-S.C. an end was the original proposed
+
+43
+00:02:49,580 --> 00:02:51,790
+object action and it was meant for video.
+
+44
+00:02:52,250 --> 00:02:53,680
+So speed was a concern.
+
+45
+00:02:54,800 --> 00:03:00,460
+So after this region proposal has been classified we didn't use a simple linear regression to generate
+
+46
+00:03:00,550 --> 00:03:07,060
+a tighter danglin box that's holding boxes we're actually basically size and fit around objects and
+
+47
+00:03:07,070 --> 00:03:07,740
+image.
+
+48
+00:03:07,940 --> 00:03:15,440
+But how do we know what the good boxes though and that's where we can come up with I an IOU metric that
+
+49
+00:03:15,440 --> 00:03:16,550
+we discussed before.
+
+50
+00:03:16,760 --> 00:03:23,060
+Remember you basically had measured how good the overlap of the predicted box was over the over the
+
+51
+00:03:23,060 --> 00:03:26,090
+labels box.
+
+52
+00:03:26,090 --> 00:03:31,580
+Now here's a problem that happens in our CNN's own basically or object detection algorithms.
+
+53
+00:03:31,780 --> 00:03:39,490
+You DO WE they don't often just propose one box they propose many boxes over to see Image image sometimes.
+
+54
+00:03:39,800 --> 00:03:44,100
+So remember in our view point five was considered a good result.
+
+55
+00:03:44,420 --> 00:03:51,650
+Well what if we have multiple boxes and you all want all given over one with zero point five.
+
+56
+00:03:51,650 --> 00:03:52,890
+This is a common problem.
+
+57
+00:03:53,180 --> 00:03:53,750
+OK.
+
+58
+00:03:54,050 --> 00:03:56,560
+So in the figures below let's go to this example.
+
+59
+00:03:56,570 --> 00:03:58,590
+We have four boxes in red.
+
+60
+00:03:58,680 --> 00:04:00,080
+These are them right here.
+
+61
+00:04:00,110 --> 00:04:08,930
+The green is a true box a true positive and as such we have one true positive and true false positives.
+
+62
+00:04:08,930 --> 00:04:09,470
+OK.
+
+63
+00:04:10,040 --> 00:04:15,460
+So the true false positives here would be these boxes here one two and three.
+
+64
+00:04:15,500 --> 00:04:16,010
+OK.
+
+65
+00:04:20,120 --> 00:04:26,520
+So the reason it's true false positives is because we have one box here the dotted line right now this
+
+66
+00:04:26,520 --> 00:04:31,440
+is basically our best box so that will count as a true positive in our predictions.
+
+67
+00:04:31,440 --> 00:04:32,000
+All right.
+
+68
+00:04:32,130 --> 00:04:34,040
+So for now it is ignore the green box.
+
+69
+00:04:34,050 --> 00:04:36,010
+He doesn't count in this coalition.
+
+70
+00:04:36,060 --> 00:04:38,180
+We just have one box that's really good.
+
+71
+00:04:38,220 --> 00:04:47,740
+He's a true positive and treat up on good or false positives so mean average position is a very tricky
+
+72
+00:04:47,830 --> 00:04:54,510
+metric in my opinion mainly because it's not that intuitive to understand how the formulas fit here.
+
+73
+00:04:54,730 --> 00:04:56,870
+What I'm going to say though is I'm going to try to explain it to you.
+
+74
+00:04:56,940 --> 00:05:02,470
+Outgo introduce into amount of left I'm not for you in a slide so you can go over on your own and try
+
+75
+00:05:02,470 --> 00:05:03,720
+to make sense of it.
+
+76
+00:05:03,760 --> 00:05:08,840
+There's also a couple of blogs that have some very good but pretty lengthy explanations for it.
+
+77
+00:05:09,070 --> 00:05:11,100
+But let's go back to the sorry.
+
+78
+00:05:11,320 --> 00:05:17,170
+So what I'm going to say is that actually in this blog here is a very good explanation for it.
+
+79
+00:05:17,290 --> 00:05:25,030
+But essentially what mean average position tries to do is that it takes it knows all the boxes the architect
+
+80
+00:05:25,060 --> 00:05:31,490
+proposes and basically what it tries to do is try to come up with a metric that defines basically what
+
+81
+00:05:31,490 --> 00:05:33,130
+it is like.
+
+82
+00:05:33,300 --> 00:05:41,680
+See object that to predict this but object to be predicted maybe for false positives and maybe like
+
+83
+00:05:41,680 --> 00:05:44,880
+some other boxes that were probably irrelevant and stuff.
+
+84
+00:05:44,890 --> 00:05:47,580
+So and then we remember this is a for one a class.
+
+85
+00:05:47,590 --> 00:05:49,280
+Remember there are different classes as well.
+
+86
+00:05:49,460 --> 00:05:56,300
+So I mean average precision is a way to measure how effective all of my object detectors were were actually
+
+87
+00:05:56,560 --> 00:05:57,490
+well actually.
+
+88
+00:05:57,660 --> 00:05:58,070
+OK.
+
+89
+00:05:58,920 --> 00:06:04,690
+So just remember this is a metric of how we baseline how we measure performance of objective
+
+90
+00:06:07,820 --> 00:06:14,420
+So now let's move on to not the maximum suppression and this is a technique that object is used to remove
+
+91
+00:06:14,420 --> 00:06:18,640
+overlapping boxes and thus improve them up scores significantly.
+
+92
+00:06:18,920 --> 00:06:24,400
+So what this does basically it looks at a probabilities associated with each box that's being generated
+
+93
+00:06:24,830 --> 00:06:30,590
+and the probabilities it looks at all the probabilities of the object being in the same class effectively.
+
+94
+00:06:30,590 --> 00:06:35,690
+So if we have like two or three boxes that say this is a call but they have high overlap.
+
+95
+00:06:35,870 --> 00:06:40,180
+What it does now it looks at the highest probability box.
+
+96
+00:06:40,230 --> 00:06:41,970
+That's this one here in red.
+
+97
+00:06:42,200 --> 00:06:46,730
+And what it does it checks the eye or you would do with the other boxes.
+
+98
+00:06:46,730 --> 00:06:52,970
+So in this image here we see that we have tree boxes here that object texture the doctor has basically
+
+99
+00:06:54,080 --> 00:07:00,340
+highlighted and these are the probabilities of it being a class so what it does it checks you over these
+
+100
+00:07:00,410 --> 00:07:07,940
+here and what it does it will drop the boxes that it deems basically not as relevant as main box.
+
+101
+00:07:07,950 --> 00:07:11,250
+So that's effectively how we clean up these boxes.
+
+102
+00:07:11,470 --> 00:07:15,420
+You do object that is produce.
+
+103
+00:07:15,450 --> 00:07:17,430
+So now let's move on to fast.
+
+104
+00:07:17,430 --> 00:07:24,870
+Our CNN's this was this was announced or released in 2015 a year after the original on CNN came out
+
+105
+00:07:25,380 --> 00:07:31,320
+and basically what happened was that the problem with our CNNs was that it was effective but pretty
+
+106
+00:07:31,320 --> 00:07:39,030
+slow as each but each proposed bounding box has to be classified by a CNI by CNN and as such doing is
+
+107
+00:07:39,120 --> 00:07:41,890
+doing it in realtime was often impossible.
+
+108
+00:07:42,060 --> 00:07:43,970
+So it required tree models.
+
+109
+00:07:44,370 --> 00:07:47,640
+And it also required tree models to be trained separately.
+
+110
+00:07:47,640 --> 00:07:53,760
+So we had to have a feature extraction CNN and SVM to predict a class and a linear regression model
+
+111
+00:07:53,760 --> 00:07:55,420
+to tighten the Bongi boxes.
+
+112
+00:07:55,640 --> 00:07:57,690
+You remember all these things we discussed earlier.
+
+113
+00:07:57,960 --> 00:08:03,220
+So this definitely is a bit of a lot of moving parts in our CNN's.
+
+114
+00:08:03,230 --> 00:08:06,600
+So what did fasta our CNN's do first.
+
+115
+00:08:06,630 --> 00:08:11,190
+Our CNN's fiercly reduced number for postboxes by removing the overlap generated.
+
+116
+00:08:11,190 --> 00:08:12,930
+So how did they do this.
+
+117
+00:08:12,930 --> 00:08:18,750
+We run the CNN across the image just once instead of many times using a technique called that region
+
+118
+00:08:19,350 --> 00:08:21,930
+of interest pooling our boy pool.
+
+119
+00:08:22,440 --> 00:08:22,880
+OK.
+
+120
+00:08:23,190 --> 00:08:29,280
+So our away pool allows us to share the forward pass of the CNN for images for image across that sort
+
+121
+00:08:29,280 --> 00:08:30,660
+of subregions.
+
+122
+00:08:30,660 --> 00:08:37,380
+This works because previously regions are simply extracted from the CNN feature map and then puled which
+
+123
+00:08:37,530 --> 00:08:44,360
+deadfall you only need to render CNN once image and basically use that output of the CNN going forward.
+
+124
+00:08:46,060 --> 00:08:52,510
+So combining the training of the CNN classifier and the bonding box regressed into a single model that's
+
+125
+00:08:52,510 --> 00:08:53,030
+what it did.
+
+126
+00:08:53,110 --> 00:08:59,280
+So instead of so we have the SVM feature classifier and that now became a sort of actually on top of
+
+127
+00:08:59,280 --> 00:09:05,870
+the CNN and old linear regression that was taking the boxes basically that became a bounding box.
+
+128
+00:09:05,910 --> 00:09:08,770
+Apulia parallel to our soft Leo.
+
+129
+00:09:09,040 --> 00:09:12,540
+So basically what it became is this.
+
+130
+00:09:12,610 --> 00:09:14,430
+So we have a feature extractor.
+
+131
+00:09:14,440 --> 00:09:22,360
+CNN we have a soft Max layers at the end of CNN being our classifier for the object type or class.
+
+132
+00:09:22,510 --> 00:09:26,120
+And then we have this in Peridot a bounding box regressive.
+
+133
+00:09:26,170 --> 00:09:33,400
+So that is how it our CNN's fast our CNN solve a lot of the delays and sluggishness of the original
+
+134
+00:09:33,400 --> 00:09:37,670
+RCN and so now a year later in 2016.
+
+135
+00:09:37,950 --> 00:09:42,560
+Our CNN's media release I should say so faster.
+
+136
+00:09:42,720 --> 00:09:44,650
+Our CNN's made significant speed increases.
+
+137
+00:09:44,650 --> 00:09:51,290
+However a region proposal still remained relatively slow as it still relied on selective search algorithm.
+
+138
+00:09:51,790 --> 00:09:57,220
+Fortunately a Microsoft Research Team figured out how to eliminate this bottleneck.
+
+139
+00:09:57,220 --> 00:10:02,770
+So how did it speed up rigid proposal select to sit your lies and features extracted from the image
+
+140
+00:10:03,280 --> 00:10:07,880
+What if we just reused as features to do region proposal instead.
+
+141
+00:10:08,100 --> 00:10:11,220
+Okay so that was the inside that made the faster.
+
+142
+00:10:11,320 --> 00:10:13,180
+And is extremely efficient.
+
+143
+00:10:13,180 --> 00:10:15,670
+So you get to take a look at a diagram from the people here.
+
+144
+00:10:15,970 --> 00:10:23,150
+So basically this line here is what is important what if we use those features to do rigid proposal.
+
+145
+00:10:23,440 --> 00:10:29,350
+That is exactly why we don't have to keep running this over and over for the equivalent of a bit using
+
+146
+00:10:29,770 --> 00:10:32,410
+selective search to generate our proposals.
+
+147
+00:10:33,880 --> 00:10:35,950
+So how do we do rigid proposals with faster.
+
+148
+00:10:35,950 --> 00:10:43,960
+Our CNN's so fast our CNN's ad a fully convolutional network on top of the features of CNN to create
+
+149
+00:10:43,990 --> 00:10:45,740
+a region proposal network.
+
+150
+00:10:45,940 --> 00:10:47,370
+That's what we're seeing here.
+
+151
+00:10:47,680 --> 00:10:48,780
+So it is now.
+
+152
+00:10:49,390 --> 00:10:56,720
+Let's go back to it is now a full fully convolutional network on top of the features of CNN.
+
+153
+00:10:56,740 --> 00:10:57,480
+So let's think about that.
+
+154
+00:10:57,480 --> 00:11:04,590
+So we have a CNN producers here and is a fully convolutional at work here that does this region proposal
+
+155
+00:11:05,980 --> 00:11:06,730
+OK.
+
+156
+00:11:06,930 --> 00:11:12,540
+So the authors of the paper state the region proposal network slides a window over the features of the
+
+157
+00:11:12,570 --> 00:11:14,930
+CNN at each window location.
+
+158
+00:11:14,940 --> 00:11:22,740
+The netbook was a school and a bounty box put anchor hence 4000 box coordinates where kids number of
+
+159
+00:11:22,740 --> 00:11:23,540
+anchors.
+
+160
+00:11:23,670 --> 00:11:25,980
+That is basically how it works.
+
+161
+00:11:26,010 --> 00:11:31,680
+I encourage you to read to people if you want to get into more detail but first our CNN's Anyway back
+
+162
+00:11:31,680 --> 00:11:34,540
+to this after each pass of the sliding window.
+
+163
+00:11:34,740 --> 00:11:41,400
+It outputs key potential ballot boxes and a confidence of how good this box is expected to be.
+
+164
+00:11:41,400 --> 00:11:44,860
+That's pretty cool and pretty complicated as well.
+
+165
+00:11:45,180 --> 00:11:47,140
+So producing the bombing boxes now.
+
+166
+00:11:47,360 --> 00:11:54,630
+So previously we mentioned we produced key potential bounding boxes these bomb box proposals are proposals
+
+167
+00:11:54,630 --> 00:12:01,740
+of common expected boxes of suits and ships aspect ratio and sizes this aspect ratio is called Anca
+
+168
+00:12:01,740 --> 00:12:02,710
+boxes.
+
+169
+00:12:02,730 --> 00:12:09,420
+So what's happening here is that all boxes the boxes we propose in this method are basically predefined
+
+170
+00:12:09,630 --> 00:12:13,150
+and there would be 60 of suits and shapes ratios and sizes.
+
+171
+00:12:13,530 --> 00:12:20,040
+So it's a region that the proposal outputs a bony box put onco and a scale of how likely the image in
+
+172
+00:12:20,040 --> 00:12:29,940
+the box will be an object so let's move on to mask our CNN's which is pixel level segmentation now Mass.
+
+173
+00:12:29,980 --> 00:12:32,430
+Our CNN's aim to combine it.
+
+174
+00:12:32,460 --> 00:12:38,170
+Obs detection action and classification with segmentation as we previously saw segmentation as the labeling
+
+175
+00:12:38,170 --> 00:12:41,880
+of objects per pixel level as we can see in the above image.
+
+176
+00:12:42,980 --> 00:12:45,220
+So how do Musse our CNN's work.
+
+177
+00:12:45,450 --> 00:12:47,590
+They are basically an extension of fast.
+
+178
+00:12:47,600 --> 00:12:53,480
+Our CNN's web binary mask is created for the objects detected by the box.
+
+179
+00:12:53,540 --> 00:13:00,100
+So you remember how you actually created a mask that was basically provided segmentation outputs.
+
+180
+00:13:00,140 --> 00:13:07,640
+That's what's happening here when in fact our CNN's so binary mask basically bit 0 or 1 is created for
+
+181
+00:13:07,640 --> 00:13:09,060
+the object in the box.
+
+182
+00:13:09,080 --> 00:13:11,190
+So we know we're looking at a box.
+
+183
+00:13:11,240 --> 00:13:15,390
+And we are creating a binary mask for each box that's found.
+
+184
+00:13:15,560 --> 00:13:21,590
+And this is the architecture of the network and the link to the actual research paper where this publication
+
+185
+00:13:21,710 --> 00:13:24,390
+was released or made.
+
+186
+00:13:24,770 --> 00:13:28,440
+So messy CNN's you something called R Y O line.
+
+187
+00:13:28,820 --> 00:13:33,580
+So the mass of put uses the CNN extracted features to create its binary mask.
+
+188
+00:13:33,710 --> 00:13:36,130
+So how do the authors of this paper achieve this.
+
+189
+00:13:36,130 --> 00:13:42,740
+They use our oil line instead of a pool as a Pooles feature map was misalign from the regions of the
+
+190
+00:13:42,740 --> 00:13:43,660
+original image.
+
+191
+00:13:43,880 --> 00:13:46,990
+That was something I read in the paper I was confused to why.
+
+192
+00:13:47,100 --> 00:13:48,350
+But they explained it pretty well.
+
+193
+00:13:48,470 --> 00:13:52,010
+So mapping a region of interest onto a future map.
+
+194
+00:13:52,010 --> 00:13:54,580
+So imagine this was the original image here.
+
+195
+00:13:54,770 --> 00:13:59,000
+And we had a feature map that we generated from CNN that is smaller.
+
+196
+00:13:59,210 --> 00:14:00,700
+OK because it usually is smaller.
+
+197
+00:14:00,710 --> 00:14:07,010
+And as we use your reporting and what happens here is I know a 100 by 100 image is now mapped onto the
+
+198
+00:14:07,040 --> 00:14:08,610
+two by to the to future map.
+
+199
+00:14:08,690 --> 00:14:14,930
+Therefore a window of 20 by 20 that's just good old size window here on your original image is mapped
+
+200
+00:14:14,930 --> 00:14:17,470
+to a 6.4 pixel here.
+
+201
+00:14:17,510 --> 00:14:21,320
+This is how a direct linear mapping would be in the future map.
+
+202
+00:14:21,460 --> 00:14:25,750
+Our pool however wrong's down pixel maps to six by six.
+
+203
+00:14:25,790 --> 00:14:30,730
+That is why our pools feature but when we use our pool instead of like a line.
+
+204
+00:14:30,740 --> 00:14:37,880
+You had a misaligned boxes or regions on the image may have been that important to any looking at a
+
+205
+00:14:37,880 --> 00:14:39,630
+big image with small objects.
+
+206
+00:14:39,950 --> 00:14:47,240
+But if you think about it if it's a big box and you're going to do a proposal on being misaligned can
+
+207
+00:14:47,240 --> 00:14:48,290
+actually be bad.
+
+208
+00:14:48,710 --> 00:14:55,910
+So I wrote a line of these bio linear interpolation to know exactly what would be the pixel at 6.4.
+
+209
+00:14:56,360 --> 00:15:02,260
+So it's a pretty cool nifty algorithm that gives you exact mapping without having to roundel pixels.
+
+210
+00:15:03,620 --> 00:15:10,640
+So these are some examples of mask RCN and segmenting and classifying images is pretty cool to see this
+
+211
+00:15:10,640 --> 00:15:11,030
+in action.
+
+212
+00:15:11,030 --> 00:15:14,570
+Actually it's very accurate in some in some videos.

20. Principles of Object Detection/4. Single Shot Detectors (SSDs).srt

@@ -0,0 +1,115 @@
+1
+00:00:00,750 --> 00:00:01,100
+OK.
+
+2
+00:00:01,140 --> 00:00:06,570
+So now let's move on to chapter 20 point tree where we talk about single shot detectors also called
+
+3
+00:00:06,630 --> 00:00:09,510
+SSD is and design not solid state drives.
+
+4
+00:00:09,510 --> 00:00:18,010
+By the way totally different SSD so single shot detectors so we previously just went through the entire
+
+5
+00:00:18,310 --> 00:00:21,880
+CNN family and we've seen how successfully that can be applied.
+
+6
+00:00:21,880 --> 00:00:22,590
+All right.
+
+7
+00:00:22,720 --> 00:00:28,690
+How ever the performance on video is still not optimal and it typically run even Anji pews at seven
+
+8
+00:00:28,690 --> 00:00:29,690
+frames per second.
+
+9
+00:00:29,860 --> 00:00:36,100
+Now SSD is to improve dispute speech by eliminating the need for rigid proposal so you can see the speed
+
+10
+00:00:36,100 --> 00:00:38,870
+of fasta our CNN's here and you'll.
+
+11
+00:00:39,160 --> 00:00:40,860
+And this is Ulo vision one.
+
+12
+00:00:41,020 --> 00:00:45,500
+But look at the speed of SSD is on relatively high resolution images.
+
+13
+00:00:45,520 --> 00:00:49,820
+They're pretty fast and this is running on a Titan SGP.
+
+14
+00:00:50,500 --> 00:00:52,060
+So this is pretty impressive.
+
+15
+00:00:52,060 --> 00:01:00,130
+So how did it achieve this improvement in speed as these use multi-skilled features and default boxes
+
+16
+00:01:00,160 --> 00:01:03,400
+as well as dropping the resolution of the images to improve speed.
+
+17
+00:01:03,400 --> 00:01:05,570
+That doesn't seem that difficult does it.
+
+18
+00:01:05,590 --> 00:01:10,740
+But anyway this allows a city of near real time speed with almost no drop in accuracy.
+
+19
+00:01:11,810 --> 00:01:15,240
+So these are comprised of two main parts.
+
+20
+00:01:15,380 --> 00:01:21,110
+We have the feature epic Scheckter and typically they use Viji 16 that was actually what was used in
+
+21
+00:01:21,110 --> 00:01:26,360
+the published people but residents a dense net actually could provide better results as they have actually
+
+22
+00:01:26,510 --> 00:01:31,970
+been better in the eyeless VRC result competition.
+
+23
+00:01:31,980 --> 00:01:37,490
+So anyway so we then we have the feature map here and we have the convolutional filter that we use for
+
+24
+00:01:37,490 --> 00:01:38,930
+object detection.
+
+25
+00:01:38,930 --> 00:01:40,620
+So this is a diagram of it here.
+
+26
+00:01:40,670 --> 00:01:43,060
+So basically this is the input image here.
+
+27
+00:01:43,130 --> 00:01:48,260
+This is a feature extractor for convolutional work here which was Viji 16.
+
+28
+00:01:48,590 --> 00:01:53,070
+And then we have the convolutional filter that we use for our detectors here.
+
+29
+00:01:53,240 --> 00:01:53,730
+OK.

20. Principles of Object Detection/5. YOLO to YOLOv3.srt

@@ -0,0 +1,203 @@
+1
+00:00:00,600 --> 00:00:05,590
+Hi and welcome to Chapter 20 point four where we talk about you and we go all the way up from yellow
+
+2
+00:00:05,640 --> 00:00:07,470
+to fish and tree.
+
+3
+00:00:07,470 --> 00:00:09,700
+So let's see what eel really is about.
+
+4
+00:00:10,710 --> 00:00:12,430
+So you know you only live once.
+
+5
+00:00:12,480 --> 00:00:15,140
+No it's actually you only look once.
+
+6
+00:00:15,510 --> 00:00:21,110
+And the idea behind you is that a single neuron that work is applied to full to a full image.
+
+7
+00:00:21,450 --> 00:00:25,830
+And this allows us to reason globally about image when generating predictions.
+
+8
+00:00:25,910 --> 00:00:31,210
+So it's basically a neural net that actually looks at the entire image for us with CNN.
+
+9
+00:00:31,560 --> 00:00:38,220
+So it is a direct development from off multi bucks but it turns a multi box from a rigid proposal into
+
+10
+00:00:38,220 --> 00:00:44,400
+an object recognition method by adding a soft actually in parallel with a box or a press box classify
+
+11
+00:00:44,400 --> 00:00:45,070
+earlier.
+
+12
+00:00:45,450 --> 00:00:50,580
+So it divides the image into regions and then predicts bounding boxes and possibilities of each region
+
+13
+00:00:51,430 --> 00:00:58,840
+YOLO YOLO uses a fully convolutional neural network allowing for an input of various sizes.
+
+14
+00:00:58,890 --> 00:00:59,840
+So that's pretty cool.
+
+15
+00:01:01,680 --> 00:01:04,820
+So let me tell you how it works.
+
+16
+00:01:04,830 --> 00:01:08,760
+So the input image is divided into an S by a script.
+
+17
+00:01:09,100 --> 00:01:14,870
+If the center of an object falls into this grid that cell is responsible for detecting that object.
+
+18
+00:01:14,900 --> 00:01:20,580
+Now each grid predicts a number of bungling boxes and confidence scores for his boxes.
+
+19
+00:01:20,580 --> 00:01:25,270
+Confidence has defined as a probability of an object multiplied by the threshold.
+
+20
+00:01:25,290 --> 00:01:31,300
+I use school and I use scores of point five or less than 0.5 or given ZERO confidence.
+
+21
+00:01:31,630 --> 00:01:36,080
+So the bounding box is defined by these parameters.
+
+22
+00:01:36,120 --> 00:01:42,690
+X y width and height where x and y the center of the box and W H are height and weight by multiplying
+
+23
+00:01:42,690 --> 00:01:47,450
+the conditional class probability and individual box confidence predictions.
+
+24
+00:01:47,460 --> 00:01:50,740
+We get to class specific confidence school of each box.
+
+25
+00:01:50,970 --> 00:01:55,910
+That's how you effectively works so that your model is quite good.
+
+26
+00:01:55,920 --> 00:01:57,830
+So this is essay by a secret here.
+
+27
+00:01:58,080 --> 00:02:02,950
+These are the bounding boxes that we just generate from each cell plus the confidence.
+
+28
+00:02:03,000 --> 00:02:05,160
+This is the class probability map.
+
+29
+00:02:05,160 --> 00:02:10,350
+So the class probability map basically indicates if you go back to it here the probability of an object
+
+30
+00:02:10,350 --> 00:02:16,740
+be multiplied by Trishul school k and then these are the final actions here.
+
+31
+00:02:16,740 --> 00:02:20,220
+So you can see this blue object here which would belong to that class.
+
+32
+00:02:20,220 --> 00:02:22,270
+Probability of a dog.
+
+33
+00:02:22,380 --> 00:02:29,400
+Then there was a bicycle and then there was a background which ended up being a car.
+
+34
+00:02:29,440 --> 00:02:36,160
+So let's talk about the last function adjustments during training Ulo uses differential weight for confidence
+
+35
+00:02:36,160 --> 00:02:40,790
+corrections from boxes that contain objects and boxes that do not contain objects.
+
+36
+00:02:41,020 --> 00:02:46,990
+It penalizes errors in small and large objects differently by predicting the square root of the box
+
+37
+00:02:47,010 --> 00:02:49,780
+width and height.
+
+38
+00:02:49,780 --> 00:02:51,550
+So this is a little architecture.
+
+39
+00:02:51,800 --> 00:02:53,980
+It's a bit it looks a bit simple doesn't it.
+
+40
+00:02:53,980 --> 00:02:57,480
+However it's a lot more going on than meets the eye.
+
+41
+00:02:57,590 --> 00:03:04,510
+In this image so let's talk about the evolution of your little sister in 2016.
+
+42
+00:03:04,580 --> 00:03:06,510
+That was the seamier fast.
+
+43
+00:03:06,770 --> 00:03:14,150
+Our CNN's was released and it was voted open civies people's choice award at CVR conference.
+
+44
+00:03:14,210 --> 00:03:15,760
+Pattern recognition.
+
+45
+00:03:15,880 --> 00:03:22,670
+Unconference election two was later released when Basho musician was added to CNN.
+
+46
+00:03:22,670 --> 00:03:30,360
+So we do use Bachan musician and as easily as here which resulted in mapping improvements.
+
+47
+00:03:30,400 --> 00:03:35,230
+Map was mean average precision of 2 percent which was quite significant.
+
+48
+00:03:35,240 --> 00:03:41,810
+It was also a fine tuned a bit to get high resolution images giving it a 4 percent increase in map and
+
+49
+00:03:41,810 --> 00:03:48,260
+then yellow tree was fine too and even fitten introduced multi-skilled training better training to better
+
+50
+00:03:48,290 --> 00:03:51,920
+help detect small objects so that is it for you.
+
+51
+00:03:51,950 --> 00:03:56,590
+And now we move onto the next section which is the tensor flu outbreak detection API.

21. TensforFlow Object Detection/Go to the folder speciefid in this file

@@ -0,0 +1,12 @@
+INSTRUCTIONS
+
+
+GO TO THIS FOLDER FROM YOUR IPYTHON NOTEBOOK
+/home/deeplearningcv/models/models/research/object_detection
+
+OPEN THIS FILE
+/home/deeplearningcv/models/models/research/object_detection/object_detection_tutorial.ipynb
+
+OR 
+
+COPY THE IPYTHON NOTEBOOK FILE IN THIS FOLDER TO THE DIRECTORY - /home/deeplearningcv/models/models/research/object_detection

21. TensorFlow Object Detection API/1. Chapter Introduction.srt

@@ -0,0 +1,27 @@
+1
+00:00:00,990 --> 00:00:07,630
+Hi and welcome to Chapter 21 where we talk about the tenets of flu object detection API.
+
+2
+00:00:07,890 --> 00:00:11,050
+So this section is split up into tree bots.
+
+3
+00:00:11,070 --> 00:00:16,890
+First part we deal with the API install and set up then we start experimenting with the actual trained
+
+4
+00:00:16,950 --> 00:00:24,180
+API is just one extreme and resonant SSD and we would we use it on a web cam and videos and images and
+
+5
+00:00:24,180 --> 00:00:29,980
+then on twenty one point tree I go into detail and how you actually want to go about training a flow
+
+6
+00:00:30,030 --> 00:00:31,230
+of the action.
+
+7
+00:00:31,500 --> 00:00:34,840
+It's not easy but it is doable if you have a GP you.

21. TensorFlow Object Detection API/2. TFOD API Install and Setup.srt

@@ -0,0 +1,255 @@
+1
+00:00:00,420 --> 00:00:00,970
+OK.
+
+2
+00:00:01,020 --> 00:00:06,500
+So in twenty one point one we deal with a t i install and setup.
+
+3
+00:00:06,630 --> 00:00:10,060
+So let's talk a bit about tensor flows object action.
+
+4
+00:00:10,150 --> 00:00:16,380
+The TFT API is one of the more mature and relatively easy to use object action frameworks.
+
+5
+00:00:16,470 --> 00:00:19,600
+Most of them are actually quite finicky and tricky to use.
+
+6
+00:00:19,650 --> 00:00:22,560
+Typically most other obligation frameworks are finicky.
+
+7
+00:00:22,560 --> 00:00:28,140
+As I just said and difficult to use and they brick quite easily as I have a lot of moving parts tend
+
+8
+00:00:28,430 --> 00:00:33,540
+to flows of detection attempts to solve that by creating a framework API that uses tensor flow.
+
+9
+00:00:33,540 --> 00:00:34,760
+No surprise there.
+
+10
+00:00:34,770 --> 00:00:40,240
+To create an object detection modules using both our CNN family as well as the SSD family.
+
+11
+00:00:41,010 --> 00:00:47,550
+So while the TI FOTA Woody API makes it far easier than until the remittance it still has a bit of a
+
+12
+00:00:47,550 --> 00:00:50,100
+learning curve by the way.
+
+13
+00:00:50,100 --> 00:00:54,890
+This is actually an output of the tensor flow optimization API with an SSD.
+
+14
+00:00:54,990 --> 00:00:55,890
+It's quite cool isn't it.
+
+15
+00:00:57,180 --> 00:00:59,350
+So now let's talk about it install and setup.
+
+16
+00:00:59,370 --> 00:01:05,040
+Now if you're using the visual machine with this already installed you don't have to go through this.
+
+17
+00:01:05,040 --> 00:01:08,600
+However it's not that hard to do so I'm just gonna go through it step by step.
+
+18
+00:01:08,610 --> 00:01:13,020
+I'm not going to do it with you because I really have it installed on my machine and I don't I think
+
+19
+00:01:13,020 --> 00:01:14,790
+if I try to reinstall it I could mess things up.
+
+20
+00:01:15,360 --> 00:01:19,580
+But this is how I did it documented everything and it worked perfectly.
+
+21
+00:01:19,590 --> 00:01:24,430
+So what you do basically activate activated computer visual library.
+
+22
+00:01:24,690 --> 00:01:29,670
+We're not going to we're not going to install a disk because they can have a lot of clashes with packages
+
+23
+00:01:29,700 --> 00:01:34,480
+and libraries being you know messy with each other not mixing well.
+
+24
+00:01:34,590 --> 00:01:36,470
+So let's clone this environment.
+
+25
+00:01:36,490 --> 00:01:44,820
+So we go into the terminal and go condo create and let's call this TFT we named this environment that
+
+26
+00:01:45,070 --> 00:01:49,860
+so in future when you want to activate it you just go source activate TFT and it's there.
+
+27
+00:01:50,560 --> 00:01:58,200
+So anyway copy this line in your terminal and clone your directory your sorry CV environment and then
+
+28
+00:01:58,200 --> 00:02:06,070
+run this line here suited apt get install put above compiler Python pipe Perl Python Alexa Mel and python.
+
+29
+00:02:06,120 --> 00:02:07,400
+Okay okay.
+
+30
+00:02:07,530 --> 00:02:15,770
+Then we do pip install system pip install context lib Jupiter mapped lib and then go back.
+
+31
+00:02:15,780 --> 00:02:21,930
+So just go back to old man home directory make up for local models and then get clone.
+
+32
+00:02:22,110 --> 00:02:26,430
+This basically from here from this getup.
+
+33
+00:02:26,510 --> 00:02:35,130
+Link here fanciful models and go back again to directory and get cloned this as well and now as we go
+
+34
+00:02:35,130 --> 00:02:43,350
+back here go into this directory cocoa API Python API and end to make this will compile and build some
+
+35
+00:02:43,350 --> 00:02:49,050
+stuff that you need and basically just copy this line here and then decide comments here so don't actually
+
+36
+00:02:49,050 --> 00:02:50,850
+run this in a terminal.
+
+37
+00:02:50,910 --> 00:02:57,240
+So this is where we get the intensify models from so you use w get put above zip and you download this
+
+38
+00:02:57,240 --> 00:03:01,720
+link here and unzip this file and then you can delete this file afterwards.
+
+39
+00:03:01,740 --> 00:03:02,450
+It's fine.
+
+40
+00:03:02,760 --> 00:03:06,840
+And then what you do you get to part that we need to get everything working.
+
+41
+00:03:06,840 --> 00:03:12,540
+So we defined this part here and then we go to our protection bills and we run the tests and if the
+
+42
+00:03:12,540 --> 00:03:19,740
+stuff's tests run run successfully by opening this file we will know if this install works correctly.
+
+43
+00:03:19,740 --> 00:03:21,410
+So go ahead and try it on your own.
+
+44
+00:03:21,420 --> 00:03:27,930
+See if it works if there's any problems don't hesitate to contact me and all the times things change.
+
+45
+00:03:27,960 --> 00:03:29,850
+We did the updates.
+
+46
+00:03:29,850 --> 00:03:33,660
+So maybe just check this link if this isn't work first before you contact me.
+
+47
+00:03:33,660 --> 00:03:34,870
+See if there's anything here.
+
+48
+00:03:34,890 --> 00:03:37,620
+Menu maybe a new vision or a new dependency.
+
+49
+00:03:37,620 --> 00:03:40,330
+You never know.
+
+50
+00:03:40,500 --> 00:03:45,190
+Running the demo so download the python file in this folder.
+
+51
+00:03:45,200 --> 00:03:50,420
+I actually do have this file in the in the in my Python the book files here.
+
+52
+00:03:50,820 --> 00:03:56,540
+However they do work you don't don't run them from there they're actually copy and paste them into this
+
+53
+00:03:56,540 --> 00:03:57,310
+territory here.
+
+54
+00:03:57,660 --> 00:04:02,380
+So let me go to our virtual machine and I'll show you exactly where to find us.
+
+55
+00:04:02,400 --> 00:04:06,560
+Okay so we're back in a virtual machine and directory I want you to go to is.
+
+56
+00:04:06,750 --> 00:04:11,940
+Remember I told you based on our presentation here I wanted you to go to models models research oblique
+
+57
+00:04:11,940 --> 00:04:17,070
+detection and put this file here or resources file here that you don't want it.
+
+58
+00:04:17,160 --> 00:04:21,720
+Oh it's actually stored in your folder as well and your notebooks folder.
+
+59
+00:04:22,230 --> 00:04:29,250
+But anyhow let's go to the surgery and I'll show you where to put this file since models models research
+
+60
+00:04:29,400 --> 00:04:32,620
+and object let's type it in that type of it.
+
+61
+00:04:34,540 --> 00:04:38,500
+Object detection and you'll see it is a notebook somewhere.
+
+62
+00:04:39,280 --> 00:04:42,110
+This one here that is a file even a run from no one.
+
+63
+00:04:42,160 --> 00:04:42,430
+Okay.
+
+64
+00:04:43,060 --> 00:04:46,540
+So in the next chapter we're going to run this file and go to the Detroit tutorial.

21. TensorFlow Object Detection API/2.1 Download the code (for those not using the Virtual Machine).html

@@ -0,0 +1 @@
+<script type="text/javascript">window.location = "https://1drv.ms/u/s!AkTkTuTv8A66da5SgRE3zXCrtQA";</script>

21. TensorFlow Object Detection API/3. Experiment with a ResNet SSD on images, webcam and videos.srt

@@ -0,0 +1,471 @@
+1
+00:00:00,650 --> 00:00:00,960
+OK.
+
+2
+00:00:01,020 --> 00:00:07,680
+So welcome to the 21 point to where we actually start playing with our SSD inside by using sari
+
+3
+00:00:11,660 --> 00:00:18,240
+Hi welcome to Chapter 21 point to where we start experimenting with the object of action SSD based on
+
+4
+00:00:18,280 --> 00:00:23,080
+resonate and we do this on images webcams and all of them feel it and videos.
+
+5
+00:00:23,100 --> 00:00:24,580
+So let's get started.
+
+6
+00:00:25,170 --> 00:00:27,000
+So now we're here in a virtual machine.
+
+7
+00:00:27,000 --> 00:00:32,000
+And this project is going to be a bit different as we have done because we remember we created a new
+
+8
+00:00:32,010 --> 00:00:38,120
+environment so we have to launch what I put in the books from a new environment so fiercely that school
+
+9
+00:00:38,130 --> 00:00:49,140
+source caps lock is on source activate T.F. the API to get us to not use on the school.
+
+10
+00:00:49,140 --> 00:00:49,680
+There we go.
+
+11
+00:00:49,860 --> 00:00:51,950
+So that's a environment that we're in right now.
+
+12
+00:00:52,230 --> 00:00:52,850
+So that's good.
+
+13
+00:00:52,900 --> 00:00:54,010
+I will have to.
+
+14
+00:00:54,010 --> 00:00:58,450
+This will probably be any name you name that previously before if you're not using my pre-installed
+
+15
+00:00:58,710 --> 00:01:00,020
+with your machine.
+
+16
+00:01:00,180 --> 00:01:07,080
+So I put iPod it's on the books and it brings up a point on the book browser of Jupiter.
+
+17
+00:01:07,560 --> 00:01:12,950
+So now what I want us to go to is really there are two ways to get a set up.
+
+18
+00:01:13,040 --> 00:01:18,540
+You're supposed to download and I put it in the book file in the resources file that file was basically
+
+19
+00:01:19,420 --> 00:01:20,090
+this here.
+
+20
+00:01:20,240 --> 00:01:20,940
+All right.
+
+21
+00:01:20,940 --> 00:01:26,130
+However I left it in the artery here in case you wanted to manually copy and paste it into the directory.
+
+22
+00:01:26,130 --> 00:01:28,640
+So let's actually go ahead and do that.
+
+23
+00:01:28,650 --> 00:01:38,940
+So copy this file control Control-C and the doctor wants you to go to was the Saathiya models models.
+
+24
+00:01:39,450 --> 00:01:45,240
+Research sorry and object detection and pissed that file in to here.
+
+25
+00:01:45,430 --> 00:01:46,160
+OK.
+
+26
+00:01:46,470 --> 00:01:48,070
+This one here is actually ulo.
+
+27
+00:01:48,180 --> 00:01:51,780
+It's not actually going to work to paste it into here.
+
+28
+00:01:52,260 --> 00:01:59,220
+So now let's go to a Titan notebook browser and find this directory.
+
+29
+00:01:59,520 --> 00:02:00,740
+So find the file.
+
+30
+00:02:00,740 --> 00:02:01,670
+I should say so.
+
+31
+00:02:01,680 --> 00:02:04,270
+Good models research.
+
+32
+00:02:04,560 --> 00:02:07,630
+Scroll down to object detection
+
+33
+00:02:09,980 --> 00:02:16,610
+right here and let's launch this file.
+
+34
+00:02:16,610 --> 00:02:23,220
+Ok so here we go this file here is actually not a file I created I just modified it slightly.
+
+35
+00:02:23,260 --> 00:02:28,900
+This is a father comes intensive flows observation API and it allows you to basically play with the
+
+36
+00:02:28,900 --> 00:02:31,850
+different features in it it's in the official Demel.
+
+37
+00:02:32,180 --> 00:02:33,770
+So let's run the first box here.
+
+38
+00:02:33,780 --> 00:02:40,990
+So imports this plot of stuff in line by the way in case I haven't mentioned it to you what this does
+
+39
+00:02:40,990 --> 00:02:47,200
+is that it generates matplotlib Matlab plots inside a phone book as opposed to having it be like an
+
+40
+00:02:47,200 --> 00:02:49,080
+open TV and a new window.
+
+41
+00:02:49,540 --> 00:02:52,650
+So anyway that's duties imports here as well.
+
+42
+00:02:53,230 --> 00:02:56,340
+And let's run this block here.
+
+43
+00:02:56,470 --> 00:02:59,830
+These old directories would basically point to different models.
+
+44
+00:02:59,830 --> 00:03:01,230
+This is the SSD.
+
+45
+00:03:01,240 --> 00:03:04,220
+This is a resident SSD will be using that street and the cocoa.
+
+46
+00:03:04,420 --> 00:03:09,840
+That's a common object data set and it's going to download it the first time if you didn't already have
+
+47
+00:03:09,840 --> 00:03:12,490
+it saved is going to lose here.
+
+48
+00:03:12,490 --> 00:03:16,680
+Actually I do have it saved so it should not download.
+
+49
+00:03:16,690 --> 00:03:21,960
+I hope it is doing something so maybe it is downloading.
+
+50
+00:03:22,010 --> 00:03:25,750
+So anyway let's run this box and wait for that to finish.
+
+51
+00:03:27,520 --> 00:03:37,780
+And we load a little nap some help a code and then we do our detection boxes here and don't mind these
+
+52
+00:03:38,590 --> 00:03:41,260
+red things that look like it's going to be at URL.
+
+53
+00:03:41,450 --> 00:03:42,750
+This will still run.
+
+54
+00:03:42,790 --> 00:03:44,360
+So it's fine.
+
+55
+00:03:44,430 --> 00:03:44,920
+So right.
+
+56
+00:03:44,950 --> 00:03:47,010
+These these boxes have from now.
+
+57
+00:03:47,320 --> 00:03:52,450
+So let's do that we do this one and remember some of us do it again.
+
+58
+00:03:53,220 --> 00:03:54,370
+Let's run this one.
+
+59
+00:03:54,550 --> 00:03:57,760
+And that actually was not up wanted yet.
+
+60
+00:03:57,760 --> 00:03:58,720
+This is what we want.
+
+61
+00:03:58,730 --> 00:04:04,020
+So it goes through the images in a Test spot and it still takes a while to run.
+
+62
+00:04:04,480 --> 00:04:05,110
+To be fair.
+
+63
+00:04:05,500 --> 00:04:11,980
+And what it's going to do is basically it's going to take that image it found out and basically run
+
+64
+00:04:11,980 --> 00:04:17,340
+all of these SSD functions that require it to classify detect objects here.
+
+65
+00:04:17,590 --> 00:04:18,590
+So here we go.
+
+66
+00:04:18,730 --> 00:04:26,440
+So the first first test image it picked up this is a dog you can see it clearly says this woman.
+
+67
+00:04:26,690 --> 00:04:30,740
+I'm pressing control and moving my mouse button and we can see it's a dog here.
+
+68
+00:04:30,770 --> 00:04:32,080
+And you can see the probabilities.
+
+69
+00:04:32,260 --> 00:04:33,310
+It's a bit hard to make out.
+
+70
+00:04:33,310 --> 00:04:37,720
+Maybe we can actually change some parameters here to make this a little more legible.
+
+71
+00:04:37,870 --> 00:04:38,560
+Insightful.
+
+72
+00:04:38,550 --> 00:04:39,790
+I applied in the book.
+
+73
+00:04:40,080 --> 00:04:41,500
+It's a dog here and a dog here.
+
+74
+00:04:41,530 --> 00:04:43,720
+So it's quite good.
+
+75
+00:04:43,720 --> 00:04:45,790
+This is the image I use in my presentation slide.
+
+76
+00:04:45,970 --> 00:04:51,940
+You can see this is a kite kite kite person person and these are the probabilities which I can read
+
+77
+00:04:52,180 --> 00:04:55,260
+it looks like 60 tree just looks like a hundred.
+
+78
+00:04:55,270 --> 00:04:57,400
+But what it is it is.
+
+79
+00:04:57,400 --> 00:05:03,010
+So now let's try it on a webcam and I'm pretty much looking like a bit of a mess right now because it's
+
+80
+00:05:03,010 --> 00:05:03,630
+quite late.
+
+81
+00:05:03,640 --> 00:05:07,710
+And I have not comb my hair for a while but I'm sober to try this.
+
+82
+00:05:07,720 --> 00:05:12,840
+So let's run this webcam should come on any second now.
+
+83
+00:05:15,680 --> 00:05:15,940
+All right.
+
+84
+00:05:15,980 --> 00:05:18,470
+This is me in my natural element here.
+
+85
+00:05:18,680 --> 00:05:26,040
+And you can see that actually went up T-shirt on ice free advertising for Apple.
+
+86
+00:05:26,060 --> 00:05:30,520
+So this is me here and this is the person box that's affecting me right now.
+
+87
+00:05:30,560 --> 00:05:31,780
+So this is actually pretty cool.
+
+88
+00:05:31,820 --> 00:05:37,640
+So let me just close this and no let's try it out on the video.
+
+89
+00:05:37,650 --> 00:05:40,880
+So this is a dash cam video I downloaded off YouTube.
+
+90
+00:05:40,960 --> 00:05:43,130
+And so let's it it
+
+91
+00:05:49,000 --> 00:05:50,170
+it sometimes takes a while to load.
+
+92
+00:05:50,170 --> 00:05:51,850
+Oh there we go.
+
+93
+00:05:52,720 --> 00:05:54,630
+So this is a tip.
+
+94
+00:05:54,850 --> 00:05:56,150
+So this is pretty cool.
+
+95
+00:05:56,230 --> 00:05:58,190
+If I if I do say so.
+
+96
+00:05:58,750 --> 00:06:05,170
+So we're running it here detecting imprisons cause I think we just saw a bike from a mistaken call again
+
+97
+00:06:06,610 --> 00:06:08,090
+and the frame rate isn't that bad.
+
+98
+00:06:08,170 --> 00:06:12,010
+Honestly it being on a C.P. you not a GP.
+
+99
+00:06:12,280 --> 00:06:14,210
+This is actually pretty sick.
+
+100
+00:06:18,900 --> 00:06:20,460
+So let's close this video now.
+
+101
+00:06:21,560 --> 00:06:27,150
+So you've just run and experimented with SSD single shot detector.
+
+102
+00:06:27,550 --> 00:06:29,090
+So I hope you found this chapter fun.
+
+103
+00:06:29,090 --> 00:06:32,660
+I found it quite fun to play with this as well.
+
+104
+00:06:32,660 --> 00:06:35,680
+What you can do is put any video you want here.
+
+105
+00:06:36,150 --> 00:06:39,840
+Another dashcam video as well and any images you want.
+
+106
+00:06:39,860 --> 00:06:45,590
+We'll go there to see what fall they look at Image Pat.
+
+107
+00:06:45,600 --> 00:06:52,170
+Basically find fine image that is obviously not defined there.
+
+108
+00:06:53,410 --> 00:06:54,450
+Keep going.
+
+109
+00:06:56,320 --> 00:06:57,510
+Testament.
+
+110
+00:06:57,720 --> 00:07:03,960
+It is as if it's about OK syntactical test images.
+
+111
+00:07:05,850 --> 00:07:06,590
+This one here.
+
+112
+00:07:07,020 --> 00:07:08,700
+So this is the directory we're looking at.
+
+113
+00:07:08,710 --> 00:07:10,650
+We had what else images in it.
+
+114
+00:07:10,700 --> 00:07:12,010
+I'm not sure what's in this file.
+
+115
+00:07:12,040 --> 00:07:16,950
+I guess it's a source file for you want to put your source for images to sometimes.
+
+116
+00:07:17,020 --> 00:07:18,300
+So this is pretty cool.
+
+117
+00:07:18,530 --> 00:07:23,420
+So you can experiment with a web cam with your test images and watch your videos.
+
+118
+00:07:23,520 --> 00:07:23,730
+OK.

21. TensorFlow Object Detection API/4. How to Train a TFOD Model.srt

@@ -0,0 +1,503 @@
+1
+00:00:00,700 --> 00:00:01,040
+OK.
+
+2
+00:00:01,050 --> 00:00:03,310
+So welcome to twenty one point three.
+
+3
+00:00:03,510 --> 00:00:09,060
+Well actually talk about and tell you how to go about creating a custom tensor for up to detection module
+
+4
+00:00:09,260 --> 00:00:10,220
+model.
+
+5
+00:00:10,890 --> 00:00:17,490
+So training the tens of the action the training process of tensile flow of detection is honestly a bit
+
+6
+00:00:17,490 --> 00:00:23,970
+messy and this is probably one of the best of the detection Well most mature object libraries on the
+
+7
+00:00:23,970 --> 00:00:25,600
+market today market.
+
+8
+00:00:25,640 --> 00:00:33,540
+But an open source market you can say and it's explained fairly well it actually is bit tricky to use.
+
+9
+00:00:34,020 --> 00:00:36,530
+So this is a step I broke down from looking at it.
+
+10
+00:00:36,570 --> 00:00:40,310
+So first we prepare a data set in the T.F. record format.
+
+11
+00:00:40,380 --> 00:00:41,440
+That's a specific record.
+
+12
+00:00:41,460 --> 00:00:43,500
+I'll show you in the next few slides.
+
+13
+00:00:43,950 --> 00:00:48,570
+Then we need to create a class label file that's a dog P.B. text file.
+
+14
+00:00:48,570 --> 00:00:53,760
+Then we need to download a pre-trained Kokoo model and we need to set up the correct file that actually
+
+15
+00:00:53,760 --> 00:00:58,490
+structure configured the object action pipeline and then start training.
+
+16
+00:00:58,500 --> 00:01:00,690
+So let's go step by step.
+
+17
+00:01:00,720 --> 00:01:03,220
+So what does it T.F. record format.
+
+18
+00:01:03,270 --> 00:01:07,650
+So it tends to flow because the action expects so it treating and test data to be in this T.F. or the
+
+19
+00:01:07,650 --> 00:01:09,220
+school record format.
+
+20
+00:01:09,300 --> 00:01:11,460
+This is pretty much what it looks like here.
+
+21
+00:01:11,910 --> 00:01:17,220
+Luckily though we can convert existing data sets that's like in the past SQL View see data set which
+
+22
+00:01:17,220 --> 00:01:22,260
+is stored in Exod. Mosha directly to ATF records file by using the script.
+
+23
+00:01:22,290 --> 00:01:24,300
+They provide this for the here.
+
+24
+00:01:24,690 --> 00:01:26,790
+So it's quite myself.
+
+25
+00:01:27,510 --> 00:01:30,030
+And no we talk about a classily files.
+
+26
+00:01:30,030 --> 00:01:32,910
+So basically this is what the castable file looks like.
+
+27
+00:01:33,850 --> 00:01:41,350
+It's basically just on a dictionary Shukria where we have the IDs and label names.
+
+28
+00:01:41,350 --> 00:01:44,660
+So that's pretty much what we need to do with redefining.
+
+29
+00:01:44,830 --> 00:01:50,560
+If we're training a detector to detect let's say London underground tube signs you'll just put the name
+
+30
+00:01:50,560 --> 00:01:56,740
+of the class here and it's I.D. and you keep adding more and more items all objects here.
+
+31
+00:02:00,660 --> 00:02:03,510
+So now we have to use pre-trained model.
+
+32
+00:02:03,840 --> 00:02:04,250
+OK.
+
+33
+00:02:04,380 --> 00:02:11,860
+So we don't train this model and this is like the resonant model that we use in the previous chapter.
+
+34
+00:02:12,240 --> 00:02:13,890
+So we don't have one here.
+
+35
+00:02:13,920 --> 00:02:20,490
+So tensile flow who has several pre-treat models and one Ko-Ko and Coko is basically a large scale object
+
+36
+00:02:20,490 --> 00:02:26,020
+protection segmentation and captioning data set and basically has a lot of features here.
+
+37
+00:02:26,370 --> 00:02:31,320
+You can go to the Web site this Cocco decision it and you will find it.
+
+38
+00:02:31,900 --> 00:02:33,950
+And so you don't live with the models here.
+
+39
+00:02:34,230 --> 00:02:38,620
+This link carries you to all the models that are available on tensor.
+
+40
+00:02:39,030 --> 00:02:44,560
+And basically just use it downloaded onto mental health and on target here.
+
+41
+00:02:44,730 --> 00:02:46,470
+So that's what we do.
+
+42
+00:02:47,100 --> 00:02:50,180
+And this is a list of all the models of healable here.
+
+43
+00:02:50,280 --> 00:02:55,620
+It gives the speed and the map scope which is quite useful helps you choose which model is most appropriate
+
+44
+00:02:55,620 --> 00:02:56,590
+for your application.
+
+45
+00:02:58,530 --> 00:03:05,310
+So now we get to configure object detection pipeline said abjection pipeline configuration file is composed
+
+46
+00:03:05,310 --> 00:03:07,780
+of five sections.
+
+47
+00:03:07,800 --> 00:03:12,610
+Basically we have a model that we define here to configuration here.
+
+48
+00:03:13,110 --> 00:03:15,310
+Then we have the training config here.
+
+49
+00:03:15,480 --> 00:03:23,350
+So we add this through here and the in part Real talk about soon the valuation config and evaluation
+
+50
+00:03:23,370 --> 00:03:24,090
+in podrida.
+
+51
+00:03:24,090 --> 00:03:26,760
+So let's take a look at this file in more detail.
+
+52
+00:03:26,790 --> 00:03:31,400
+So this is a sample of the model file the model section here.
+
+53
+00:03:31,890 --> 00:03:34,900
+What it looks like is this is the model config.
+
+54
+00:03:35,130 --> 00:03:35,720
+I should say.
+
+55
+00:03:35,820 --> 00:03:36,340
+OK.
+
+56
+00:03:36,690 --> 00:03:42,330
+So we basically what we have to note is that we have some templates already that we can use inside of
+
+57
+00:03:42,470 --> 00:03:43,240
+of flow.
+
+58
+00:03:43,530 --> 00:03:49,530
+So we just basically make sure that all classes match up to the classes that enable custom are being
+
+59
+00:03:49,530 --> 00:03:50,910
+attacked to Dexter.
+
+60
+00:03:51,420 --> 00:03:53,850
+So this one was taken from the president one on one.
+
+61
+00:03:54,480 --> 00:03:56,940
+And those Treen in the Pascrell VRC dataset.
+
+62
+00:03:57,330 --> 00:04:01,950
+So as I said you don't need to rewrite this file just edit to one belonging to the preacher and model
+
+63
+00:04:02,070 --> 00:04:06,750
+to be using Dymo defines the model defines all unnecessary.
+
+64
+00:04:06,850 --> 00:04:15,300
+Our Or our CNN and as the parameters and when using pretreated models as best we leave this configuration
+
+65
+00:04:15,360 --> 00:04:19,170
+configuration file unchanged unchanged just a bit.
+
+66
+00:04:19,170 --> 00:04:26,320
+This red box here with the classes and notices that trade input Greedo and the yvel config and involved
+
+67
+00:04:26,360 --> 00:04:27,900
+in podrida sections.
+
+68
+00:04:27,900 --> 00:04:31,990
+So this is what we have to try and change these in red.
+
+69
+00:04:32,070 --> 00:04:33,840
+Basically directory mappings.
+
+70
+00:04:33,840 --> 00:04:38,010
+So we have to make sure that they're actually correct and pointing to the correct files that you want
+
+71
+00:04:38,010 --> 00:04:38,780
+to use.
+
+72
+00:04:38,820 --> 00:04:40,740
+This will be your label file.
+
+73
+00:04:40,800 --> 00:04:44,440
+This will be your record file you're treating UTF record file.
+
+74
+00:04:44,670 --> 00:04:46,270
+This is what disappoints do here.
+
+75
+00:04:46,500 --> 00:04:50,930
+So for validation and for record for training and for validation.
+
+76
+00:04:51,510 --> 00:04:53,990
+And this is Leavell parts of boat as well.
+
+77
+00:04:54,110 --> 00:04:54,680
+OK.
+
+78
+00:04:54,910 --> 00:04:59,450
+So remember the labels are the same ideas will be the same it's the same file.
+
+79
+00:04:59,890 --> 00:05:00,350
+OK.
+
+80
+00:05:04,360 --> 00:05:06,870
+So now here's the directory structure of the project.
+
+81
+00:05:06,910 --> 00:05:09,560
+So this is how and where we put our files.
+
+82
+00:05:09,850 --> 00:05:14,860
+So the PBX ex-felon label file that goes inside a directory called data.
+
+83
+00:05:15,250 --> 00:05:22,270
+Then we have a record files which is a train and evaluation of record files or validation whatever you
+
+84
+00:05:22,270 --> 00:05:24,230
+want to call it same thing.
+
+85
+00:05:24,640 --> 00:05:29,820
+And then we have a new directory here models and then we have subdirectory model here.
+
+86
+00:05:30,160 --> 00:05:32,360
+This is where we put our pipeline config file.
+
+87
+00:05:32,500 --> 00:05:33,990
+That's this file here.
+
+88
+00:05:34,390 --> 00:05:40,330
+And then we just have the trained directory and the evaluation directory here as well under the model
+
+89
+00:05:40,660 --> 00:05:46,340
+inside the model form which is inside of the models for the here.
+
+90
+00:05:46,350 --> 00:05:48,820
+So now this is how we start the training process.
+
+91
+00:05:48,890 --> 00:05:56,810
+So we go to basically terminal and we just copy this line of code here and make sure to code the lines
+
+92
+00:05:56,810 --> 00:05:57,970
+in a read here.
+
+93
+00:05:58,280 --> 00:06:03,090
+These correspond to the model you are using and the directory that we just created here.
+
+94
+00:06:03,260 --> 00:06:03,710
+OK.
+
+95
+00:06:03,890 --> 00:06:10,010
+This is the day that actually it needs to be pointing to and it doesn't do that.
+
+96
+00:06:10,010 --> 00:06:16,200
+And then you can actually bring up tents board to monitor your treating progress which is pretty cool.
+
+97
+00:06:16,700 --> 00:06:18,770
+It's going to be district directory here.
+
+98
+00:06:19,030 --> 00:06:22,230
+That again does your data directly at the specified here.
+
+99
+00:06:25,340 --> 00:06:30,590
+And then before that I should have mentioned this earlier but it's important when you're labeling it
+
+100
+00:06:30,590 --> 00:06:35,450
+images use a software that actually produces it in the correct format.
+
+101
+00:06:35,450 --> 00:06:38,120
+So this is how we use annotations here.
+
+102
+00:06:38,150 --> 00:06:43,710
+So this is my wife with my dog Samuel and software we use as a label.
+
+103
+00:06:43,800 --> 00:06:50,190
+AMG I think this is two elves they have here this really label I am the label image obviously.
+
+104
+00:06:50,540 --> 00:06:52,200
+So download it if you want to do it.
+
+105
+00:06:52,220 --> 00:06:55,910
+It's available for Windows Mac and Linux.
+
+106
+00:06:55,910 --> 00:07:01,690
+And this is the format that Pascal VRC Exham a format that we use now.
+
+107
+00:07:01,820 --> 00:07:09,380
+It is not what we used us but we generate the image anti-Chavez in this format using the software.
+
+108
+00:07:09,410 --> 00:07:11,160
+It actually does it automatically for you.
+
+109
+00:07:11,630 --> 00:07:17,420
+And we can use our tents for a script I mentioned earlier to convert this file directly back to the
+
+110
+00:07:17,420 --> 00:07:20,170
+text of the record files.
+
+111
+00:07:20,270 --> 00:07:21,910
+So this is a summary here.
+
+112
+00:07:22,010 --> 00:07:24,890
+We didn't do a full project here for the following reasons.
+
+113
+00:07:25,160 --> 00:07:30,390
+Training and SSTO even a faster are CNN One is you is very impractical.
+
+114
+00:07:30,440 --> 00:07:31,630
+It is going to take forever.
+
+115
+00:07:31,700 --> 00:07:38,840
+So you definitely need a GP or a cloud to use to effectively train this A.F. all the data sets are there
+
+116
+00:07:38,870 --> 00:07:46,190
+huge there are quite a few gigs of storage and also to setting up a GPU when a local system is a nightmare
+
+117
+00:07:46,190 --> 00:07:46,790
+sometimes.
+
+118
+00:07:46,880 --> 00:07:53,660
+It's a very scary task but once you get it working it's good you feel very happy because it's so much
+
+119
+00:07:53,660 --> 00:07:54,950
+faster.
+
+120
+00:07:55,620 --> 00:07:58,150
+So I've of all time general steps here.
+
+121
+00:07:58,220 --> 00:08:05,040
+The Trina's model there are also some good tutorials I found online that do this as well.
+
+122
+00:08:05,090 --> 00:08:10,430
+Basically try to make it as simple as possible if you going through all the steps telling you what to
+
+123
+00:08:10,430 --> 00:08:13,210
+pay attention to and what's important.
+
+124
+00:08:13,220 --> 00:08:16,880
+I've actually tried this on my system as well so I know what works.
+
+125
+00:08:16,910 --> 00:08:23,150
+So I wish you all the best of luck when making your own object to actual text detectors.
+
+126
+00:08:23,300 --> 00:08:23,610
+Thank you.

22. Object Detection with YOLO & Darkflow Build a London Underground Sign Detector/1. Chapter Introduction.srt

@@ -0,0 +1,23 @@
+1
+00:00:00,280 --> 00:00:06,920
+Hi and welcome to Chapter 22 where we look at object detection using yellow vision tree and dark blue.
+
+2
+00:00:06,970 --> 00:00:09,680
+So this section is built up into tree parts.
+
+3
+00:00:09,690 --> 00:00:16,440
+Firstly we get you up and running by installing yellow dock nets and outflow then we start experimenting
+
+4
+00:00:16,440 --> 00:00:24,750
+with real and still images webcam feed and videos and then we build on a yellow object detector and
+
+5
+00:00:24,750 --> 00:00:28,350
+we're going to detect London Underground signs in our project.
+
+6
+00:00:28,740 --> 00:00:30,200
+So let's get started.

22. Object Detection with YOLO & Darkflow Build a London Underground Sign Detector/2. Setting up and install Yolo DarkNet and DarkFlow.srt

@@ -0,0 +1,363 @@
+1
+00:00:00,380 --> 00:00:06,570
+You say hi welcome to Chapter 22 point one where we get to Yale and Dartmouth and Da'ath set up and
+
+2
+00:00:06,570 --> 00:00:07,770
+installed.
+
+3
+00:00:08,340 --> 00:00:10,010
+So just to remind you.
+
+4
+00:00:10,010 --> 00:00:14,100
+Yellow stands for you only look once and it's a pretty awesome object doctor.
+
+5
+00:00:14,370 --> 00:00:20,190
+You can take a look at the Web site here it looks a bit shady but it's very reputable and very very
+
+6
+00:00:20,190 --> 00:00:20,520
+good.
+
+7
+00:00:20,610 --> 00:00:21,800
+It's very good technology.
+
+8
+00:00:21,810 --> 00:00:22,860
+Those guys developed.
+
+9
+00:00:23,100 --> 00:00:25,790
+So basically we have to install dot at first.
+
+10
+00:00:25,800 --> 00:00:27,170
+And what does darknet.
+
+11
+00:00:27,300 --> 00:00:29,530
+It's the official name for the EULA framework.
+
+12
+00:00:29,550 --> 00:00:32,360
+I mean these guys are kind of awesome should read somebody's papers.
+
+13
+00:00:32,380 --> 00:00:36,120
+They're very entertaining and very informative as well.
+
+14
+00:00:36,120 --> 00:00:41,760
+So to get this installed that Cisco to we go back to our terminal here.
+
+15
+00:00:42,000 --> 00:00:44,080
+This is all commands we enter into minal.
+
+16
+00:00:44,430 --> 00:00:48,940
+We stay in a home directory and we make a folder called dot net.
+
+17
+00:00:49,110 --> 00:00:51,270
+We get Clun repository here.
+
+18
+00:00:51,660 --> 00:00:58,170
+And then we go back we go into this net folder and we go to make and then we use this file to get the
+
+19
+00:00:58,170 --> 00:00:59,130
+weights here.
+
+20
+00:00:59,400 --> 00:01:05,190
+And then we just run this line in blue to executed on the test image that they've provided in one of
+
+21
+00:01:05,230 --> 00:01:10,330
+their sample their trees and in their dark directory you'll see a file called predictions.
+
+22
+00:01:10,500 --> 00:01:13,890
+And this will be the output from the test file.
+
+23
+00:01:13,920 --> 00:01:21,130
+So when you run this line here I said you basically run this line and blue copy and paste it.
+
+24
+00:01:21,210 --> 00:01:22,130
+This is what you will see.
+
+25
+00:01:22,120 --> 00:01:25,430
+Takes about maybe 10 seconds to run a little to model it for us.
+
+26
+00:01:25,500 --> 00:01:29,470
+We just see the output of the model here and it basically went in.
+
+27
+00:01:29,490 --> 00:01:34,530
+Once it's done you'll see this I didn't paste it felt all the way in because it would but it would have
+
+28
+00:01:34,530 --> 00:01:35,470
+been too small.
+
+29
+00:01:35,880 --> 00:01:42,210
+But you see this at the end it's a probability object probabilities found here.
+
+30
+00:01:42,930 --> 00:01:44,000
+And that's pretty cool.
+
+31
+00:01:44,010 --> 00:01:46,300
+So you know what to find this file now.
+
+32
+00:01:46,380 --> 00:01:50,390
+It's basically a file in the directory called predictions.
+
+33
+00:01:50,640 --> 00:01:53,310
+And this is it can enter any sample image you want.
+
+34
+00:01:53,550 --> 00:02:01,280
+Basically it files to test files a document data and you can this is a self-hater recently when it's
+
+35
+00:02:01,280 --> 00:02:02,470
+over my friend.
+
+36
+00:02:02,640 --> 00:02:05,140
+And you can just try it quite easily.
+
+37
+00:02:05,310 --> 00:02:05,790
+You can try.
+
+38
+00:02:05,970 --> 00:02:08,760
+Was your half horses anything you want.
+
+39
+00:02:08,760 --> 00:02:12,430
+So feel have fun playing with the yellow.
+
+40
+00:02:13,230 --> 00:02:15,820
+So but what else can we do from you.
+
+41
+00:02:16,020 --> 00:02:17,970
+Was using yellow from the command line.
+
+42
+00:02:18,240 --> 00:02:20,790
+But can we use it inside of Pitre.
+
+43
+00:02:21,060 --> 00:02:23,300
+Well yes we can with Dulfer.
+
+44
+00:02:23,580 --> 00:02:26,040
+And I'll now introduce you to darf flow in the next chapter.
+
+45
+00:02:27,780 --> 00:02:28,080
+OK.
+
+46
+00:02:28,080 --> 00:02:32,320
+So just to show you guys how this works without actually showing you images.
+
+47
+00:02:32,370 --> 00:02:35,550
+We're going to actually run this in our machine.
+
+48
+00:02:35,640 --> 00:02:36,210
+OK.
+
+49
+00:02:36,510 --> 00:02:40,570
+So somebody said we have to go to type in here.
+
+50
+00:02:40,810 --> 00:02:42,670
+Dot dot net Right.
+
+51
+00:02:42,690 --> 00:02:43,090
+Yep.
+
+52
+00:02:43,170 --> 00:02:44,940
+See the dot net.
+
+53
+00:02:44,980 --> 00:02:49,430
+So actually I didn't see what they call it actually yeah it is.
+
+54
+00:02:49,430 --> 00:02:50,300
+You don't know it.
+
+55
+00:02:50,310 --> 00:02:51,080
+Why didn't it
+
+56
+00:02:54,890 --> 00:02:55,350
+go.
+
+57
+00:02:55,770 --> 00:02:58,040
+So we can we're the here.
+
+58
+00:02:58,050 --> 00:03:05,940
+So now let's just go to the outline and I may have done that a bit too quickly because it actually pressed
+
+59
+00:03:06,020 --> 00:03:08,680
+into soon as I pasted that line.
+
+60
+00:03:08,700 --> 00:03:10,890
+This takes about 10 seconds or so to load the models.
+
+61
+00:03:10,900 --> 00:03:12,150
+It's going to be done quickly.
+
+62
+00:03:12,390 --> 00:03:17,170
+So now it's loading the weights and it's done and now with what it's going to do it's going to classify
+
+63
+00:03:17,170 --> 00:03:18,390
+that test image.
+
+64
+00:03:18,630 --> 00:03:20,730
+So let's go to dinner directory here.
+
+65
+00:03:21,010 --> 00:03:21,640
+All right.
+
+66
+00:03:21,720 --> 00:03:23,600
+So that's going to be in the Dot Net directory.
+
+67
+00:03:23,640 --> 00:03:27,090
+And this was the output here that we're going to generate.
+
+68
+00:03:27,150 --> 00:03:30,840
+And let's see if that was probably my previously saved image.
+
+69
+00:03:31,020 --> 00:03:31,850
+Yep it was.
+
+70
+00:03:31,850 --> 00:03:34,630
+It is going to make a new image soon.
+
+71
+00:03:34,770 --> 00:03:35,490
+So let's wait
+
+72
+00:03:39,510 --> 00:03:47,620
+spot usually took last time I think about 25 or so seconds when the time that search should be done
+
+73
+00:03:47,710 --> 00:03:48,850
+soon.
+
+74
+00:03:48,850 --> 00:03:49,930
+There we go.
+
+75
+00:03:49,930 --> 00:03:53,140
+It took six seconds and probably have something running in the background.
+
+76
+00:03:53,440 --> 00:03:54,500
+So here we go.
+
+77
+00:03:54,550 --> 00:04:00,420
+So this is the test image here that it updated its very nicely and neatly labeled here.
+
+78
+00:04:00,760 --> 00:04:05,310
+So if you wanted to do more as images go to your data directory here.
+
+79
+00:04:05,620 --> 00:04:10,960
+And this is actually where I have the self-pay used in the test and the presentation I should say.
+
+80
+00:04:11,200 --> 00:04:15,720
+So let's try this one here called person so let's see how that works.
+
+81
+00:04:15,730 --> 00:04:17,650
+So let's go back to this line.
+
+82
+00:04:17,710 --> 00:04:19,410
+However we don't use dog.
+
+83
+00:04:19,480 --> 00:04:22,750
+We use person.
+
+84
+00:04:22,790 --> 00:04:33,860
+So again the little the model when it first folded the slides for you guys no.
+
+85
+00:04:34,130 --> 00:04:35,520
+All right there we go.
+
+86
+00:04:35,900 --> 00:04:37,840
+So this is the up it up to file here.
+
+87
+00:04:38,330 --> 00:04:42,170
+So let's go back to a dock and a directory and look at our predictions file.
+
+88
+00:04:42,410 --> 00:04:43,520
+And this is pretty cool.
+
+89
+00:04:43,550 --> 00:04:48,200
+We have a person a horse and a dog all accurately labeled and very neatly done too.
+
+90
+00:04:48,530 --> 00:04:51,290
+So yellow is pretty awesome as you can see.
+
+91
+00:04:51,590 --> 00:04:56,390
+So I encourage you to experiment with doing pictures and get the hang of playing with yellow.

22. Object Detection with YOLO & Darkflow Build a London Underground Sign Detector/2.1 Guide to the MacOS Install.html

@@ -0,0 +1 @@
+<script type="text/javascript">window.location = "https://gist.github.com/simonw/0f93bec220be9cf8250533b603bf6dba";</script>

22. Object Detection with YOLO & Darkflow Build a London Underground Sign Detector/2.2 Download the YOLO files (if not using the VM).html

@@ -0,0 +1 @@
+<script type="text/javascript">window.location = "https://1drv.ms/u/s!AkTkTuTv8A66dAvPsd9zbDSYLeI";</script>

22. Object Detection with YOLO & Darkflow Build a London Underground Sign Detector/3. Experiment with YOLO on still images, webcam and videos.srt

@@ -0,0 +1,547 @@
+1
+00:00:00,470 --> 00:00:06,750
+I will come to Chapter 22 point to where we talk about dark blue and dark blue is basically how we can
+
+2
+00:00:06,750 --> 00:00:13,150
+interface with Ulo inside of plate and run some cool tests on images videos and draw web cam as well.
+
+3
+00:00:14,240 --> 00:00:20,150
+So basically this is actually is a Jeff of basically being using the video.
+
+4
+00:00:20,150 --> 00:00:21,360
+It's pretty cool.
+
+5
+00:00:21,380 --> 00:00:22,990
+Hope it's distracting you right now.
+
+6
+00:00:23,360 --> 00:00:27,850
+But basically this is how we actually create the story and create.
+
+7
+00:00:27,890 --> 00:00:35,360
+But this is how we actually set up the environment for Ulo Darfur I should say as the install isn't
+
+8
+00:00:35,360 --> 00:00:35,620
+dead.
+
+9
+00:00:35,630 --> 00:00:38,950
+Hard to do just a bunch of things we have to do.
+
+10
+00:00:39,200 --> 00:00:44,490
+So what I've done here basically have created a new environment using Anaconda.
+
+11
+00:00:44,510 --> 00:00:46,030
+Basically you can call whatever you want.
+
+12
+00:00:46,030 --> 00:00:51,830
+I've used I use it as my tensor flew with the environment previously so you can call it that if you
+
+13
+00:00:51,830 --> 00:00:55,520
+want to call it you know anything you want to do.
+
+14
+00:00:55,520 --> 00:00:56,930
+It's already installed right now.
+
+15
+00:00:56,930 --> 00:01:02,300
+So if you're using this virtual machine which I hope we do which I hope you don't want it you don't
+
+16
+00:01:02,300 --> 00:01:06,360
+have to do this and tedious install again but it doesn't actually take that long.
+
+17
+00:01:06,410 --> 00:01:12,020
+So just run these commands line by line in the terminal and it should be fine.
+
+18
+00:01:12,150 --> 00:01:12,660
+OK.
+
+19
+00:01:13,220 --> 00:01:14,130
+Except for this one.
+
+20
+00:01:14,180 --> 00:01:18,050
+Basically you're going to have to create the environment manually on your own but that's not too hard
+
+21
+00:01:18,050 --> 00:01:18,340
+to do.
+
+22
+00:01:18,340 --> 00:01:21,780
+So sort of duck flow install.
+
+23
+00:01:21,790 --> 00:01:28,130
+So now go see a terminal and basically start with your home territory in the terminal and make it an
+
+24
+00:01:28,140 --> 00:01:29,160
+article of flow.
+
+25
+00:01:29,160 --> 00:01:36,690
+Go into it install site and basically clone this repository here go to that for the Pipp install and
+
+26
+00:01:36,690 --> 00:01:43,950
+do this business all talk by the way and go get this get the weights here and then basically start to
+
+27
+00:01:44,010 --> 00:01:48,070
+this surgery hip to see if she can.
+
+28
+00:01:48,160 --> 00:01:50,860
+And now let's mess around with the all in Python.
+
+29
+00:01:50,900 --> 00:01:57,460
+So now he can actually import duckbilled thought in that build and import from the starchy import this
+
+30
+00:01:57,670 --> 00:01:58,660
+function here.
+
+31
+00:01:59,110 --> 00:02:04,570
+So no I'm going to go back to this and we're going to go to our vision machine and actually use this
+
+32
+00:02:04,570 --> 00:02:05,070
+now.
+
+33
+00:02:05,370 --> 00:02:05,680
+OK.
+
+34
+00:02:05,710 --> 00:02:11,830
+So from what I play on the browser you don't go into the deepening C-v territory you go into awfuller
+
+35
+00:02:12,520 --> 00:02:17,840
+and go to the Darfur master directory here and you'll see a file called a tutorial.
+
+36
+00:02:18,100 --> 00:02:19,520
+Open this notebook here.
+
+37
+00:02:19,570 --> 00:02:22,360
+This is why I compiled you guys.
+
+38
+00:02:22,680 --> 00:02:25,850
+And basically this is how we used offline side of Pitre.
+
+39
+00:02:26,250 --> 00:02:29,840
+So firstly let's run this block of code.
+
+40
+00:02:30,000 --> 00:02:37,020
+What this does here this actually loads our model All right using tensor Flo's T.F. net and all this
+
+41
+00:02:37,020 --> 00:02:40,590
+stuff we actually load the model we want from the CFQ directory.
+
+42
+00:02:40,680 --> 00:02:42,190
+We load the so we want.
+
+43
+00:02:42,420 --> 00:02:43,720
+We set up trouble.
+
+44
+00:02:43,980 --> 00:02:49,770
+And if you're using a GPS you can specify AGP one however you you use false since this is a virtual
+
+45
+00:02:49,770 --> 00:02:52,810
+machine and we don't have access to you from here.
+
+46
+00:02:52,810 --> 00:02:53,380
+All right.
+
+47
+00:02:53,460 --> 00:02:58,570
+And then we passed these options that we created here to RTFM that class object.
+
+48
+00:02:58,630 --> 00:02:59,600
+Right.
+
+49
+00:03:00,420 --> 00:03:06,750
+So you can see it basically updated the model here loaded everything happening successfully around and
+
+50
+00:03:06,750 --> 00:03:08,060
+we now have a model.
+
+51
+00:03:08,370 --> 00:03:15,150
+So now what we need to do is convert our open C-v BGR image to R.G. format.
+
+52
+00:03:15,150 --> 00:03:21,990
+This is something that's pretty annoying about open C-v with loads images and b g r by default.
+
+53
+00:03:22,010 --> 00:03:26,160
+RGV said it see if you can be actually made a function that can get it for us.
+
+54
+00:03:26,160 --> 00:03:27,340
+So it's not a big deal.
+
+55
+00:03:27,660 --> 00:03:30,600
+So we're running this on our sample horse's image.
+
+56
+00:03:30,600 --> 00:03:32,850
+And let's go and see what that image looks like.
+
+57
+00:03:32,860 --> 00:03:38,100
+Go to darf flu go to think of a sample image here.
+
+58
+00:03:38,430 --> 00:03:41,760
+And I believe the one we called with horses or sample horses.
+
+59
+00:03:41,760 --> 00:03:42,540
+This one.
+
+60
+00:03:42,870 --> 00:03:50,780
+So now we've run this image inside of our vitamin book and we got basically some bounding boxes here
+
+61
+00:03:51,360 --> 00:03:57,410
+and some confidence scores and a label but we don't actually have an image.
+
+62
+00:03:57,450 --> 00:04:03,190
+So that's the play our results using open C-v Tado.
+
+63
+00:04:03,190 --> 00:04:05,380
+So no this is actually pretty cool no.
+
+64
+00:04:05,620 --> 00:04:11,200
+So we've displayed at our bounding boxes here using up and see if we would do labels.
+
+65
+00:04:11,440 --> 00:04:13,450
+So this is pretty neat.
+
+66
+00:04:13,450 --> 00:04:16,080
+So we can encapsulate that function.
+
+67
+00:04:16,120 --> 00:04:19,470
+That's actually the function we call here display results.
+
+68
+00:04:19,480 --> 00:04:21,290
+I'm actually using this I'm not entirely sure.
+
+69
+00:04:21,330 --> 00:04:23,170
+I think I was using it.
+
+70
+00:04:23,170 --> 00:04:24,290
+No I'm not using it.
+
+71
+00:04:24,340 --> 00:04:28,430
+It's pretty much pointless unless you are missing it down here.
+
+72
+00:04:28,510 --> 00:04:30,530
+Sorry my mistake.
+
+73
+00:04:30,780 --> 00:04:32,320
+Happened to me to me a lot.
+
+74
+00:04:32,320 --> 00:04:35,970
+Read my own code wrong so much.
+
+75
+00:04:36,080 --> 00:04:41,630
+I think something is wrong with me sometimes anyhow so this is a code here to load it to run your little
+
+76
+00:04:41,680 --> 00:04:42,970
+troll webcam.
+
+77
+00:04:42,970 --> 00:04:46,100
+So we basically load them all exactly the same way we did before.
+
+78
+00:04:46,390 --> 00:04:49,320
+We actually don't need to do it twice but it's part of.
+
+79
+00:04:49,320 --> 00:04:55,140
+I just posted it here in case you wanted to run this separately from this on top and we re-initialize
+
+80
+00:04:55,140 --> 00:05:00,310
+a web cam here from open TV and we run this so it's going to take a while to run out of what 10 seconds
+
+81
+00:05:00,370 --> 00:05:01,740
+seven seconds before.
+
+82
+00:05:02,140 --> 00:05:05,910
+And now our webcam for him is going to pop up here shortly.
+
+83
+00:05:07,340 --> 00:05:08,810
+Well no results.
+
+84
+00:05:08,830 --> 00:05:09,660
+It's not defined.
+
+85
+00:05:09,730 --> 00:05:11,590
+That's quite funny.
+
+86
+00:05:11,620 --> 00:05:13,500
+So let's do this again sorry.
+
+87
+00:05:13,530 --> 00:05:14,200
+By the way it's
+
+88
+00:05:18,120 --> 00:05:23,580
+now one thing you're going to notice is that it actually runs very slowly on a super UBS system.
+
+89
+00:05:23,670 --> 00:05:29,180
+The frame rate is nowhere near as good as SSD.
+
+90
+00:05:29,290 --> 00:05:34,420
+Any minute any second now I should say webcam for him is going to pop up here.
+
+91
+00:05:45,750 --> 00:05:50,540
+We have so you have a webcam image.
+
+92
+00:05:50,540 --> 00:05:54,410
+Right now it doesn't look smooth at all.
+
+93
+00:05:54,410 --> 00:06:00,770
+And we see multiple boxes here now the multiple box problem could be is actually because of the trouble
+
+94
+00:06:00,770 --> 00:06:04,960
+we set if we set this close system because it's very slow.
+
+95
+00:06:05,030 --> 00:06:08,290
+If we said attritional And actually I don't have the Trishul parameter here.
+
+96
+00:06:08,570 --> 00:06:12,550
+So let's go back to the top and set Trishul here.
+
+97
+00:06:14,110 --> 00:06:15,640
+And DUDAS here.
+
+98
+00:06:16,790 --> 00:06:17,200
+All right.
+
+99
+00:06:17,240 --> 00:06:18,720
+You can send us to point five.
+
+100
+00:06:18,710 --> 00:06:22,570
+You'll actually see much less banging boxes on that image.
+
+101
+00:06:22,580 --> 00:06:27,480
+So now let's run Ulo basically on video.
+
+102
+00:06:29,740 --> 00:06:31,400
+See how it looks.
+
+103
+00:06:31,430 --> 00:06:31,720
+OK.
+
+104
+00:06:31,760 --> 00:06:33,140
+May have seen this in my entire video.
+
+105
+00:06:33,140 --> 00:06:38,890
+So this is it running frame by frame on an elephant on the couch and a person sometimes.
+
+106
+00:06:38,900 --> 00:06:40,820
+So let's stop this for now.
+
+107
+00:06:41,300 --> 00:06:47,010
+And what we can do if you want to experiment is let's actually load this model.
+
+108
+00:06:47,100 --> 00:06:49,160
+All right separately though.
+
+109
+00:06:49,550 --> 00:06:55,430
+So let's actually we can just run it loaded from up here and it's said the Trishul two point six.
+
+110
+00:06:55,430 --> 00:06:55,850
+All right.
+
+111
+00:06:58,450 --> 00:07:02,880
+Doesn't take that long to do maybe about 10 seconds to run.
+
+112
+00:07:02,880 --> 00:07:03,890
+There we go it's done.
+
+113
+00:07:03,960 --> 00:07:07,690
+And now let's go back to this video.
+
+114
+00:07:09,790 --> 00:07:10,660
+There we go.
+
+115
+00:07:10,660 --> 00:07:11,920
+And you should see definitely.
+
+116
+00:07:11,920 --> 00:07:12,780
+There we go.
+
+117
+00:07:12,790 --> 00:07:14,200
+It's just one box.
+
+118
+00:07:14,200 --> 00:07:16,570
+And this is actually for me it isn't about.
+
+119
+00:07:16,890 --> 00:07:17,970
+And actually no disappear.
+
+120
+00:07:17,980 --> 00:07:23,260
+So maybe that threshold is too high but you can see we only have one bounding box here and it's getting
+
+121
+00:07:23,430 --> 00:07:27,480
+it as an elephant right all the time except when it disappears.
+
+122
+00:07:27,700 --> 00:07:29,310
+So this is pretty sick.
+
+123
+00:07:30,400 --> 00:07:32,330
+So let's close up now.
+
+124
+00:07:32,690 --> 00:07:37,200
+So that concludes only yellow a tutorial.
+
+125
+00:07:37,520 --> 00:07:39,710
+These are the staunch options out of left here.
+
+126
+00:07:40,070 --> 00:07:45,410
+You may have seen me scrolling back and forth trying to figure out why discoid wasn't working.
+
+127
+00:07:45,590 --> 00:07:52,130
+What happens in open sea is that if something opens a webcam and then this code it crashes inside like
+
+128
+00:07:52,130 --> 00:07:54,500
+this could disappear results were not found.
+
+129
+00:07:54,830 --> 00:07:57,480
+What happens is that we now need to run this line here.
+
+130
+00:07:57,740 --> 00:08:00,280
+These two lines to be released a webcam.
+
+131
+00:08:00,320 --> 00:08:05,210
+So what happened is that when I try to initiate a webcam again it just got stuck and it had to go to
+
+132
+00:08:05,210 --> 00:08:08,020
+kernel and restart the notebook and wait bit.
+
+133
+00:08:08,350 --> 00:08:13,820
+So it's good knowledge to know when you're playing with this and maybe something you mess up something.
+
+134
+00:08:13,820 --> 00:08:14,790
+You know what to do.
+
+135
+00:08:15,050 --> 00:08:21,790
+Just run this line to basically recapture your webcam.
+
+136
+00:08:21,820 --> 00:08:26,060
+All right so later on we're going to see how to actually make and model.
+
+137
+00:08:26,070 --> 00:08:28,640
+So this is a model we were going to make in the next section.

22. Object Detection with YOLO & Darkflow Build a London Underground Sign Detector/4. Build your own YOLO Object Detector - Detecting London Underground Signs.srt

@@ -0,0 +1,1011 @@
+1
+00:00:00,580 --> 00:00:01,190
+I guess.
+
+2
+00:00:01,200 --> 00:00:08,970
+Hi and welcome back to Chapter 22 point tree where we're about to build on our very own customized yellow
+
+3
+00:00:09,060 --> 00:00:10,180
+object detector.
+
+4
+00:00:10,590 --> 00:00:15,980
+This one is going to detect London Underground signs so let's get started and see how we do this.
+
+5
+00:00:15,990 --> 00:00:17,380
+So just something to note.
+
+6
+00:00:17,380 --> 00:00:22,620
+Firstly doing this without a GP who is going to be very slow and we're not going to actually make a
+
+7
+00:00:22,620 --> 00:00:27,100
+very good object the doctor is going to have fit quite a bit.
+
+8
+00:00:27,330 --> 00:00:31,890
+And also because I'm teaching from my virtual box I don't have access to my GP so we don't have a choice
+
+9
+00:00:31,920 --> 00:00:38,610
+but to use OCP you and also the Trina's using a GP who doesn't require a different set up than anything
+
+10
+00:00:38,730 --> 00:00:39,480
+we're doing here.
+
+11
+00:00:39,750 --> 00:00:43,640
+Just a few additional commands but I'll show it to you in this slide.
+
+12
+00:00:43,650 --> 00:00:44,570
+OK.
+
+13
+00:00:45,460 --> 00:00:53,310
+So first of all to make a custom image doctor you basically have to create your own custom data set.
+
+14
+00:00:53,340 --> 00:00:58,880
+So this software called label image which is found here available for Windows Mac and Linux.
+
+15
+00:00:58,920 --> 00:01:00,480
+It's quite easy to use.
+
+16
+00:01:00,510 --> 00:01:06,600
+And what do you have to do is basically set select a form that will open a directory that you want to
+
+17
+00:01:06,600 --> 00:01:07,680
+use.
+
+18
+00:01:07,680 --> 00:01:15,780
+This is the directory I used here for my TEFL images and you basically said to format Pascal VRC and
+
+19
+00:01:15,810 --> 00:01:18,010
+basically you just drag and drop.
+
+20
+00:01:18,060 --> 00:01:19,860
+Actually I can show it to you right now.
+
+21
+00:01:19,860 --> 00:01:22,980
+Let's quickly go to label image here.
+
+22
+00:01:22,980 --> 00:01:23,850
+All right.
+
+23
+00:01:24,180 --> 00:01:27,960
+So this is actually some images here.
+
+24
+00:01:28,200 --> 00:01:29,210
+So let's see.
+
+25
+00:01:29,400 --> 00:01:35,220
+Let's clear this one is to leave it and let's delete it and actually label some images here and let's
+
+26
+00:01:35,230 --> 00:01:38,050
+make sure we're all with it afterward.
+
+27
+00:01:38,280 --> 00:01:40,850
+Let's create a box here.
+
+28
+00:01:41,760 --> 00:01:42,050
+All right.
+
+29
+00:01:42,090 --> 00:01:44,960
+And let's call this London Underground.
+
+30
+00:01:45,060 --> 00:01:49,560
+And now once we do that it's going to be saved as our class here.
+
+31
+00:01:49,980 --> 00:01:50,700
+All right.
+
+32
+00:01:50,700 --> 00:01:51,880
+So that's pretty cool.
+
+33
+00:01:52,080 --> 00:01:56,740
+What I want to tell you is make sure the format is set on Pascrell VRC not Iolo.
+
+34
+00:01:56,970 --> 00:01:57,910
+And that's odd.
+
+35
+00:01:58,140 --> 00:02:02,470
+But you'll actually use to use your banalities as possible the A C.
+
+36
+00:02:02,580 --> 00:02:03,910
+So let's create another box.
+
+37
+00:02:03,910 --> 00:02:08,050
+Here is how we do a second box an image and press OK.
+
+38
+00:02:08,370 --> 00:02:10,950
+So now we have those two boxes here.
+
+39
+00:02:11,310 --> 00:02:14,050
+And all we have to do is just complete this.
+
+40
+00:02:14,050 --> 00:02:19,610
+See if it saves a TD or a tree here in Exham or format usually you want to save it.
+
+41
+00:02:19,680 --> 00:02:26,380
+You can do this manually all use a script to rename it properly but call it like 0 0 1 x amount.
+
+42
+00:02:27,120 --> 00:02:28,170
+And there we go.
+
+43
+00:02:28,470 --> 00:02:32,340
+So let's go back to our presentation.
+
+44
+00:02:32,340 --> 00:02:40,490
+So this is what we do so for and I've actually got 100 images of London underground tube sign in various
+
+45
+00:02:40,940 --> 00:02:42,390
+parts of London.
+
+46
+00:02:42,500 --> 00:02:45,540
+That's what you're going to have to do even to have to get images yourself.
+
+47
+00:02:45,550 --> 00:02:48,340
+It is true web scraper or image Skipworth sorry.
+
+48
+00:02:48,680 --> 00:02:53,000
+Or just manually using a google images and finding them on your own.
+
+49
+00:02:53,030 --> 00:02:55,640
+So there are some things I learned the hard way.
+
+50
+00:02:55,640 --> 00:03:02,300
+Make sure your images names are labeled like 001 much like some random digits like I did here because
+
+51
+00:03:02,330 --> 00:03:06,580
+you're going to have to mentally adjust them afterward so you can easily do a script to fix it.
+
+52
+00:03:06,590 --> 00:03:08,070
+But I didn't know at the time.
+
+53
+00:03:08,150 --> 00:03:11,090
+So I actually had to relabel my images twice.
+
+54
+00:03:11,120 --> 00:03:11,840
+It was a bit tedious.
+
+55
+00:03:11,840 --> 00:03:12,750
+It didn't have to do that.
+
+56
+00:03:12,770 --> 00:03:17,210
+But it was easier because it just it was a hundred images it was easier than doing a script to do it.
+
+57
+00:03:17,210 --> 00:03:18,840
+At that point in time.
+
+58
+00:03:19,130 --> 00:03:22,490
+And then we use the saved files in Pascal format.
+
+59
+00:03:22,550 --> 00:03:23,910
+Just remember that.
+
+60
+00:03:24,090 --> 00:03:30,570
+Because you went out in Jess's and just the dog axonal files so what else to know.
+
+61
+00:03:30,670 --> 00:03:31,160
+OK.
+
+62
+00:03:31,450 --> 00:03:37,210
+So we need to set up this file structure here and I'll go to a visual machine very shortly to show either
+
+63
+00:03:37,450 --> 00:03:38,770
+the file structure.
+
+64
+00:03:38,770 --> 00:03:41,500
+Actually I can do it to you now but let me discuss it first.
+
+65
+00:03:41,710 --> 00:03:42,060
+OK.
+
+66
+00:03:42,160 --> 00:03:48,600
+So this ensures I will tell you why this is important enough to will again if you don't do it this way.
+
+67
+00:03:48,790 --> 00:03:55,570
+But it will come in handy in the end mainly because when the actual files have the folder name that
+
+68
+00:03:55,630 --> 00:03:59,880
+it's in the image name and basically the pat to the file.
+
+69
+00:04:00,130 --> 00:04:08,740
+So I did this in Windows initially and what I had to do was to go into make a Python script to examine
+
+70
+00:04:08,740 --> 00:04:14,860
+this XML file and basically make all the changes for me manually or probably a blow the scripts if you
+
+71
+00:04:14,860 --> 00:04:21,150
+want afterward I'll clean it up a bit but it was not fun to do to correct these mistakes.
+
+72
+00:04:21,250 --> 00:04:24,010
+So let me go to the virtual machine now.
+
+73
+00:04:24,390 --> 00:04:25,100
+It's here.
+
+74
+00:04:26,440 --> 00:04:28,850
+And let's find this territory.
+
+75
+00:04:29,350 --> 00:04:33,120
+Actually for Call it actually I was looking at now.
+
+76
+00:04:33,340 --> 00:04:43,010
+So no I believe it was in top nets here.
+
+77
+00:04:44,480 --> 00:04:46,800
+Actually I mean distric dark flow.
+
+78
+00:04:47,060 --> 00:04:51,850
+So definitely darknet Darfur Doxil master Treen images annotations.
+
+79
+00:05:02,560 --> 00:05:06,770
+OK so ignore all of these files I have in the search area.
+
+80
+00:05:06,860 --> 00:05:09,840
+These were just a backup copy I made of them.
+
+81
+00:05:09,830 --> 00:05:12,800
+So annotations these are the files are important.
+
+82
+00:05:12,810 --> 00:05:21,080
+Now what's funny is that usually you train these doctors with hundreds of thousands of images.
+
+83
+00:05:21,290 --> 00:05:25,220
+I actually trained them only using 5 images.
+
+84
+00:05:25,220 --> 00:05:26,470
+Now I'll tell you why.
+
+85
+00:05:26,720 --> 00:05:28,150
+That's because I'm using a spear.
+
+86
+00:05:28,460 --> 00:05:34,120
+And every time I tried to go to maybe six seven images it would crash during training.
+
+87
+00:05:34,190 --> 00:05:38,320
+So I actually actually had to do a lot of trial and error to get the thing to work right.
+
+88
+00:05:38,330 --> 00:05:42,520
+So basically I spent a lot of time collecting images that I never used.
+
+89
+00:05:42,740 --> 00:05:48,030
+So it's a bit sad but that's OK at least we've learned from my mistake.
+
+90
+00:05:48,050 --> 00:05:50,320
+So let's take a look at this XML file.
+
+91
+00:05:50,420 --> 00:05:55,620
+So as you can see we have annotations annotations being the name of the folder that it's in.
+
+92
+00:05:55,760 --> 00:05:57,670
+We have the file name here.
+
+93
+00:05:57,950 --> 00:06:01,130
+This final game here corresponds to the GOP.
+
+94
+00:06:01,140 --> 00:06:02,260
+Finally I'm here.
+
+95
+00:06:02,750 --> 00:06:03,630
+Sorry.
+
+96
+00:06:03,670 --> 00:06:04,420
+Images.
+
+97
+00:06:04,670 --> 00:06:06,750
+Finally I'm here OK let's go back to it.
+
+98
+00:06:06,770 --> 00:06:08,630
+Get it she added.
+
+99
+00:06:09,140 --> 00:06:14,260
+And other thing to note is that it has a part of the image name here as well.
+
+100
+00:06:14,360 --> 00:06:18,120
+So you're actually the training gasifier training module.
+
+101
+00:06:18,140 --> 00:06:20,720
+Any low actually looks at these file names here.
+
+102
+00:06:20,810 --> 00:06:23,360
+If you have a mistake here it is not going to work.
+
+103
+00:06:23,490 --> 00:06:23,770
+OK.
+
+104
+00:06:23,840 --> 00:06:25,760
+So it's one thing to know.
+
+105
+00:06:25,760 --> 00:06:28,430
+So let's go back to this here.
+
+106
+00:06:28,550 --> 00:06:33,300
+So this was a directory I told you about just in case you were wondering.
+
+107
+00:06:33,320 --> 00:06:36,180
+So yeah I just mentioned this this needs to be corrected.
+
+108
+00:06:37,540 --> 00:06:40,410
+And now we need to go to see if you tertiary.
+
+109
+00:06:40,660 --> 00:06:47,200
+And we need to go to DCF to see if Sorry lower to find to see if you'll file and copy this file and
+
+110
+00:06:47,200 --> 00:06:48,090
+rename it.
+
+111
+00:06:48,130 --> 00:06:52,440
+So since we're using a one class you can call it whatever you want but we can decide whether you'll
+
+112
+00:06:52,550 --> 00:06:54,340
+under school one.
+
+113
+00:06:54,400 --> 00:06:56,490
+So let's see what that file looks like now.
+
+114
+00:07:01,080 --> 00:07:02,310
+Let's go here.
+
+115
+00:07:02,350 --> 00:07:03,770
+Good to see you directory.
+
+116
+00:07:03,780 --> 00:07:13,490
+And look for the file I made previously that would be you know where is it to search and disco.
+
+117
+00:07:13,500 --> 00:07:14,120
+There we go.
+
+118
+00:07:15,360 --> 00:07:19,230
+So let's open this file and we have some information here.
+
+119
+00:07:19,470 --> 00:07:21,440
+So no it looks fine.
+
+120
+00:07:21,450 --> 00:07:26,760
+However we do have to this is a file and we copied it from the original file.
+
+121
+00:07:26,860 --> 00:07:29,160
+The awesome changes we're going to have to make in this file.
+
+122
+00:07:29,250 --> 00:07:37,880
+And I'll tell you in addition so you see this red block I have highlighted here that is at the top of
+
+123
+00:07:37,880 --> 00:07:39,190
+the file here.
+
+124
+00:07:39,310 --> 00:07:40,810
+This bit here.
+
+125
+00:07:40,890 --> 00:07:49,180
+64 subdivisions with in height Let's go back to the presentation we need to make sure these are these
+
+126
+00:07:49,180 --> 00:07:49,860
+numbers here.
+
+127
+00:07:50,110 --> 00:07:54,400
+Now in the original file there are not a naturally or could these are commented out I believe.
+
+128
+00:07:54,400 --> 00:07:59,500
+So we just need to make sure that this looks like this and Jinyan the heightened Loree reducing the
+
+129
+00:07:59,500 --> 00:08:02,310
+height and weight makes it run much faster too.
+
+130
+00:08:02,350 --> 00:08:03,540
+So that's a good thing.
+
+131
+00:08:04,950 --> 00:08:10,540
+So that's added to configure the bottom part of the configuration file also needs to be edited.
+
+132
+00:08:10,560 --> 00:08:12,680
+So is two things we need to edits here.
+
+133
+00:08:13,050 --> 00:08:20,220
+One the convolutional last congressional league here it needs a number of number specific number filters
+
+134
+00:08:20,240 --> 00:08:25,800
+here and that's basically from this formula here and this formula depend on a number of classes you
+
+135
+00:08:25,800 --> 00:08:26,510
+use.
+
+136
+00:08:26,730 --> 00:08:33,400
+So since we use one class is just basically five into one plus six which is tity.
+
+137
+00:08:33,420 --> 00:08:39,430
+However if we use trig classes it will just be five plus times nine which is 45.
+
+138
+00:08:39,900 --> 00:08:42,730
+And the other thing we need to do is set the number of classes here.
+
+139
+00:08:43,020 --> 00:08:51,080
+So let's quickly take a look of that at the bottom just to make sure it's what I said it is good.
+
+140
+00:08:51,150 --> 00:08:53,710
+This is in the conference last convolutional league here.
+
+141
+00:08:53,970 --> 00:08:58,340
+And number filters are set to Tuti and classes is a 2:1.
+
+142
+00:08:58,440 --> 00:09:00,510
+So we're good to go so far.
+
+143
+00:09:01,050 --> 00:09:02,920
+Let's go back to that presentation.
+
+144
+00:09:03,000 --> 00:09:07,910
+And now we also need to create and edit or labels or label textfile.
+
+145
+00:09:08,160 --> 00:09:14,760
+So this one is fairly easy we just need to remember when we were labeling it in Lippe label image that
+
+146
+00:09:14,760 --> 00:09:19,360
+program and we had a object category named London underground.
+
+147
+00:09:19,380 --> 00:09:21,130
+We just need to list out there.
+
+148
+00:09:21,210 --> 00:09:26,610
+So if we had to reliables like cat dog Donald Trump would need to put each one on a new line in his
+
+149
+00:09:26,610 --> 00:09:27,350
+file.
+
+150
+00:09:27,400 --> 00:09:30,470
+So let's go to this directory and talk floor master.
+
+151
+00:09:31,230 --> 00:09:32,800
+And take a look at that.
+
+152
+00:09:35,150 --> 00:09:37,580
+Just to make sure it's done correctly.
+
+153
+00:09:37,660 --> 00:09:38,180
+There we go.
+
+154
+00:09:38,180 --> 00:09:42,310
+So you see this is London London and the ground is level there.
+
+155
+00:09:49,310 --> 00:09:49,660
+OK.
+
+156
+00:09:49,770 --> 00:09:50,810
+So let's keep going.
+
+157
+00:09:52,680 --> 00:10:00,650
+So now we have to do training so to do training we have to go until it's aminal and in Darfur dockmaster
+
+158
+00:10:01,190 --> 00:10:04,980
+Darfur mastered archery execute the following line.
+
+159
+00:10:05,040 --> 00:10:06,530
+That's this line here.
+
+160
+00:10:06,820 --> 00:10:07,470
+All right.
+
+161
+00:10:08,350 --> 00:10:12,960
+So essentially this is a line here we use if we're using a c.p.
+
+162
+00:10:13,090 --> 00:10:15,090
+So take note of this.
+
+163
+00:10:15,100 --> 00:10:16,620
+This has to be correct.
+
+164
+00:10:16,720 --> 00:10:19,900
+So we have to have all of their arteries identify correctly.
+
+165
+00:10:19,900 --> 00:10:24,110
+So it's CMAG and the school configuration file.
+
+166
+00:10:24,280 --> 00:10:30,920
+So you underscore one school class and we LoDo we it's we have previously.
+
+167
+00:10:30,970 --> 00:10:33,910
+So some people put the weights in a bin directory.
+
+168
+00:10:34,080 --> 00:10:42,270
+You just make sure it's lined up correctly and then we have transitions folder and tree and images will
+
+169
+00:10:42,540 --> 00:10:47,540
+be specified in the box if you're using a Jeep you can use this as well.
+
+170
+00:10:47,770 --> 00:10:52,320
+It tells you how much percent that the GPO memory you want to use.
+
+171
+00:10:52,320 --> 00:11:00,300
+So now after some time takes about an hour to always want to see the train of five bucks but that's
+
+172
+00:11:00,300 --> 00:11:05,370
+just what five images you can imagine how long it'll take to train if you have hundreds of thousands
+
+173
+00:11:05,370 --> 00:11:06,370
+of images.
+
+174
+00:11:06,600 --> 00:11:12,420
+So after we do that training is complete and then our model is saved and it's actually here checkpoint
+
+175
+00:11:12,720 --> 00:11:13,530
+file here.
+
+176
+00:11:13,860 --> 00:11:18,450
+Now when we load it back we just have to specify the checkpoint and it knows basically from the model
+
+177
+00:11:18,450 --> 00:11:20,790
+name what checkpoint to look for.
+
+178
+00:11:21,360 --> 00:11:23,600
+So this is how we load it into partem.
+
+179
+00:11:23,910 --> 00:11:28,610
+Basically the same thing we did before except we know specifying this model here.
+
+180
+00:11:28,890 --> 00:11:31,680
+And oh a checkpoint that's all.
+
+181
+00:11:32,370 --> 00:11:33,650
+And this is it.
+
+182
+00:11:33,690 --> 00:11:34,570
+This is what we just build.
+
+183
+00:11:34,570 --> 00:11:36,770
+This is where we spend all the time doing.
+
+184
+00:11:36,870 --> 00:11:42,170
+We built a London Underground classify some of these images on the training data sets of cheating here.
+
+185
+00:11:42,420 --> 00:11:47,860
+But a few of them are actually on the set and actually put them out quite well.
+
+186
+00:11:47,880 --> 00:11:53,160
+I was actually very impressed with how well this worked given that we only trained about five images
+
+187
+00:11:53,730 --> 00:11:58,600
+so let's actually go to division of machine and execute that training.
+
+188
+00:11:58,890 --> 00:12:01,000
+So let me just get the line here.
+
+189
+00:12:07,230 --> 00:12:13,620
+That was not lying it's just lying there reading these comments that are relevant.
+
+190
+00:12:13,620 --> 00:12:16,930
+So this is the line we want to use.
+
+191
+00:12:17,140 --> 00:12:20,270
+So we go to this territory here from two no.
+
+192
+00:12:20,590 --> 00:12:23,800
+So let's open up with your virtual machine.
+
+193
+00:12:23,800 --> 00:12:26,230
+So it's good to see the back.
+
+194
+00:12:26,280 --> 00:12:28,740
+So it's really dark.
+
+195
+00:12:29,080 --> 00:12:31,560
+It's not always mix them up.
+
+196
+00:12:31,570 --> 00:12:38,160
+Ellis the off flew again and then we just run this line up.
+
+197
+00:12:38,170 --> 00:12:39,350
+Something went wrong.
+
+198
+00:12:39,640 --> 00:12:42,330
+That is because we're not in the correct environment.
+
+199
+00:12:42,440 --> 00:12:48,300
+What I do with my environment service activities are the answers to API.
+
+200
+00:12:49,000 --> 00:12:54,220
+And I don't own a pistol but I want to piece it and have it run at the same time.
+
+201
+00:12:54,280 --> 00:13:01,530
+So I'm just going to make sure we do have that character courage and so let's copy it here.
+
+202
+00:13:02,660 --> 00:13:09,510
+Go back to it here and let's specify Let's see 25 ebox so you can actually watch a run.
+
+203
+00:13:10,040 --> 00:13:13,290
+And this should work fine.
+
+204
+00:13:13,340 --> 00:13:14,150
+Let's see it go.
+
+205
+00:13:18,590 --> 00:13:23,710
+I'm actually going to leave those five images for you so you can actually start training this class
+
+206
+00:13:23,800 --> 00:13:25,630
+this object on your own.
+
+207
+00:13:26,640 --> 00:13:27,860
+So you can have fun doing that.
+
+208
+00:13:30,370 --> 00:13:33,440
+So it takes a while to get started and you do see some warnings as well.
+
+209
+00:13:33,810 --> 00:13:39,060
+You see some statistics here saying that there are seven objects detected in images just at five.
+
+210
+00:13:39,240 --> 00:13:44,360
+That's because one or two of those images had more than one London Underground sign.
+
+211
+00:13:44,670 --> 00:13:46,640
+And you can see it's going to e.g. park.
+
+212
+00:13:46,860 --> 00:13:47,990
+It's actually going quite quickly.
+
+213
+00:13:48,010 --> 00:13:52,110
+So draining family parks isn't going to take that long.
+
+214
+00:13:52,380 --> 00:13:53,520
+Maybe just over an hour.
+
+215
+00:13:53,550 --> 00:13:56,370
+Actually And there we go.
+
+216
+00:13:56,370 --> 00:13:58,600
+It's going to get it's going to be finished soon.
+
+217
+00:14:00,810 --> 00:14:02,330
+In fact that's probably finished now.
+
+218
+00:14:03,780 --> 00:14:06,010
+A Wait how many e-books they specify.
+
+219
+00:14:06,160 --> 00:14:07,150
+Twenty five bucks.
+
+220
+00:14:07,150 --> 00:14:11,380
+Oh damn those five got so used to treating five bucks.
+
+221
+00:14:12,940 --> 00:14:14,520
+Anyway while that's running.
+
+222
+00:14:14,740 --> 00:14:18,030
+Let's actually use the model we created before.
+
+223
+00:14:18,250 --> 00:14:18,930
+OK.
+
+224
+00:14:19,270 --> 00:14:22,420
+So that would actually be in this file here.
+
+225
+00:14:22,420 --> 00:14:25,920
+No that's the tenths of one it was.
+
+226
+00:14:26,020 --> 00:14:27,310
+Yes this one.
+
+227
+00:14:27,340 --> 00:14:29,540
+So that's test only object detector.
+
+228
+00:14:29,950 --> 00:14:32,480
+So this is the object we're going to load here.
+
+229
+00:14:32,650 --> 00:14:33,270
+OK.
+
+230
+00:14:33,670 --> 00:14:36,800
+So I turn them all up to 400 ebox.
+
+231
+00:14:36,810 --> 00:14:43,030
+I actually did it once at 500 but then I added in some more images and it turned that 400 just tested.
+
+232
+00:14:43,210 --> 00:14:49,030
+This is the trouble we're going to use so let's This shouldn't take that long to load.
+
+233
+00:14:52,490 --> 00:14:55,910
+Almost done thanks for telling me it's funny.
+
+234
+00:14:55,920 --> 00:14:56,750
+Tell you
+
+235
+00:15:02,030 --> 00:15:02,480
+OK.
+
+236
+00:15:02,820 --> 00:15:03,100
+Good.
+
+237
+00:15:03,150 --> 00:15:04,540
+We've loaded our model.
+
+238
+00:15:04,760 --> 00:15:09,800
+Now it's actually cycled through some test images here from our test dataset.
+
+239
+00:15:11,520 --> 00:15:13,800
+Damn it's pretty good.
+
+240
+00:15:13,860 --> 00:15:14,580
+Good.
+
+241
+00:15:14,580 --> 00:15:15,690
+This one got it right.
+
+242
+00:15:15,690 --> 00:15:20,520
+But then if I do this as a one London underground tube same it clearly is not.
+
+243
+00:15:20,540 --> 00:15:21,870
+So let's see what else.
+
+244
+00:15:22,240 --> 00:15:28,200
+Again looking at maybe some curve here perhaps got this one right.
+
+245
+00:15:28,520 --> 00:15:29,620
+Got this one right.
+
+246
+00:15:29,630 --> 00:15:31,060
+This one right.
+
+247
+00:15:31,100 --> 00:15:35,090
+This one I got two boxes so we should have used them none maximal suppression.
+
+248
+00:15:35,110 --> 00:15:37,450
+I got him still didn't get any here.
+
+249
+00:15:37,490 --> 00:15:44,230
+Oddly got this one surprisingly And this one and this one too.
+
+250
+00:15:44,280 --> 00:15:50,700
+So as you can see our model actually performs fairly well given that we only used 5 images.
+
+251
+00:15:50,790 --> 00:15:56,730
+So I encourage you to experiment experiment with this make your own optic detect detector and maybe
+
+252
+00:15:56,730 --> 00:15:58,270
+show it to the rest of the students in the class.
+
+253
+00:15:58,290 --> 00:16:04,710
+Upload it to upload your model wait's to all of you Demy form.