SebasLopez-ai commited on
Commit
3e16037
·
0 Parent(s):

Initial commit

Browse files
This view is limited to 50 files because it contains too many changes.   See raw diff
Files changed (50) hide show
  1. .gitattributes +6 -0
  2. .gitignore +53 -0
  3. README.md +86 -0
  4. REPORT.md +253 -0
  5. app/app.py +218 -0
  6. app/static/style.css +397 -0
  7. app/templates/index.html +163 -0
  8. models/custom_cnn.keras +3 -0
  9. models/transfer_learning.keras +3 -0
  10. notebooks_knowledge&presentation/.In_cloud_transferlearning_gpu_options.md +49 -0
  11. notebooks_knowledge&presentation/EXTRA Transfer Learning II.ipynb +1049 -0
  12. notebooks_knowledge&presentation/Ppt.Slices/04. Custom CNN.png +3 -0
  13. notebooks_knowledge&presentation/Ppt.Slices/04.1 Cifar-10 Custom Cnn.png +3 -0
  14. notebooks_knowledge&presentation/Ppt.Slices/04.2 cnn_table.sumary().png +3 -0
  15. notebooks_knowledge&presentation/Ppt.Slices/05 Training Custom CNN.png +3 -0
  16. notebooks_knowledge&presentation/Ppt.Slices/cnn.sumary().png +3 -0
  17. notebooks_knowledge&presentation/alternative_models_reference.md +43 -0
  18. notebooks_knowledge&presentation/app_documentation_simple.md +46 -0
  19. notebooks_knowledge&presentation/fast_deployment_guide.md +65 -0
  20. notebooks_knowledge&presentation/jupyter notebooks/1. CIFAR10_Image_Classification_CNN.ipynb +0 -0
  21. notebooks_knowledge&presentation/jupyter notebooks/2. CIFAR10_Image_Classification_CNN_Presentation 09.52.53.ipynb +690 -0
  22. notebooks_knowledge&presentation/terminal_commands.md +76 -0
  23. outputs/augmentation_examples.png +3 -0
  24. outputs/class_distribution.png +3 -0
  25. outputs/custom_cnn_cm.png +3 -0
  26. outputs/custom_cnn_history.png +3 -0
  27. outputs/custom_cnn_metrics.json +8 -0
  28. outputs/misclassified_examples.png +3 -0
  29. outputs/model_comparison.png +3 -0
  30. outputs/sample_images.png +3 -0
  31. outputs/transfer_learning_cm.png +3 -0
  32. outputs/transfer_learning_history.png +3 -0
  33. outputs/transfer_learning_metrics.json +8 -0
  34. requirements.txt +10 -0
  35. src/__init__.py +1 -0
  36. src/data_loader.py +113 -0
  37. src/evaluate.py +156 -0
  38. src/model_builder.py +153 -0
  39. src/train.py +147 -0
  40. test_images/.gitkeep +0 -0
  41. test_images/test_CNN/airplane.png +3 -0
  42. test_images/test_CNN/automobile.png +3 -0
  43. test_images/test_CNN/bird.png +3 -0
  44. test_images/test_CNN/cat 444.png +3 -0
  45. test_images/test_CNN/cat.png +3 -0
  46. test_images/test_CNN/dog.png +3 -0
  47. test_images/test_CNN/frog.png +3 -0
  48. test_images/test_CNN/horse 21.png +3 -0
  49. test_images/test_CNN/horse.png +3 -0
  50. test_images/test_CNN/truck.png +3 -0
.gitattributes ADDED
@@ -0,0 +1,6 @@
 
 
 
 
 
 
 
1
+ *.keras filter=lfs diff=lfs merge=lfs -text
2
+ *.h5 filter=lfs diff=lfs merge=lfs -text
3
+ *.png filter=lfs diff=lfs merge=lfs -text
4
+ *.jpg filter=lfs diff=lfs merge=lfs -text
5
+ *.jpeg filter=lfs diff=lfs merge=lfs -text
6
+ *.pdf filter=lfs diff=lfs merge=lfs -text
.gitignore ADDED
@@ -0,0 +1,53 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Byte-compiled / optimized / DLL files
2
+ __pycache__/
3
+ *.py[cod]
4
+ *$py.class
5
+
6
+ # C extensions
7
+ *.so
8
+
9
+ # Distribution / packaging
10
+ .Python
11
+ build/
12
+ develop-eggs/
13
+ dist/
14
+ downloads/
15
+ eggs/
16
+ .eggs/
17
+ lib/
18
+ lib64/
19
+ parts/
20
+ sdist/
21
+ var/
22
+ wheels/
23
+ share/python-wheels/
24
+ *.egg-info/
25
+ .installed.cfg
26
+ *.egg
27
+ MANIFEST
28
+
29
+ # Virtual environments
30
+ venv/
31
+ env/
32
+ ENV/
33
+ env.bak/
34
+ venv.bak/
35
+
36
+ # Jupyter Notebook
37
+ .ipynb_checkpoints
38
+
39
+ # OS generated files
40
+ .DS_Store
41
+ .DS_Store?
42
+ ._*
43
+ .Spotlight-V100
44
+ .Trashes
45
+ ehthumbs.db
46
+ Thumbs.db
47
+
48
+ # VS Code
49
+ .vscode/
50
+
51
+ # Models and large files (depending on your preference, uncomment if you don't want to push models)
52
+ # models/*.h5
53
+ # models/*.keras
README.md ADDED
@@ -0,0 +1,86 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ![logo_ironhack_blue 7](https://user-images.githubusercontent.com/23629340/40541063-a07a0a8a-601a-11e8-91b5-2f13e4e6b441.png)
2
+
3
+ # Project I | Deep Learning: Image Classification with CNN
4
+
5
+ ## Task Description
6
+
7
+ Students will build a Convolutional Neural Network (CNN) model to classify images from a given dataset into predefined categories/classes.
8
+
9
+ ## Datasets (pick one!)
10
+
11
+ 1. The dataset for this task is the CIFAR-10 dataset, which consists of 60,000 32x32 color images in 10 classes, with 6,000 images per class. You can download the dataset from [here](https://www.cs.toronto.edu/~kriz/cifar.html).
12
+ 2. The second dataset contains about 28,000 medium quality animal images belonging to 10 categories: dog, cat, horse, spyder, butterfly, chicken, sheep, cow, squirrel, elephant. The link is [here](https://www.kaggle.com/datasets/alessiocorrado99/animals10/data).
13
+
14
+ ## Assessment Components
15
+
16
+ 1. **Data Preprocessing**
17
+ - Data loading and preprocessing (e.g., normalization, resizing, augmentation).
18
+ - Create visualizations of some images, and labels.
19
+
20
+ 2. **Model Architecture**
21
+ - Design a CNN architecture suitable for image classification.
22
+ - Include convolutional layers, pooling layers, and fully connected layers.
23
+
24
+ 3. **Model Training**
25
+ - Train the CNN model using appropriate optimization techniques (e.g., stochastic gradient descent, Adam).
26
+ - Utilize techniques such as early stopping to prevent overfitting.
27
+
28
+ 4. **Model Evaluation**
29
+ - Evaluate the trained model on a separate validation set.
30
+ - Compute and report metrics such as accuracy, precision, recall, and F1-score.
31
+ - Visualize the confusion matrix to understand model performance across different classes.
32
+
33
+ 5. **Transfer Learning**
34
+ - Evaluate the accuracy of your model on a pre-trained models like ImagNet, VGG16, Inception... (pick one an justify your choice)
35
+ - You may find this [link](https://www.tensorflow.org/tutorials/images/transfer_learning_with_hub) helpful.
36
+ - [This](https://pytorch.org/tutorials/beginner/transfer_learning_tutorial.html) is the Pytorch version.
37
+ - Perform transfer learning with your chosen pre-trained models i.e., you will probably try a few and choose the best one.
38
+
39
+ 5. **Code Quality**
40
+ - Well-structured and commented code.
41
+ - Proper documentation of functions and processes.
42
+ - Efficient use of libraries and resources.
43
+
44
+ 6. **Report**
45
+ - Write a concise report detailing the approach taken, including:
46
+ - Description of the chosen CNN architecture.
47
+ - Explanation of preprocessing steps.
48
+ - Details of the training process (e.g., learning rate, batch size, number of epochs).
49
+ - Results and analysis of models performance.
50
+ - What is your best model. Why?
51
+ - Insights gained from the experimentation process.
52
+ - Include visualizations and diagrams where necessary.
53
+
54
+ 7. **Model deployment**
55
+ - Pick the best model
56
+ - Build an app using Flask - Can you host somewhere other than your laptop? **+5 Bonus points if you use [Tensorflow Serving](https://www.tensorflow.org/tfx/guide/serving)**
57
+ - User should be able to upload one or multiples images get predictions including probabilities for each prediction
58
+
59
+
60
+ ## Evaluation Criteria
61
+
62
+ - Accuracy of the trained models on the validation set. **30 points**
63
+ - Clarity and completeness of the report. **20 points**
64
+ - Quality of code implementation. **5 points**
65
+ - Proper handling of data preprocessing and models training. **30 points**
66
+ - Demonstration of understanding key concepts of deep learning. **5 points**
67
+ - Model deployment. **10 points**
68
+
69
+ <span style="color:red; weight: bold">**Passing Score is 70 points**</span>.
70
+
71
+ ## Submission Details
72
+
73
+ - Deadline for submission: end of the week or as communicated by your teaching team.
74
+ - Submit the following:
75
+ 1. Python code files (`*.py`, `ipynb`) containing the model implementation and training process.
76
+ 2. A data folder with 5-10 images to test the deployed model/app if hosted somewhere else other than your laptop (strongly recommended! Not a must have)
77
+ 2. A PDF report documenting the approach, results, and analysis.
78
+ 3. Any additional files necessary for reproducing the results (e.g., requirements.txt, README.md).
79
+ 4. PPT presentation
80
+
81
+ ## Additional Notes
82
+
83
+ - Students are encourage to experiment with different architectures, hyper-parameters, and optimization techniques.
84
+ - Provide guidance and resources for troubleshooting common issues during model training and evaluation.
85
+ - Students will discuss their approaches and findings in class during assessment evaluation sessions.
86
+
REPORT.md ADDED
@@ -0,0 +1,253 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # 📊 Project Report: CIFAR-10 Image Classification with CNN
2
+
3
+ **Author:** Sebastian Lopez
4
+ **Date:** February 2026
5
+ **Environment:** Python 3.10 | TensorFlow 2.18.1 | Keras 3.6.0
6
+ **Dataset:** CIFAR-10 (60,000 images, 10 classes, 32×32 RGB)
7
+
8
+ ---
9
+
10
+ ## 1. Introduction
11
+
12
+ This project builds and evaluates two deep learning models for classifying images from the CIFAR-10 dataset into 10 categories: airplane, automobile, bird, cat, deer, dog, frog, horse, ship, and truck.
13
+
14
+ **Models developed:**
15
+ 1. **Custom CNN** — A purpose-built convolutional neural network
16
+ 2. **MobileNetV2 Transfer Learning** — Leveraging pretrained ImageNet features
17
+
18
+ ---
19
+
20
+ ## 2. Data Preprocessing
21
+
22
+ ### 2.1 Normalization
23
+ All pixel values were scaled from [0, 255] to [0.0, 1.0] by dividing by 255. This ensures consistent gradient magnitudes during backpropagation.
24
+
25
+ ### 2.2 Label Encoding
26
+ Integer labels (0–9) were one-hot encoded into 10-dimensional binary vectors using `keras.utils.to_categorical()`.
27
+
28
+ ### 2.3 Data Augmentation (Custom CNN only)
29
+ Applied real-time augmentation during training to reduce overfitting:
30
+
31
+ | Augmentation | Range |
32
+ |:-------------------|:------------|
33
+ | Rotation | ±15° |
34
+ | Width/Height shift | ±10% |
35
+ | Horizontal flip | Random |
36
+ | Zoom | ±10% |
37
+
38
+ ### 2.4 Dataset Split
39
+ - **Training:** 45,000 images (90% of train set)
40
+ - **Validation:** 5,000 images (10% of train set)
41
+ - **Test:** 10,000 images (held-out)
42
+
43
+ ---
44
+
45
+ ## 3. Model Architectures
46
+
47
+ ### 3.1 Custom CNN
48
+
49
+ A 3-block convolutional network designed specifically for 32×32 CIFAR-10 images:
50
+
51
+ ```
52
+ Block 1: Conv2D(32) × 2 → BatchNorm → MaxPool(2×2) → Dropout(0.25)
53
+ Block 2: Conv2D(64) × 2 → BatchNorm → MaxPool(2×2) → Dropout(0.25)
54
+ Block 3: Conv2D(128) × 2 → BatchNorm → MaxPool(2×2) → Dropout(0.25)
55
+ Head: Flatten → Dense(256) → BatchNorm → Dropout(0.5) → Dense(10, softmax)
56
+ ```
57
+
58
+ **Design rationale:**
59
+ - Progressive filter increase captures increasingly complex features
60
+ - BatchNormalization stabilizes training
61
+ - Dropout at every block provides strong regularization
62
+ - `padding='same'` preserves spatial dimensions within blocks
63
+
64
+ ### 3.2 MobileNetV2 Transfer Learning
65
+
66
+ ```
67
+ Input(32×32×3) → UpSampling2D(3×) → MobileNetV2(frozen, ImageNet weights)
68
+ → GlobalAveragePooling2D → Dense(256) → Dropout(0.5) → Dense(10, softmax)
69
+ ```
70
+
71
+ **Why MobileNetV2?**
72
+ - **Efficient**: ~3.4M parameters — lightweight enough for CPU training
73
+ - **Proven**: Excellent ImageNet accuracy despite small size
74
+ - **Practical**: Faster to train than VGG16 (~138M params) or ResNet50 (~25M params)
75
+ - **Upscaling**: 32×32 images are upscaled to 96×96 via UpSampling2D to meet MobileNetV2's minimum input requirements
76
+
77
+ **Fine-tuning strategy:**
78
+ 1. Phase 1: Train only the classification head (base frozen, lr=0.001)
79
+ 2. Phase 2: Unfreeze top 20 layers of MobileNetV2, retrain with lr=0.0001
80
+
81
+ ---
82
+
83
+ ## 4. Training Configuration
84
+
85
+ | Parameter | Custom CNN | Transfer Learning |
86
+ |:------------------|:-----------|:------------------|
87
+ | Optimizer | Adam | Adam |
88
+ | Initial LR | 0.001 | 0.001 → 0.0001 |
89
+ | Batch size | 64 | 64 |
90
+ | Max epochs | 100 | 50 + 30 |
91
+ | Early stopping | patience=10| patience=10/8 |
92
+ | Data augmentation | Yes | No (Phase 1) |
93
+ | LR reduction | ×0.5 on plateau | ×0.5 on plateau |
94
+
95
+ ---
96
+
97
+ ## 5. Results
98
+
99
+ ### 5.1 Metrics Summary
100
+
101
+ | Metric | Custom CNN | MobileNetV2 TL |
102
+ |:-----------|:-----------|:---------------|
103
+ | Accuracy | 0.8530 (85.30%) | 0.8340 (83.40%) |
104
+ | Precision | 0.8556 | 0.8353 |
105
+ | Recall | 0.8530 | 0.8340 |
106
+ | F1-Score | 0.8512 | 0.8339 |
107
+
108
+ > **Note:** Run both training scripts to populate these results. The JSON outputs in `outputs/` contain the exact numbers.
109
+
110
+ ### 5.2 Training Curves
111
+ - `outputs/custom_cnn_history.png` — Custom CNN accuracy and loss curves
112
+ - `outputs/transfer_learning_history.png` — Transfer learning curves
113
+
114
+ ### 5.3 Confusion Matrices
115
+ - `outputs/custom_cnn_cm.png` — Custom CNN confusion matrix
116
+ - `outputs/transfer_learning_cm.png` — Transfer learning confusion matrix
117
+ - `outputs/model_comparison.png` — Side-by-side metrics comparison
118
+
119
+ ### 5.4 Per-Class Confusion Analysis
120
+
121
+ Both confusion matrices reveal systematic error patterns driven by visual similarity at 32×32 resolution:
122
+
123
+ | Confusion Pair | Custom CNN Errors | MobileNetV2 Errors | Explanation |
124
+ |:---------------|:-----------------:|:-------------------:|:------------|
125
+ | **cat ↔ dog** | 198 | 256 | Both are furry, four-legged animals that share similar color palettes and body proportions at low resolution |
126
+ | **bird ↔ frog** | 87 | 32 | Small subjects against green/natural backgrounds; at 32×32, both reduce to small colored blobs |
127
+ | **automobile ↔ truck** | 82 | 107 | Both are wheeled vehicles with similar boxy shapes; the main discriminator (size) is lost at low resolution |
128
+ | **airplane ↔ ship** | 49 | 85 | Both frequently appear against blue backgrounds (sky vs. water), confusing color-based features |
129
+
130
+ **Key observation:** The cat class is the hardest to classify for both models (~65% accuracy), while frog, truck, and ship consistently achieve >90% accuracy. This suggests the model relies heavily on distinctive color patterns (green for frog, blue/gray for ship) rather than fine-grained shape features.
131
+
132
+ See `outputs/misclassified_examples.png` for a visual sample of misclassifications.
133
+
134
+ ---
135
+
136
+ ## 6. Best Model Selection
137
+
138
+ ### 🏆 Winner: Custom CNN (85.3% accuracy)
139
+
140
+ Lets see...
141
+
142
+ ## 7. Model Deployment
143
+
144
+ The best model is deployed via a **Flask web application**:
145
+
146
+ - **URL:** `http://localhost:5001`
147
+ - **Port note:** Port 5001 is used instead of Flask's default 5000 because macOS Monterey (12+) reserves port 5000 for the AirPlay Receiver service
148
+ - **Features:**
149
+ - Drag-and-drop or click-to-upload interface
150
+ - Supports single and multiple image uploads
151
+ - Displays top-10 predictions with probability bars
152
+ - API endpoint at `/api/predict` for programmatic access
153
+ - **Preprocessing:** Uploaded images are resized to 32×32, normalized to [0,1]
154
+
155
+ ---
156
+
157
+ ## 8. Key Insights
158
+
159
+ 1. **Data augmentation significantly reduces overfitting** for the custom CNN, allowing it to train longer before early stopping triggers
160
+ 2. **Transfer learning with frozen base layers** converges faster but doesn't always outperform custom architectures — especially when the source domain (ImageNet, 224×224) differs significantly from the target (CIFAR-10, 32×32)
161
+ 3. **Fine-tuning** the top layers of MobileNetV2 with a reduced learning rate provides an additional accuracy boost but cannot fully overcome the resolution mismatch
162
+ 4. **Confusion matrix analysis** reveals that visually similar classes (cat/dog, automobile/truck) are the most commonly confused pairs — see Section 5.4
163
+ 5. **Batch normalization** between convolutional layers stabilizes training and allows higher learning rates
164
+
165
+ ## 9. Bias & Limitations
166
+
167
+ ### 9.1 Dataset Representation Bias (Geographic/Cultural)
168
+
169
+ CIFAR-10 was collected from internet sources, predominantly Western/English-language websites. This introduces geographic bias in how each class is represented:
170
+
171
+ | Class | Bias Example |
172
+ |:------|:-------------|
173
+ | **Automobile** | Mostly American/European car designs — the model may struggle with tuk-tuks, rickshaws, or vehicle types common in Asia and Africa |
174
+ | **Truck** | Predominantly modern pickup and delivery trucks — would it recognize flatbed trucks from rural areas? |
175
+ | **Ship** | Mostly large vessels in open water — canoes, kayaks, or fishing boats from other cultures are underrepresented |
176
+ | **Horse** | Photographed primarily in Western contexts (ranches, paddocks) — horses in different cultural settings may confuse the model |
177
+ | **Bird** | Heavily weighted toward North American bird species — tropical or exotic birds are underrepresented |
178
+
179
+ ### 9.2 Why This Matters
180
+
181
+ 1. **No geographic diversity audit**: The model learned to classify "an automobile as seen by English-speaking internet users in North America" — not a universal definition
182
+ 2. **Background/context bias**: Objects were photographed in typical contexts (planes in blue sky, ships in water). An airplane on a tarmac or a ship in dry dock would likely be harder to classify
183
+ 3. **Color/lighting bias**: Most photos were taken in daylight. Night images or unusual lighting conditions may significantly degrade performance
184
+ 4. **Resolution bias**: All images are compressed to 32×32, which means the model relies heavily on **color patterns and rough shapes** rather than fine details — this is itself a form of information loss bias
185
+
186
+ ### 9.3 Other Limitations
187
+
188
+ - **Artificial class balance**: CIFAR-10 is perfectly balanced (6,000 per class), which is unrealistic — real-world data is almost never balanced
189
+ - **Open-set recognition**: The model has no concept of "none of the above" — uploading an image of a banana will still produce a confident prediction for one of the 10 classes
190
+ - **Temporal bias**: CIFAR-10 images are from a specific time period. Modern cars, ships, and trucks look different than those from when the dataset was compiled
191
+ - **No cross-validation**: Results are based on a single train/validation/test split, which could introduce variance
192
+
193
+ ---
194
+
195
+ ## 10. Folder Structure
196
+
197
+ ```
198
+ project-1-deep-learning-image-classification-with-cnn/
199
+ ├── README.md ← Project brief
200
+ ├── requirements.txt ← Dependencies (pinned versions)
201
+ ├── Dockerfile ← Docker container config
202
+ ├── .gitignore ← Git ignore rules
203
+ ├── Report/ ← Documentation & presentation
204
+ │ ├── REPORT.md ← This report
205
+ │ ├── CIFAR10_Image_Classification_CNN.ipynb ← Full notebook
206
+ │ └── CIFAR10_Image_Classification_CNN_Presentation.ipynb ← Slide deck
207
+ ├── src/ ← Source modules
208
+ │ ├── data_loader.py
209
+ │ ├── model_builder.py
210
+ │ ├── train.py
211
+ │ └── evaluate.py
212
+ ├── notebooks/ ← Executable pipeline scripts
213
+ │ ├── 01_data_exploration.py
214
+ │ ├── 02_custom_cnn.py
215
+ │ ├── 03_transfer_learning_cpu.py ← CPU-optimized (pre-resize with cv2)
216
+ │ ├── 03_transfer_learning_gpu.py ← GPU version (UpSampling2D in graph)
217
+ │ ├── 04_model_comparison.py ← Compare both models
218
+ │ ├── 05_deploy.py ← Launch Flask app (port 5001)
219
+ │ └── 06_misclassifications.py ← Generate misclassification examples
220
+ ├── app/ ← Flask web application
221
+ │ ├── app.py
222
+ │ ├── templates/index.html
223
+ │ └── static/style.css
224
+ ├── models/ ← Saved trained models (.keras)
225
+ ├── outputs/ ← Plots, metrics, reports
226
+ ├── test_images/ ← Sample images for testing the app
227
+ └── Other documentation/ ← Learning guides & extra docs
228
+ ```
229
+
230
+ ---
231
+
232
+ ## 10. How to Run
233
+
234
+ ```bash
235
+ # 1. Install dependencies
236
+ conda activate ironhack.nn
237
+
238
+ # 2. Explore the data
239
+ python notebooks/01_data_exploration.py
240
+
241
+ # 3. Train the custom CNN
242
+ python notebooks/02_custom_cnn.py
243
+
244
+ # 4. Train with transfer learning (CPU version — pre-resizes images with cv2)
245
+ python notebooks/03_transfer_learning_cpu.py
246
+
247
+ # 5. Compare both models
248
+ python notebooks/04_model_comparison.py
249
+
250
+ # 6. Launch the Flask app (port 5001 — macOS reserves 5000 for AirPlay)
251
+ python notebooks/05_deploy.py
252
+ # → Open http://localhost:5001
253
+ ```
app/app.py ADDED
@@ -0,0 +1,218 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ Flask Web Application for CIFAR-10 Image Classification.
3
+
4
+ This app allows users to upload one or multiple images and receive
5
+ predictions with class probabilities from the best trained model.
6
+
7
+ Usage:
8
+ conda run -n ironhack.nn python app/app.py
9
+
10
+ Then navigate to http://localhost:5001 in your browser.
11
+ (Port 5001 is used because macOS Monterey+ reserves 5000 for AirPlay.)
12
+ """
13
+
14
+ import os
15
+ import sys
16
+ import json
17
+ import numpy as np
18
+ from flask import Flask, request, render_template, jsonify
19
+ from PIL import Image
20
+ import io
21
+
22
+ # Add project root to path so we can import our modules
23
+ sys.path.insert(0, os.path.join(os.path.dirname(__file__), '..'))
24
+
25
+ from src.data_loader import CLASS_NAMES
26
+
27
+ # TensorFlow / Keras imports
28
+ from keras.api.models import load_model
29
+ from keras.api.utils import img_to_array
30
+
31
+ # ── Configuration ───────────────────────────────────────────────
32
+ PROJECT_ROOT = os.path.join(os.path.dirname(__file__), '..')
33
+
34
+ # Try to load the best model (transfer learning first, then custom CNN)
35
+ TRANSFER_MODEL_PATH = os.path.join(PROJECT_ROOT, 'models', 'transfer_learning.keras')
36
+ CUSTOM_MODEL_PATH = os.path.join(PROJECT_ROOT, 'models', 'custom_cnn.keras')
37
+
38
+ app = Flask(__name__)
39
+ app.config['MAX_CONTENT_LENGTH'] = 16 * 1024 * 1024 # 16 MB max upload
40
+
41
+ # Global models dictionary
42
+ models = {}
43
+ default_model_name = None
44
+
45
+
46
+ def load_all_models():
47
+ """Load both Custom CNN and MobileNetV2 models."""
48
+ global models, default_model_name
49
+
50
+ if os.path.exists(TRANSFER_MODEL_PATH):
51
+ models['transfer'] = {
52
+ 'model': load_model(TRANSFER_MODEL_PATH),
53
+ 'name': 'MobileNetV2 Transfer Learning'
54
+ }
55
+ print(f"✅ Loaded model: {models['transfer']['name']}")
56
+ default_model_name = 'transfer'
57
+
58
+ if os.path.exists(CUSTOM_MODEL_PATH):
59
+ models['custom'] = {
60
+ 'model': load_model(CUSTOM_MODEL_PATH),
61
+ 'name': 'Custom CNN'
62
+ }
63
+ print(f"✅ Loaded model: {models['custom']['name']}")
64
+ default_model_name = 'custom'
65
+
66
+ if not models:
67
+ print("⚠️ No trained models found! Please run training scripts first.")
68
+ default_model_name = None
69
+
70
+
71
+ def preprocess_image(image_bytes):
72
+ """
73
+ Preprocess a single uploaded image for prediction.
74
+
75
+ Steps:
76
+ 1. Open and convert to RGB
77
+ 2. Resize to 32x32 (CIFAR-10 input size)
78
+ 3. Normalize pixel values to [0, 1]
79
+ 4. Add batch dimension
80
+
81
+ Args:
82
+ image_bytes: Raw image bytes from upload.
83
+
84
+ Returns:
85
+ numpy array of shape (1, 32, 32, 3), float32, normalized.
86
+ """
87
+ img = Image.open(io.BytesIO(image_bytes)).convert('RGB')
88
+ img = img.resize((32, 32))
89
+ img_array = img_to_array(img) / 255.0
90
+ img_array = np.expand_dims(img_array, axis=0)
91
+ return img_array
92
+
93
+
94
+ def get_predictions(img_array, model_choice, top_n=5):
95
+ """
96
+ Get top-N class predictions with probabilities.
97
+
98
+ Args:
99
+ img_array: Preprocessed image array (1, 32, 32, 3).
100
+ model_choice: String key selecting which model to use ('custom' or 'transfer').
101
+ top_n: Number of top predictions to return.
102
+
103
+ Returns:
104
+ list of dicts: [{'class': str, 'probability': float}, ...]
105
+ """
106
+ if not models or model_choice not in models:
107
+ return [{'class': 'No model loaded', 'probability': 0.0}]
108
+
109
+ selected_model = models[model_choice]['model']
110
+ predictions = selected_model.predict(img_array, verbose=0)[0]
111
+ top_indices = predictions.argsort()[-top_n:][::-1]
112
+
113
+ results = []
114
+ for idx in top_indices:
115
+ results.append({
116
+ 'class': CLASS_NAMES[idx],
117
+ 'probability': float(predictions[idx])
118
+ })
119
+ return results
120
+
121
+
122
+ # ── Routes ──────────────────────────────────────────────────────
123
+
124
+ @app.route('/')
125
+ def index():
126
+ """Render the main upload page."""
127
+ current_model = models.get(default_model_name, {}).get('name', 'Loading...') if default_model_name else 'No model loaded'
128
+ return render_template('index.html', model_name=current_model)
129
+
130
+
131
+ @app.route('/predict', methods=['POST'])
132
+ def predict():
133
+ """
134
+ Handle image upload(s) and return predictions.
135
+
136
+ Accepts single or multiple files via the 'files' form field.
137
+ Returns JSON with predictions for each uploaded image.
138
+ """
139
+ if 'files' not in request.files:
140
+ return jsonify({'error': 'No files uploaded'}), 400
141
+
142
+ files = request.files.getlist('files')
143
+ if not files or all(f.filename == '' for f in files):
144
+ return jsonify({'error': 'No files selected'}), 400
145
+
146
+ model_choice = request.form.get('model_choice', default_model_name)
147
+ all_results = []
148
+
149
+ for file in files:
150
+ if file.filename == '':
151
+ continue
152
+
153
+ try:
154
+ image_bytes = file.read()
155
+ img_array = preprocess_image(image_bytes)
156
+ predictions = get_predictions(img_array, model_choice, top_n=10)
157
+
158
+ all_results.append({
159
+ 'filename': file.filename,
160
+ 'predictions': predictions
161
+ })
162
+ except Exception as e:
163
+ all_results.append({
164
+ 'filename': file.filename,
165
+ 'error': str(e)
166
+ })
167
+
168
+ current_model_name = models.get(model_choice, {}).get('name', 'Unknown Model')
169
+
170
+ return render_template('index.html',
171
+ model_name=current_model_name,
172
+ selected_model=model_choice,
173
+ results=all_results)
174
+
175
+
176
+ @app.route('/api/predict', methods=['POST'])
177
+ def api_predict():
178
+ """
179
+ API endpoint: returns JSON predictions (for programmatic access).
180
+ """
181
+ if 'files' not in request.files:
182
+ return jsonify({'error': 'No files uploaded'}), 400
183
+
184
+ files = request.files.getlist('files')
185
+ model_choice = request.form.get('model_choice', default_model_name)
186
+ current_model_name = models.get(model_choice, {}).get('name', 'Unknown Model')
187
+
188
+ all_results = []
189
+
190
+ for file in files:
191
+ if file.filename == '':
192
+ continue
193
+ try:
194
+ image_bytes = file.read()
195
+ img_array = preprocess_image(image_bytes)
196
+ predictions = get_predictions(img_array, model_choice, top_n=10)
197
+ all_results.append({
198
+ 'filename': file.filename,
199
+ 'predictions': predictions
200
+ })
201
+ except Exception as e:
202
+ all_results.append({
203
+ 'filename': file.filename,
204
+ 'error': str(e)
205
+ })
206
+
207
+ return jsonify({'model': current_model_name, 'results': all_results})
208
+
209
+
210
+ # ── Main ────────────────────────────────────────────────────────
211
+
212
+ if __name__ == '__main__':
213
+ load_all_models()
214
+ port = int(os.environ.get('PORT', 5001))
215
+ print(f"\n 🚀 Starting Flask app at http://localhost:{port}")
216
+ print(f" Available Models: {', '.join([m['name'] for m in models.values()])}")
217
+ print(f" Classes: {', '.join(CLASS_NAMES)}")
218
+ app.run(host='0.0.0.0', port=port, debug=True)
app/static/style.css ADDED
@@ -0,0 +1,397 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ /* ═══════════════════════════════════════════════════════════
2
+ CIFAR-10 Image Classifier — Flask App Styles
3
+ Modern, premium design with glassmorphism and gradients
4
+ ═══════════════════════════════════════════════════════════ */
5
+
6
+ :root {
7
+ --primary: #6C5CE7;
8
+ --primary-light: #A29BFE;
9
+ --secondary: #B388FF;
10
+ --accent: #CE93D8;
11
+ --grape: #7C4DFF;
12
+ --grape-light: #B47CFF;
13
+ --grape-glow: rgba(124, 77, 255, 0.25);
14
+ --bg-dark: #0c0c1d;
15
+ --bg-card: rgba(255, 255, 255, 0.05);
16
+ --bg-card-hover: rgba(255, 255, 255, 0.08);
17
+ --text: #E8E8F0;
18
+ --text-muted: #9B9BB0;
19
+ --border: rgba(255, 255, 255, 0.08);
20
+ --shadow: 0 8px 32px rgba(0, 0, 0, 0.3);
21
+ --radius: 16px;
22
+ --radius-sm: 8px;
23
+ --section-gap: 40px;
24
+ }
25
+
26
+ * {
27
+ margin: 0;
28
+ padding: 0;
29
+ box-sizing: border-box;
30
+ }
31
+
32
+ body {
33
+ font-family: 'Inter', -apple-system, BlinkMacSystemFont, sans-serif;
34
+ background: var(--bg-dark);
35
+ background-image:
36
+ radial-gradient(ellipse at 20% 50%, rgba(108, 92, 231, 0.15) 0%, transparent 50%),
37
+ radial-gradient(ellipse at 80% 20%, rgba(124, 77, 255, 0.10) 0%, transparent 50%),
38
+ radial-gradient(ellipse at 50% 80%, rgba(179, 136, 255, 0.08) 0%, transparent 50%);
39
+ color: var(--text);
40
+ min-height: 100vh;
41
+ line-height: 1.6;
42
+ }
43
+
44
+ .container {
45
+ max-width: 800px;
46
+ margin: 0 auto;
47
+ padding: 40px 20px;
48
+ }
49
+
50
+ /* ─── Header ──────────────────────────────────────────── */
51
+
52
+ .header {
53
+ text-align: center;
54
+ margin-bottom: var(--section-gap);
55
+ }
56
+
57
+ .logo {
58
+ display: flex;
59
+ align-items: center;
60
+ justify-content: center;
61
+ margin-bottom: var(--section-gap);
62
+ }
63
+
64
+ .header h1 {
65
+ font-size: 2rem;
66
+ font-weight: 700;
67
+ background: linear-gradient(135deg, var(--primary-light), var(--grape-light));
68
+ -webkit-background-clip: text;
69
+ -webkit-text-fill-color: transparent;
70
+ background-clip: text;
71
+ }
72
+
73
+ .subtitle {
74
+ color: var(--text-muted);
75
+ font-size: 0.95rem;
76
+ margin-bottom: 16px;
77
+ }
78
+
79
+ .model-badge {
80
+ display: inline-block;
81
+ background: linear-gradient(135deg, var(--primary), var(--grape));
82
+ color: white;
83
+ padding: 2px 12px;
84
+ border-radius: 20px;
85
+ font-size: 0.85rem;
86
+ font-weight: 600;
87
+ }
88
+
89
+ .class-tags {
90
+ display: flex;
91
+ flex-direction: column;
92
+ align-items: center;
93
+ gap: 10px;
94
+ margin-top: 0;
95
+ }
96
+
97
+ .tags-row {
98
+ display: flex;
99
+ justify-content: center;
100
+ gap: 8px;
101
+ flex-wrap: wrap;
102
+ }
103
+
104
+ .tag {
105
+ background: rgba(124, 77, 255, 0.08);
106
+ border: 1px solid rgba(124, 77, 255, 0.2);
107
+ padding: 4px 12px;
108
+ border-radius: 20px;
109
+ font-size: 0.8rem;
110
+ color: var(--primary-light);
111
+ transition: all 0.2s ease;
112
+ }
113
+
114
+ .tag:hover {
115
+ background: rgba(124, 77, 255, 0.15);
116
+ color: var(--text);
117
+ border-color: var(--grape);
118
+ transform: translateY(-1px);
119
+ box-shadow: 0 2px 12px var(--grape-glow);
120
+ }
121
+
122
+ /* ─── Model Selection ────────────────────────────────── */
123
+
124
+ .model-selection {
125
+ margin-bottom: var(--section-gap);
126
+ text-align: center;
127
+ }
128
+
129
+ .model-label {
130
+ font-weight: 500;
131
+ font-size: 13px;
132
+ color: var(--text-muted);
133
+ margin-right: 8px;
134
+ }
135
+
136
+ .model-select {
137
+ padding: 5px 10px;
138
+ border-radius: 6px;
139
+ border: 1px solid var(--grape);
140
+ background: rgba(124, 77, 255, 0.08);
141
+ backdrop-filter: blur(10px);
142
+ color: var(--text);
143
+ font-family: inherit;
144
+ font-size: 13px;
145
+ cursor: pointer;
146
+ outline: none;
147
+ transition: all 0.2s ease;
148
+ }
149
+
150
+ .model-select:hover {
151
+ border-color: var(--grape-light);
152
+ box-shadow: 0 0 0 2px var(--grape-glow);
153
+ }
154
+
155
+ .model-select:focus {
156
+ border-color: var(--grape-light);
157
+ box-shadow: 0 0 0 2px var(--grape-glow);
158
+ }
159
+
160
+ /* ─── Upload Section ──────────────────────────────────── */
161
+
162
+ .upload-section {
163
+ margin-bottom: var(--section-gap);
164
+ }
165
+
166
+ .dropzone {
167
+ background: var(--bg-card);
168
+ border: 2px dashed var(--border);
169
+ border-radius: var(--radius);
170
+ padding: 35px 20px;
171
+ text-align: center;
172
+ cursor: pointer;
173
+ transition: all 0.3s ease;
174
+ position: relative;
175
+ }
176
+
177
+ .dropzone:hover,
178
+ .dropzone.dragover {
179
+ border-color: var(--grape);
180
+ background: rgba(124, 77, 255, 0.08);
181
+ transform: translateY(-2px);
182
+ box-shadow: 0 8px 24px var(--grape-glow);
183
+ }
184
+
185
+ .dropzone-icon {
186
+ font-size: 3rem;
187
+ display: block;
188
+ margin-bottom: 12px;
189
+ }
190
+
191
+ .dropzone-text {
192
+ font-size: 1.1rem;
193
+ font-weight: 500;
194
+ color: var(--text);
195
+ }
196
+
197
+ .dropzone-subtext {
198
+ font-size: 0.85rem;
199
+ color: var(--text-muted);
200
+ margin-top: 4px;
201
+ }
202
+
203
+ .file-input {
204
+ position: absolute;
205
+ top: 0;
206
+ left: 0;
207
+ width: 100%;
208
+ height: 100%;
209
+ opacity: 0;
210
+ cursor: pointer;
211
+ }
212
+
213
+ .file-list {
214
+ margin-top: 12px;
215
+ display: flex;
216
+ flex-direction: column;
217
+ gap: 4px;
218
+ }
219
+
220
+ .file-item {
221
+ background: var(--bg-card);
222
+ padding: 8px 16px;
223
+ border-radius: var(--radius-sm);
224
+ font-size: 0.85rem;
225
+ color: var(--text-muted);
226
+ border: 1px solid var(--border);
227
+ }
228
+
229
+ .submit-btn {
230
+ display: block;
231
+ width: 100%;
232
+ margin-top: 20px;
233
+ padding: 16px;
234
+ font-size: 1.1rem;
235
+ font-weight: 600;
236
+ font-family: 'Inter', sans-serif;
237
+ color: white;
238
+ background: linear-gradient(135deg, var(--primary), var(--grape));
239
+ border: none;
240
+ border-radius: var(--radius-sm);
241
+ cursor: pointer;
242
+ transition: all 0.3s ease;
243
+ box-shadow: 0 4px 16px var(--grape-glow);
244
+ }
245
+
246
+ .submit-btn:hover:not(:disabled) {
247
+ transform: translateY(-2px);
248
+ box-shadow: 0 6px 24px rgba(124, 77, 255, 0.4);
249
+ }
250
+
251
+ .submit-btn:disabled {
252
+ opacity: 0.4;
253
+ cursor: not-allowed;
254
+ }
255
+
256
+ .btn-icon {
257
+ margin-right: 6px;
258
+ }
259
+
260
+ /* ─── Results Section ─────────────────────────────────── */
261
+
262
+ .results-section {
263
+ margin-bottom: var(--section-gap);
264
+ }
265
+
266
+ .results-title {
267
+ font-size: 1.4rem;
268
+ font-weight: 600;
269
+ margin-bottom: 24px;
270
+ text-align: center;
271
+ }
272
+
273
+ .result-card {
274
+ background: var(--bg-card);
275
+ border: 1px solid var(--border);
276
+ border-radius: var(--radius);
277
+ padding: 24px;
278
+ margin-bottom: 20px;
279
+ backdrop-filter: blur(10px);
280
+ box-shadow: var(--shadow);
281
+ transition: transform 0.2s ease;
282
+ }
283
+
284
+ .result-card:hover {
285
+ transform: translateY(-2px);
286
+ }
287
+
288
+ .result-header {
289
+ display: flex;
290
+ justify-content: space-between;
291
+ align-items: center;
292
+ margin-bottom: 16px;
293
+ padding-bottom: 12px;
294
+ border-bottom: 1px solid var(--border);
295
+ flex-wrap: wrap;
296
+ gap: 8px;
297
+ }
298
+
299
+ .filename {
300
+ font-weight: 500;
301
+ font-size: 0.95rem;
302
+ }
303
+
304
+ .top-prediction {
305
+ background: linear-gradient(135deg, rgba(124, 77, 255, 0.15), rgba(179, 136, 255, 0.05));
306
+ border: 1px solid rgba(124, 77, 255, 0.3);
307
+ color: var(--grape-light);
308
+ padding: 4px 14px;
309
+ border-radius: 20px;
310
+ font-size: 0.85rem;
311
+ }
312
+
313
+ .predictions {
314
+ display: flex;
315
+ flex-direction: column;
316
+ gap: 8px;
317
+ }
318
+
319
+ .prediction-row {
320
+ display: flex;
321
+ align-items: center;
322
+ gap: 12px;
323
+ }
324
+
325
+ .pred-class {
326
+ width: 100px;
327
+ font-size: 0.9rem;
328
+ font-weight: 500;
329
+ text-align: right;
330
+ color: var(--text-muted);
331
+ }
332
+
333
+ .prob-bar-container {
334
+ flex: 1;
335
+ height: 24px;
336
+ background: rgba(255, 255, 255, 0.04);
337
+ border-radius: 12px;
338
+ overflow: hidden;
339
+ position: relative;
340
+ }
341
+
342
+ .prob-bar {
343
+ height: 100%;
344
+ background: linear-gradient(90deg, var(--primary), var(--grape));
345
+ border-radius: 12px;
346
+ min-width: 2px;
347
+ transition: width 0.8s ease;
348
+ }
349
+
350
+ .pred-prob {
351
+ width: 60px;
352
+ font-size: 0.85rem;
353
+ font-weight: 600;
354
+ color: var(--primary-light);
355
+ text-align: right;
356
+ }
357
+
358
+ .error-msg {
359
+ color: #ef9a9a;
360
+ padding: 12px;
361
+ background: rgba(239, 154, 154, 0.1);
362
+ border-radius: var(--radius-sm);
363
+ border: 1px solid rgba(239, 154, 154, 0.2);
364
+ }
365
+
366
+ /* ─── Footer ──────────────────────────────────────────── */
367
+
368
+ .footer {
369
+ text-align: center;
370
+ padding-top: var(--section-gap);
371
+ margin-top: var(--section-gap);
372
+ border-top: 1px solid rgba(124, 77, 255, 0.15);
373
+ color: var(--text-muted);
374
+ font-size: 0.85rem;
375
+ }
376
+
377
+ /* ─── Responsive ──────────────────────────────────────── */
378
+
379
+ @media (max-width: 600px) {
380
+ .header h1 {
381
+ font-size: 1.5rem;
382
+ }
383
+
384
+ .result-header {
385
+ flex-direction: column;
386
+ align-items: flex-start;
387
+ }
388
+
389
+ .pred-class {
390
+ width: 80px;
391
+ font-size: 0.8rem;
392
+ }
393
+
394
+ .dropzone {
395
+ padding: 30px 16px;
396
+ }
397
+ }
app/templates/index.html ADDED
@@ -0,0 +1,163 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ <!DOCTYPE html>
2
+ <html lang="en">
3
+
4
+ <head>
5
+ <meta charset="UTF-8">
6
+ <meta name="viewport" content="width=device-width, initial-scale=1.0">
7
+ <meta name="description"
8
+ content="CIFAR-10 Image Classification — Upload images to classify them into 10 categories using a deep learning CNN model.">
9
+ <title>CIFAR-10 Image Classifier</title>
10
+ <link rel="stylesheet" href="{{ url_for('static', filename='style.css') }}">
11
+ <link href="https://fonts.googleapis.com/css2?family=Inter:wght@300;400;500;600;700&display=swap" rel="stylesheet">
12
+ </head>
13
+
14
+ <body>
15
+ <div class="container">
16
+ <!-- Header -->
17
+ <header class="header">
18
+ <div class="logo">
19
+ <h1>CIFAR-10 Image Classifier</h1>
20
+ </div>
21
+
22
+ <div class="class-tags">
23
+ <div class="tags-row tags-row-4">
24
+ <span class="tag">✈️ Airplane</span>
25
+ <span class="tag">🚗 Automobile</span>
26
+ <span class="tag">🚢 Ship</span>
27
+ <span class="tag">🚚 Truck</span>
28
+ </div>
29
+ <div class="tags-row tags-row-6">
30
+ <span class="tag">🐦 Bird</span>
31
+ <span class="tag">🐱 Cat</span>
32
+ <span class="tag">🦌 Deer</span>
33
+ <span class="tag">🐕 Dog</span>
34
+ <span class="tag">🐸 Frog</span>
35
+ <span class="tag">🐴 Horse</span>
36
+ </div>
37
+ </div>
38
+ </header>
39
+
40
+ <!-- Upload Form -->
41
+ <section class="upload-section">
42
+ <form action="/predict" method="POST" enctype="multipart/form-data" id="uploadForm">
43
+ <div class="model-selection">
44
+ <label for="modelChoice" class="model-label">🧠 Choose Model:</label>
45
+ <select name="model_choice" id="modelChoice" class="model-select">
46
+ <option value="custom" {% if selected_model=='custom' %}selected{% endif %}>CNN Model (Custom)
47
+ </option>
48
+ <option value="transfer" {% if selected_model=='transfer' %}selected{% endif %}>MobileNetV2
49
+ (Transfer Learning)</option>
50
+ </select>
51
+ </div>
52
+
53
+ <div class="dropzone" id="dropzone">
54
+ <div class="dropzone-content">
55
+ <span class="dropzone-icon">📸</span>
56
+ <p class="dropzone-text">Drag & drop images here</p>
57
+ <p class="dropzone-subtext">or click to browse — supports multiple files</p>
58
+ </div>
59
+ <input type="file" name="files" id="fileInput" multiple accept="image/*" class="file-input">
60
+ </div>
61
+
62
+ <div class="file-list" id="fileList"></div>
63
+ <button type="submit" class="submit-btn" id="submitBtn" disabled>
64
+ <span class="btn-icon">🔍</span> Classify Images
65
+ </button>
66
+ </form>
67
+ </section>
68
+
69
+ <!-- Results -->
70
+ {% if results %}
71
+ <section class="results-section">
72
+ <h2 class="results-title">🎯 Classification Results</h2>
73
+
74
+ {% for result in results %}
75
+ <div class="result-card">
76
+ <div class="result-header">
77
+ <span class="filename">📄 {{ result.filename }}</span>
78
+ {% if result.predictions %}
79
+ <span class="top-prediction">
80
+ Best: <strong>{{ result.predictions[0].class | capitalize }}</strong>
81
+ ({{ "%.1f" | format(result.predictions[0].probability * 100) }}%)
82
+ </span>
83
+ {% endif %}
84
+ </div>
85
+
86
+ {% if result.error %}
87
+ <div class="error-msg">⚠️ Error: {{ result.error }}</div>
88
+ {% else %}
89
+ <div class="predictions">
90
+ {% for pred in result.predictions %}
91
+ <div class="prediction-row">
92
+ <span class="pred-class">{{ pred.class | capitalize }}</span>
93
+ <div class="prob-bar-container">
94
+ <div class="prob-bar" style="width: {{ (pred.probability * 100) | round(1) }}%"></div>
95
+ </div>
96
+ <span class="pred-prob">{{ "%.1f" | format(pred.probability * 100) }}%</span>
97
+ </div>
98
+ {% endfor %}
99
+ </div>
100
+ {% endif %}
101
+ </div>
102
+ {% endfor %}
103
+ </section>
104
+ {% endif %}
105
+
106
+ <!-- Footer -->
107
+ <footer class="footer">
108
+ <p>Ironhack — Deep Learning Project | CNN Image Classification with CIFAR-10</p>
109
+ </footer>
110
+ </div>
111
+
112
+ <script>
113
+ // ─── Drag & Drop + File List ────────────────────
114
+ const dropzone = document.getElementById('dropzone');
115
+ const fileInput = document.getElementById('fileInput');
116
+ const fileList = document.getElementById('fileList');
117
+ const submitBtn = document.getElementById('submitBtn');
118
+
119
+ dropzone.addEventListener('click', () => fileInput.click());
120
+
121
+ dropzone.addEventListener('dragover', (e) => {
122
+ e.preventDefault();
123
+ dropzone.classList.add('dragover');
124
+ });
125
+
126
+ dropzone.addEventListener('dragleave', () => {
127
+ dropzone.classList.remove('dragover');
128
+ });
129
+
130
+ dropzone.addEventListener('drop', (e) => {
131
+ e.preventDefault();
132
+ dropzone.classList.remove('dragover');
133
+ fileInput.files = e.dataTransfer.files;
134
+ updateFileList();
135
+ });
136
+
137
+ fileInput.addEventListener('change', updateFileList);
138
+
139
+ function updateFileList() {
140
+ fileList.innerHTML = '';
141
+ const files = fileInput.files;
142
+ if (files.length > 0) {
143
+ submitBtn.disabled = false;
144
+ for (let f of files) {
145
+ const div = document.createElement('div');
146
+ div.className = 'file-item';
147
+ div.textContent = `📎 ${f.name} (${(f.size / 1024).toFixed(1)} KB)`;
148
+ fileList.appendChild(div);
149
+ }
150
+ } else {
151
+ submitBtn.disabled = true;
152
+ }
153
+ }
154
+
155
+ // ─── Submit loading state ────────────────────────
156
+ document.getElementById('uploadForm').addEventListener('submit', function () {
157
+ submitBtn.disabled = true;
158
+ submitBtn.innerHTML = '<span class="btn-icon">⏳</span> Classifying...';
159
+ });
160
+ </script>
161
+ </body>
162
+
163
+ </html>
models/custom_cnn.keras ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6da05a207a55bbd1ba43269fc7094bb3bb1aa3199c4448bca009a85b78b8a3ad
3
+ size 9902170
models/transfer_learning.keras ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5516031beafea75d838dc61097ee0bd08eef101f2e9c661495f512ab15cac7e5
3
+ size 19458191
notebooks_knowledge&presentation/.In_cloud_transferlearning_gpu_options.md ADDED
@@ -0,0 +1,49 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Transfer Learning: GPU Cloud Options
2
+
3
+ When performing Transfer Learning (like using MobileNetV2 or VGG16) on the CIFAR-10 dataset, training can be extremely slow on a local CPU (up to 20-25 hours).
4
+
5
+ To speed up the process to just minutes, you have several excellent free cloud-based GPU options, as well as a strategy to optimize your local CPU training.
6
+
7
+ ---
8
+
9
+ ## 🥇 Option 1: Google Colab (FREE, easiest, recommended)
10
+ Run your notebook on Google's free T4/V100 GPU in the cloud.
11
+
12
+ 1. Go to [colab.research.google.com](https://colab.research.google.com/)
13
+ 2. Upload your `.ipynb` notebook or paste your code.
14
+ 3. Set **Runtime → Change runtime type → T4 GPU**
15
+ 4. *Result:* What takes 20h on your CPU takes ~8 minutes on a Colab GPU ✅
16
+ 5. *Cost:* Free (with limits), or ~$10/month for Colab Pro if you need more compute.
17
+
18
+ ---
19
+
20
+ ## 🥈 Option 2: Kaggle Notebooks (FREE, 30h GPU/week)
21
+ Run notebooks directly on Kaggle with free GPU access.
22
+
23
+ 1. Go to [kaggle.com/code](https://www.kaggle.com/code)
24
+ 2. CIFAR-10 is even available as a built-in Kaggle dataset, making data loading instant.
25
+ 3. Very similar interface to Colab, with zero setup required.
26
+ 4. *Result:* Free T4 GPU, 30 hours/week allowance.
27
+ 5. *Cost:* Free.
28
+
29
+ ---
30
+
31
+ ## 🥉 Option 3: Fix the Script to Run Reasonably Fast on Your CPU
32
+ If you must run locally, you can optimize the CPU performance by resizing the data *before* training and reducing epochs.
33
+
34
+ | Configuration | Estimated Time on Your Local CPU |
35
+ |---|---|
36
+ | Current script (as-is) | 20–25 hours ❌ |
37
+ | After optimization (pre-resize + fewer epochs) | ~30–60 minutes ✅ |
38
+
39
+ *Note: The optimized CPU approach is included in the Presentation Notebook.*
40
+
41
+ ---
42
+
43
+ ## 🏅 Option 4: Lightning.ai (Free Cloud GPU)
44
+ Lightning Studios provides free GPU compute in a full notebook/IDE environment.
45
+
46
+ 1. Go to [lightning.ai](https://lightning.ai/)
47
+ 2. Spin up a free Studio.
48
+ 3. Offers a more generous free tier and advanced environment compared to standard Colab.
49
+ 4. *Cost:* Free tier available.
notebooks_knowledge&presentation/EXTRA Transfer Learning II.ipynb ADDED
@@ -0,0 +1,1049 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "cells": [
3
+ {
4
+ "cell_type": "markdown",
5
+ "metadata": {
6
+ "id": "hRTa3Ee15WsJ"
7
+ },
8
+ "source": [
9
+ "# Transfer learning and fine-tuning"
10
+ ]
11
+ },
12
+ {
13
+ "cell_type": "markdown",
14
+ "metadata": {
15
+ "id": "dQHMcypT3vDT"
16
+ },
17
+ "source": [
18
+ "<table class=\"tfo-notebook-buttons\" align=\"left\">\n",
19
+ " <td>\n",
20
+ " <a target=\"_blank\" href=\"https://www.tensorflow.org/tutorials/images/transfer_learning\"><img src=\"https://www.tensorflow.org/images/tf_logo_32px.png\" />View on TensorFlow.org</a>\n",
21
+ " </td>\n",
22
+ " <td>\n",
23
+ " <a target=\"_blank\" href=\"https://colab.research.google.com/github/tensorflow/docs/blob/master/site/en/tutorials/images/transfer_learning.ipynb?force_kitty_mode=1&force_corgi_mode=1\"><img src=\"https://www.tensorflow.org/images/colab_logo_32px.png\" />Run in Google Colab</a>\n",
24
+ " </td>\n",
25
+ " <td>\n",
26
+ " <a target=\"_blank\" href=\"https://github.com/tensorflow/docs/blob/master/site/en/tutorials/images/transfer_learning.ipynb\"><img src=\"https://www.tensorflow.org/images/GitHub-Mark-32px.png\" />View source on GitHub</a>\n",
27
+ " </td>\n",
28
+ " <td>\n",
29
+ " <a href=\"https://storage.googleapis.com/tensorflow_docs/docs/site/en/tutorials/images/transfer_learning.ipynb\"><img src=\"https://www.tensorflow.org/images/download_logo_32px.png\" />Download notebook</a>\n",
30
+ " </td>\n",
31
+ "</table>"
32
+ ]
33
+ },
34
+ {
35
+ "cell_type": "markdown",
36
+ "metadata": {
37
+ "id": "2X4KyhORdSeO"
38
+ },
39
+ "source": [
40
+ "In this tutorial, you will learn how to classify images of cats and dogs by using transfer learning from a pre-trained network.\n",
41
+ "\n",
42
+ "A pre-trained model is a saved network that was previously trained on a large dataset, typically on a large-scale image-classification task. You either use the pretrained model as is or use transfer learning to customize this model to a given task.\n",
43
+ "\n",
44
+ "The intuition behind transfer learning for image classification is that if a model is trained on a large and general enough dataset, this model will effectively serve as a generic model of the visual world. You can then take advantage of these learned feature maps without having to start from scratch by training a large model on a large dataset.\n",
45
+ "\n",
46
+ "In this notebook, you will try two ways to customize a pretrained model:\n",
47
+ "\n",
48
+ "1. Feature Extraction: Use the representations learned by a previous network to extract meaningful features from new samples. You simply add a new classifier, which will be trained from scratch, on top of the pretrained model so that you can repurpose the feature maps learned previously for the dataset.\n",
49
+ "\n",
50
+ " You do not need to (re)train the entire model. The base convolutional network already contains features that are generically useful for classifying pictures. However, the final, classification part of the pretrained model is specific to the original classification task, and subsequently specific to the set of classes on which the model was trained.\n",
51
+ "\n",
52
+ "1. Fine-Tuning: Unfreeze a few of the top layers of a frozen model base and jointly train both the newly-added classifier layers and the last layers of the base model. This allows us to \"fine-tune\" the higher-order feature representations in the base model in order to make them more relevant for the specific task.\n",
53
+ "\n",
54
+ "You will follow the general machine learning workflow.\n",
55
+ "\n",
56
+ "1. Examine and understand the data\n",
57
+ "1. Build an input pipeline, in this case using Keras ImageDataGenerator\n",
58
+ "1. Compose the model\n",
59
+ " * Load in the pretrained base model (and pretrained weights)\n",
60
+ " * Stack the classification layers on top\n",
61
+ "1. Train the model\n",
62
+ "1. Evaluate model\n"
63
+ ]
64
+ },
65
+ {
66
+ "cell_type": "code",
67
+ "execution_count": null,
68
+ "metadata": {
69
+ "id": "TqOt6Sv7AsMi"
70
+ },
71
+ "outputs": [],
72
+ "source": [
73
+ "import matplotlib.pyplot as plt\n",
74
+ "import numpy as np\n",
75
+ "import os\n",
76
+ "import tensorflow as tf"
77
+ ]
78
+ },
79
+ {
80
+ "cell_type": "markdown",
81
+ "metadata": {
82
+ "id": "v77rlkCKW0IJ"
83
+ },
84
+ "source": [
85
+ "## Data preprocessing"
86
+ ]
87
+ },
88
+ {
89
+ "cell_type": "markdown",
90
+ "metadata": {
91
+ "id": "0GoKGm1duzgk"
92
+ },
93
+ "source": [
94
+ "### Data download"
95
+ ]
96
+ },
97
+ {
98
+ "cell_type": "markdown",
99
+ "metadata": {
100
+ "id": "vHP9qMJxt2oz"
101
+ },
102
+ "source": [
103
+ "In this tutorial, you will use a dataset containing several thousand images of cats and dogs. Download and extract a zip file containing the images, then create a `tf.data.Dataset` for training and validation using the `tf.keras.utils.image_dataset_from_directory` utility. You can learn more about loading images in this [tutorial](https://www.tensorflow.org/tutorials/load_data/images)."
104
+ ]
105
+ },
106
+ {
107
+ "cell_type": "code",
108
+ "execution_count": null,
109
+ "metadata": {
110
+ "id": "ro4oYaEmxe4r"
111
+ },
112
+ "outputs": [],
113
+ "source": [
114
+ "_URL = 'https://storage.googleapis.com/mledu-datasets/cats_and_dogs_filtered.zip'\n",
115
+ "path_to_zip = tf.keras.utils.get_file('cats_and_dogs.zip', origin=_URL, extract=True)\n",
116
+ "PATH = os.path.join(os.path.dirname(path_to_zip), 'cats_and_dogs_filtered')\n",
117
+ "\n",
118
+ "train_dir = os.path.join(PATH, 'train')\n",
119
+ "validation_dir = os.path.join(PATH, 'validation')\n",
120
+ "\n",
121
+ "BATCH_SIZE = 32\n",
122
+ "IMG_SIZE = (160, 160)\n",
123
+ "\n",
124
+ "train_dataset = tf.keras.utils.image_dataset_from_directory(train_dir,\n",
125
+ " shuffle=True,\n",
126
+ " batch_size=BATCH_SIZE,\n",
127
+ " image_size=IMG_SIZE)"
128
+ ]
129
+ },
130
+ {
131
+ "cell_type": "code",
132
+ "execution_count": null,
133
+ "metadata": {
134
+ "id": "cAvtLwi7_J__"
135
+ },
136
+ "outputs": [],
137
+ "source": [
138
+ "validation_dataset = tf.keras.utils.image_dataset_from_directory(validation_dir,\n",
139
+ " shuffle=True,\n",
140
+ " batch_size=BATCH_SIZE,\n",
141
+ " image_size=IMG_SIZE)"
142
+ ]
143
+ },
144
+ {
145
+ "cell_type": "markdown",
146
+ "metadata": {
147
+ "id": "yO1Q2JaW5sIy"
148
+ },
149
+ "source": [
150
+ "Show the first nine images and labels from the training set:"
151
+ ]
152
+ },
153
+ {
154
+ "cell_type": "code",
155
+ "execution_count": null,
156
+ "metadata": {
157
+ "id": "K5BeQyKThC_Y"
158
+ },
159
+ "outputs": [],
160
+ "source": [
161
+ "class_names = train_dataset.class_names\n",
162
+ "\n",
163
+ "plt.figure(figsize=(10, 10))\n",
164
+ "for images, labels in train_dataset.take(1):\n",
165
+ " for i in range(9):\n",
166
+ " ax = plt.subplot(3, 3, i + 1)\n",
167
+ " plt.imshow(images[i].numpy().astype(\"uint8\"))\n",
168
+ " plt.title(class_names[labels[i]])\n",
169
+ " plt.axis(\"off\")"
170
+ ]
171
+ },
172
+ {
173
+ "cell_type": "markdown",
174
+ "metadata": {
175
+ "id": "EZqCX_mpV3Mx"
176
+ },
177
+ "source": [
178
+ "As the original dataset doesn't contain a test set, you will create one. To do so, determine how many batches of data are available in the validation set using `tf.data.experimental.cardinality`, then move 20% of them to a test set."
179
+ ]
180
+ },
181
+ {
182
+ "cell_type": "code",
183
+ "execution_count": null,
184
+ "metadata": {
185
+ "id": "uFFIYrTFV9RO"
186
+ },
187
+ "outputs": [],
188
+ "source": [
189
+ "val_batches = tf.data.experimental.cardinality(validation_dataset)\n",
190
+ "test_dataset = validation_dataset.take(val_batches // 5)\n",
191
+ "validation_dataset = validation_dataset.skip(val_batches // 5)"
192
+ ]
193
+ },
194
+ {
195
+ "cell_type": "code",
196
+ "execution_count": null,
197
+ "metadata": {
198
+ "id": "Q9pFlFWgBKgH"
199
+ },
200
+ "outputs": [],
201
+ "source": [
202
+ "print('Number of validation batches: %d' % tf.data.experimental.cardinality(validation_dataset))\n",
203
+ "print('Number of test batches: %d' % tf.data.experimental.cardinality(test_dataset))"
204
+ ]
205
+ },
206
+ {
207
+ "cell_type": "markdown",
208
+ "metadata": {
209
+ "id": "MakSrdd--RKg"
210
+ },
211
+ "source": [
212
+ "### Configure the dataset for performance"
213
+ ]
214
+ },
215
+ {
216
+ "cell_type": "markdown",
217
+ "metadata": {
218
+ "id": "22XWC7yjkZu4"
219
+ },
220
+ "source": [
221
+ "Use buffered prefetching to load images from disk without having I/O become blocking. To learn more about this method see the [data performance](https://www.tensorflow.org/guide/data_performance) guide."
222
+ ]
223
+ },
224
+ {
225
+ "cell_type": "code",
226
+ "execution_count": null,
227
+ "metadata": {
228
+ "id": "p3UUPdm86LNC"
229
+ },
230
+ "outputs": [],
231
+ "source": [
232
+ "AUTOTUNE = tf.data.AUTOTUNE\n",
233
+ "\n",
234
+ "train_dataset = train_dataset.prefetch(buffer_size=AUTOTUNE)\n",
235
+ "validation_dataset = validation_dataset.prefetch(buffer_size=AUTOTUNE)\n",
236
+ "test_dataset = test_dataset.prefetch(buffer_size=AUTOTUNE)"
237
+ ]
238
+ },
239
+ {
240
+ "cell_type": "markdown",
241
+ "metadata": {
242
+ "id": "MYfcVwYLiR98"
243
+ },
244
+ "source": [
245
+ "### Use data augmentation"
246
+ ]
247
+ },
248
+ {
249
+ "cell_type": "markdown",
250
+ "metadata": {
251
+ "id": "bDWc5Oad1daX"
252
+ },
253
+ "source": [
254
+ "When you don't have a large image dataset, it's a good practice to artificially introduce sample diversity by applying random, yet realistic, transformations to the training images, such as rotation and horizontal flipping. This helps expose the model to different aspects of the training data and reduce [overfitting](https://www.tensorflow.org/tutorials/keras/overfit_and_underfit). You can learn more about data augmentation in this [tutorial](https://www.tensorflow.org/tutorials/images/data_augmentation)."
255
+ ]
256
+ },
257
+ {
258
+ "cell_type": "code",
259
+ "execution_count": null,
260
+ "metadata": {
261
+ "id": "3P99QiMGit1A"
262
+ },
263
+ "outputs": [],
264
+ "source": [
265
+ "data_augmentation = tf.keras.Sequential([\n",
266
+ " tf.keras.layers.RandomFlip('horizontal'),\n",
267
+ " tf.keras.layers.RandomRotation(0.2),\n",
268
+ "])"
269
+ ]
270
+ },
271
+ {
272
+ "cell_type": "markdown",
273
+ "metadata": {
274
+ "id": "s9SlcbhrarOO"
275
+ },
276
+ "source": [
277
+ "Note: These layers are active only during training, when you call `Model.fit`. They are inactive when the model is used in inference mode in `Model.evaluate`, `Model.predict`, or `Model.call`."
278
+ ]
279
+ },
280
+ {
281
+ "cell_type": "markdown",
282
+ "metadata": {
283
+ "id": "9mD3rE2Lm7-d"
284
+ },
285
+ "source": [
286
+ "Let's repeatedly apply these layers to the same image and see the result."
287
+ ]
288
+ },
289
+ {
290
+ "cell_type": "code",
291
+ "execution_count": null,
292
+ "metadata": {
293
+ "id": "aQullOUHkm67"
294
+ },
295
+ "outputs": [],
296
+ "source": [
297
+ "for image, _ in train_dataset.take(1):\n",
298
+ " plt.figure(figsize=(10, 10))\n",
299
+ " first_image = image[0]\n",
300
+ " for i in range(9):\n",
301
+ " ax = plt.subplot(3, 3, i + 1)\n",
302
+ " augmented_image = data_augmentation(tf.expand_dims(first_image, 0))\n",
303
+ " plt.imshow(augmented_image[0] / 255)\n",
304
+ " plt.axis('off')"
305
+ ]
306
+ },
307
+ {
308
+ "cell_type": "markdown",
309
+ "metadata": {
310
+ "id": "bAywKtuVn8uK"
311
+ },
312
+ "source": [
313
+ "### Rescale pixel values\n",
314
+ "\n",
315
+ "In a moment, you will download `tf.keras.applications.MobileNetV2` for use as your base model. This model expects pixel values in `[-1, 1]`, but at this point, the pixel values in your images are in `[0, 255]`. To rescale them, use the preprocessing method included with the model."
316
+ ]
317
+ },
318
+ {
319
+ "cell_type": "code",
320
+ "execution_count": null,
321
+ "metadata": {
322
+ "id": "cO0HM9JAQUFq"
323
+ },
324
+ "outputs": [],
325
+ "source": [
326
+ "preprocess_input = tf.keras.applications.mobilenet_v2.preprocess_input"
327
+ ]
328
+ },
329
+ {
330
+ "cell_type": "markdown",
331
+ "metadata": {
332
+ "id": "xnr81qRMzcs5"
333
+ },
334
+ "source": [
335
+ "Note: Alternatively, you could rescale pixel values from `[0, 255]` to `[-1, 1]` using `tf.keras.layers.Rescaling`."
336
+ ]
337
+ },
338
+ {
339
+ "cell_type": "code",
340
+ "execution_count": null,
341
+ "metadata": {
342
+ "id": "R2NyJn4KQMux"
343
+ },
344
+ "outputs": [],
345
+ "source": [
346
+ "rescale = tf.keras.layers.Rescaling(1./127.5, offset=-1)"
347
+ ]
348
+ },
349
+ {
350
+ "cell_type": "markdown",
351
+ "metadata": {
352
+ "id": "Wz7qgImhTxw4"
353
+ },
354
+ "source": [
355
+ "Note: If using other `tf.keras.applications`, be sure to check the API doc to determine if they expect pixels in `[-1, 1]` or `[0, 1]`, or use the included `preprocess_input` function."
356
+ ]
357
+ },
358
+ {
359
+ "cell_type": "markdown",
360
+ "metadata": {
361
+ "id": "OkH-kazQecHB"
362
+ },
363
+ "source": [
364
+ "## Create the base model from the pre-trained convnets\n",
365
+ "You will create the base model from the **MobileNet V2** model developed at Google. This is pre-trained on the ImageNet dataset, a large dataset consisting of 1.4M images and 1000 classes. ImageNet is a research training dataset with a wide variety of categories like `jackfruit` and `syringe`. This base of knowledge will help us classify cats and dogs from our specific dataset.\n",
366
+ "\n",
367
+ "First, you need to pick which layer of MobileNet V2 you will use for feature extraction. The very last classification layer (on \"top\", as most diagrams of machine learning models go from bottom to top) is not very useful. Instead, you will follow the common practice to depend on the very last layer before the flatten operation. This layer is called the \"bottleneck layer\". The bottleneck layer features retain more generality as compared to the final/top layer.\n",
368
+ "\n",
369
+ "First, instantiate a MobileNet V2 model pre-loaded with weights trained on ImageNet. By specifying the **include_top=False** argument, you load a network that doesn't include the classification layers at the top, which is ideal for feature extraction."
370
+ ]
371
+ },
372
+ {
373
+ "cell_type": "code",
374
+ "execution_count": null,
375
+ "metadata": {
376
+ "id": "19IQ2gqneqmS"
377
+ },
378
+ "outputs": [],
379
+ "source": [
380
+ "# Create the base model from the pre-trained model MobileNet V2\n",
381
+ "IMG_SHAPE = IMG_SIZE + (3,)\n",
382
+ "base_model = tf.keras.applications.MobileNetV2(input_shape=IMG_SHAPE,\n",
383
+ " include_top=False,\n",
384
+ " weights='imagenet')"
385
+ ]
386
+ },
387
+ {
388
+ "cell_type": "markdown",
389
+ "metadata": {
390
+ "id": "AqcsxoJIEVXZ"
391
+ },
392
+ "source": [
393
+ "This feature extractor converts each `160x160x3` image into a `5x5x1280` block of features. Let's see what it does to an example batch of images:"
394
+ ]
395
+ },
396
+ {
397
+ "cell_type": "code",
398
+ "execution_count": null,
399
+ "metadata": {
400
+ "id": "Y-2LJL0EEUcx"
401
+ },
402
+ "outputs": [],
403
+ "source": [
404
+ "image_batch, label_batch = next(iter(train_dataset))\n",
405
+ "feature_batch = base_model(image_batch)\n",
406
+ "print(feature_batch.shape)"
407
+ ]
408
+ },
409
+ {
410
+ "cell_type": "markdown",
411
+ "metadata": {
412
+ "id": "rlx56nQtfe8Y"
413
+ },
414
+ "source": [
415
+ "## Feature extraction\n",
416
+ "In this step, you will freeze the convolutional base created from the previous step and to use as a feature extractor. Additionally, you add a classifier on top of it and train the top-level classifier."
417
+ ]
418
+ },
419
+ {
420
+ "cell_type": "markdown",
421
+ "metadata": {
422
+ "id": "CnMLieHBCwil"
423
+ },
424
+ "source": [
425
+ "### Freeze the convolutional base"
426
+ ]
427
+ },
428
+ {
429
+ "cell_type": "markdown",
430
+ "metadata": {
431
+ "id": "7fL6upiN3ekS"
432
+ },
433
+ "source": [
434
+ "It is important to freeze the convolutional base before you compile and train the model. Freezing (by setting layer.trainable = False) prevents the weights in a given layer from being updated during training. MobileNet V2 has many layers, so setting the entire model's `trainable` flag to False will freeze all of them."
435
+ ]
436
+ },
437
+ {
438
+ "cell_type": "code",
439
+ "execution_count": null,
440
+ "metadata": {
441
+ "id": "OTCJH4bphOeo"
442
+ },
443
+ "outputs": [],
444
+ "source": [
445
+ "base_model.trainable = False"
446
+ ]
447
+ },
448
+ {
449
+ "cell_type": "markdown",
450
+ "metadata": {
451
+ "id": "jsNHwpm7BeVM"
452
+ },
453
+ "source": [
454
+ "### Important note about BatchNormalization layers\n",
455
+ "\n",
456
+ "Many models contain `tf.keras.layers.BatchNormalization` layers. This layer is a special case and precautions should be taken in the context of fine-tuning, as shown later in this tutorial.\n",
457
+ "\n",
458
+ "When you set `layer.trainable = False`, the `BatchNormalization` layer will run in inference mode, and will not update its mean and variance statistics.\n",
459
+ "\n",
460
+ "When you unfreeze a model that contains BatchNormalization layers in order to do fine-tuning, you should keep the BatchNormalization layers in inference mode by passing `training = False` when calling the base model. Otherwise, the updates applied to the non-trainable weights will destroy what the model has learned.\n",
461
+ "\n",
462
+ "For more details, see the [Transfer learning guide](https://www.tensorflow.org/guide/keras/transfer_learning)."
463
+ ]
464
+ },
465
+ {
466
+ "cell_type": "code",
467
+ "execution_count": null,
468
+ "metadata": {
469
+ "id": "KpbzSmPkDa-N"
470
+ },
471
+ "outputs": [],
472
+ "source": [
473
+ "# Let's take a look at the base model architecture\n",
474
+ "base_model.summary()"
475
+ ]
476
+ },
477
+ {
478
+ "cell_type": "markdown",
479
+ "metadata": {
480
+ "id": "wdMRM8YModbk"
481
+ },
482
+ "source": [
483
+ "### Add a classification head"
484
+ ]
485
+ },
486
+ {
487
+ "cell_type": "markdown",
488
+ "metadata": {
489
+ "id": "QBc31c4tMOdH"
490
+ },
491
+ "source": [
492
+ "To generate predictions from the block of features, average over the spatial `5x5` spatial locations, using a `tf.keras.layers.GlobalAveragePooling2D` layer to convert the features to a single 1280-element vector per image."
493
+ ]
494
+ },
495
+ {
496
+ "cell_type": "code",
497
+ "execution_count": null,
498
+ "metadata": {
499
+ "id": "dLnpMF5KOALm"
500
+ },
501
+ "outputs": [],
502
+ "source": [
503
+ "global_average_layer = tf.keras.layers.GlobalAveragePooling2D()\n",
504
+ "feature_batch_average = global_average_layer(feature_batch)\n",
505
+ "print(feature_batch_average.shape)"
506
+ ]
507
+ },
508
+ {
509
+ "cell_type": "markdown",
510
+ "metadata": {
511
+ "id": "O1p0OJBR6dOT"
512
+ },
513
+ "source": [
514
+ "Apply a `tf.keras.layers.Dense` layer to convert these features into a single prediction per image. You don't need an activation function here because this prediction will be treated as a `logit`, or a raw prediction value. Positive numbers predict class 1, negative numbers predict class 0."
515
+ ]
516
+ },
517
+ {
518
+ "cell_type": "code",
519
+ "execution_count": null,
520
+ "metadata": {
521
+ "id": "Wv4afXKj6cVa"
522
+ },
523
+ "outputs": [],
524
+ "source": [
525
+ "prediction_layer = tf.keras.layers.Dense(1, activation='sigmoid')\n",
526
+ "prediction_batch = prediction_layer(feature_batch_average)\n",
527
+ "print(prediction_batch.shape)"
528
+ ]
529
+ },
530
+ {
531
+ "cell_type": "markdown",
532
+ "metadata": {
533
+ "id": "HXvz-ZkTa9b3"
534
+ },
535
+ "source": [
536
+ "Build a model by chaining together the data augmentation, rescaling, `base_model` and feature extractor layers using the [Keras Functional API](https://www.tensorflow.org/guide/keras/functional). As previously mentioned, use `training=False` as our model contains a `BatchNormalization` layer."
537
+ ]
538
+ },
539
+ {
540
+ "cell_type": "code",
541
+ "execution_count": null,
542
+ "metadata": {
543
+ "id": "DgzQX6Veb2WT"
544
+ },
545
+ "outputs": [],
546
+ "source": [
547
+ "inputs = tf.keras.Input(shape=(160, 160, 3))\n",
548
+ "x = data_augmentation(inputs)\n",
549
+ "x = preprocess_input(x)\n",
550
+ "x = base_model(x, training=False)\n",
551
+ "x = global_average_layer(x)\n",
552
+ "x = tf.keras.layers.Dropout(0.2)(x)\n",
553
+ "outputs = prediction_layer(x)\n",
554
+ "model = tf.keras.Model(inputs, outputs)"
555
+ ]
556
+ },
557
+ {
558
+ "cell_type": "code",
559
+ "execution_count": null,
560
+ "metadata": {
561
+ "id": "I8ARiyMFsgbH"
562
+ },
563
+ "outputs": [],
564
+ "source": [
565
+ "model.summary()"
566
+ ]
567
+ },
568
+ {
569
+ "cell_type": "markdown",
570
+ "metadata": {
571
+ "id": "lxOcmVr0ydFZ"
572
+ },
573
+ "source": [
574
+ "The 8+ million parameters in MobileNet are frozen, but there are 1.2 thousand _trainable_ parameters in the Dense layer. These are divided between two `tf.Variable` objects, the weights and biases."
575
+ ]
576
+ },
577
+ {
578
+ "cell_type": "code",
579
+ "execution_count": null,
580
+ "metadata": {
581
+ "id": "krvBumovycVA"
582
+ },
583
+ "outputs": [],
584
+ "source": [
585
+ "len(model.trainable_variables)"
586
+ ]
587
+ },
588
+ {
589
+ "cell_type": "code",
590
+ "execution_count": null,
591
+ "metadata": {
592
+ "id": "jeGk93R2ahav"
593
+ },
594
+ "outputs": [],
595
+ "source": [
596
+ "tf.keras.utils.plot_model(model, show_shapes=True)"
597
+ ]
598
+ },
599
+ {
600
+ "cell_type": "markdown",
601
+ "metadata": {
602
+ "id": "g0ylJXE_kRLi"
603
+ },
604
+ "source": [
605
+ "### Compile the model\n",
606
+ "\n",
607
+ "Compile the model before training it. Since there are two classes and a sigmoid oputput, use the `BinaryAccuracy`."
608
+ ]
609
+ },
610
+ {
611
+ "cell_type": "code",
612
+ "execution_count": null,
613
+ "metadata": {
614
+ "id": "RpR8HdyMhukJ"
615
+ },
616
+ "outputs": [],
617
+ "source": [
618
+ "base_learning_rate = 0.0001\n",
619
+ "model.compile(optimizer=tf.keras.optimizers.Adam(learning_rate=base_learning_rate),\n",
620
+ " loss=tf.keras.losses.BinaryCrossentropy(),\n",
621
+ " metrics=[tf.keras.metrics.BinaryAccuracy(threshold=0.5, name='accuracy')])"
622
+ ]
623
+ },
624
+ {
625
+ "cell_type": "markdown",
626
+ "metadata": {
627
+ "id": "RxvgOYTDSWTx"
628
+ },
629
+ "source": [
630
+ "### Train the model\n",
631
+ "\n",
632
+ "After training for 10 epochs, you should see ~96% accuracy on the validation set.\n"
633
+ ]
634
+ },
635
+ {
636
+ "cell_type": "code",
637
+ "execution_count": null,
638
+ "metadata": {
639
+ "id": "Om4O3EESkab1"
640
+ },
641
+ "outputs": [],
642
+ "source": [
643
+ "initial_epochs = 10\n",
644
+ "\n",
645
+ "loss0, accuracy0 = model.evaluate(validation_dataset)"
646
+ ]
647
+ },
648
+ {
649
+ "cell_type": "code",
650
+ "execution_count": null,
651
+ "metadata": {
652
+ "id": "8cYT1c48CuSd"
653
+ },
654
+ "outputs": [],
655
+ "source": [
656
+ "print(\"initial loss: {:.2f}\".format(loss0))\n",
657
+ "print(\"initial accuracy: {:.2f}\".format(accuracy0))"
658
+ ]
659
+ },
660
+ {
661
+ "cell_type": "code",
662
+ "execution_count": null,
663
+ "metadata": {
664
+ "id": "JsaRFlZ9B6WK"
665
+ },
666
+ "outputs": [],
667
+ "source": [
668
+ "history = model.fit(train_dataset,\n",
669
+ " epochs=initial_epochs,\n",
670
+ " validation_data=validation_dataset)"
671
+ ]
672
+ },
673
+ {
674
+ "cell_type": "markdown",
675
+ "metadata": {
676
+ "id": "Hd94CKImf8vi"
677
+ },
678
+ "source": [
679
+ "### Learning curves\n",
680
+ "\n",
681
+ "Let's take a look at the learning curves of the training and validation accuracy/loss when using the MobileNetV2 base model as a fixed feature extractor."
682
+ ]
683
+ },
684
+ {
685
+ "cell_type": "code",
686
+ "execution_count": null,
687
+ "metadata": {
688
+ "id": "53OTCh3jnbwV"
689
+ },
690
+ "outputs": [],
691
+ "source": [
692
+ "acc = history.history['accuracy']\n",
693
+ "val_acc = history.history['val_accuracy']\n",
694
+ "\n",
695
+ "loss = history.history['loss']\n",
696
+ "val_loss = history.history['val_loss']\n",
697
+ "\n",
698
+ "plt.figure(figsize=(8, 8))\n",
699
+ "plt.subplot(2, 1, 1)\n",
700
+ "plt.plot(acc, label='Training Accuracy')\n",
701
+ "plt.plot(val_acc, label='Validation Accuracy')\n",
702
+ "plt.legend(loc='lower right')\n",
703
+ "plt.ylabel('Accuracy')\n",
704
+ "plt.ylim([min(plt.ylim()),1])\n",
705
+ "plt.title('Training and Validation Accuracy')\n",
706
+ "\n",
707
+ "plt.subplot(2, 1, 2)\n",
708
+ "plt.plot(loss, label='Training Loss')\n",
709
+ "plt.plot(val_loss, label='Validation Loss')\n",
710
+ "plt.legend(loc='upper right')\n",
711
+ "plt.ylabel('Cross Entropy')\n",
712
+ "plt.ylim([0,1.0])\n",
713
+ "plt.title('Training and Validation Loss')\n",
714
+ "plt.xlabel('epoch')\n",
715
+ "plt.show()"
716
+ ]
717
+ },
718
+ {
719
+ "cell_type": "markdown",
720
+ "metadata": {
721
+ "id": "foWMyyUHbc1j"
722
+ },
723
+ "source": [
724
+ "Note: If you are wondering why the validation metrics are clearly better than the training metrics, the main factor is because layers like `tf.keras.layers.BatchNormalization` and `tf.keras.layers.Dropout` affect accuracy during training. They are turned off when calculating validation loss.\n",
725
+ "\n",
726
+ "To a lesser extent, it is also because training metrics report the average for an epoch, while validation metrics are evaluated after the epoch, so validation metrics see a model that has trained slightly longer."
727
+ ]
728
+ },
729
+ {
730
+ "cell_type": "markdown",
731
+ "metadata": {
732
+ "id": "CqwV-CRdS6Nv"
733
+ },
734
+ "source": [
735
+ "## Fine tuning\n",
736
+ "In the feature extraction experiment, you were only training a few layers on top of an MobileNetV2 base model. The weights of the pre-trained network were **not** updated during training.\n",
737
+ "\n",
738
+ "One way to increase performance even further is to train (or \"fine-tune\") the weights of the top layers of the pre-trained model alongside the training of the classifier you added. The training process will force the weights to be tuned from generic feature maps to features associated specifically with the dataset.\n",
739
+ "\n",
740
+ "Note: This should only be attempted after you have trained the top-level classifier with the pre-trained model set to non-trainable. If you add a randomly initialized classifier on top of a pre-trained model and attempt to train all layers jointly, the magnitude of the gradient updates will be too large (due to the random weights from the classifier) and your pre-trained model will forget what it has learned.\n",
741
+ "\n",
742
+ "Also, you should try to fine-tune a small number of top layers rather than the whole MobileNet model. In most convolutional networks, the higher up a layer is, the more specialized it is. The first few layers learn very simple and generic features that generalize to almost all types of images. As you go higher up, the features are increasingly more specific to the dataset on which the model was trained. The goal of fine-tuning is to adapt these specialized features to work with the new dataset, rather than overwrite the generic learning."
743
+ ]
744
+ },
745
+ {
746
+ "cell_type": "markdown",
747
+ "metadata": {
748
+ "id": "CPXnzUK0QonF"
749
+ },
750
+ "source": [
751
+ "### Un-freeze the top layers of the model\n"
752
+ ]
753
+ },
754
+ {
755
+ "cell_type": "markdown",
756
+ "metadata": {
757
+ "id": "rfxv_ifotQak"
758
+ },
759
+ "source": [
760
+ "All you need to do is unfreeze the `base_model` and set the bottom layers to be un-trainable. Then, you should recompile the model (necessary for these changes to take effect), and resume training."
761
+ ]
762
+ },
763
+ {
764
+ "cell_type": "code",
765
+ "execution_count": null,
766
+ "metadata": {
767
+ "id": "4nzcagVitLQm"
768
+ },
769
+ "outputs": [],
770
+ "source": [
771
+ "base_model.trainable = True"
772
+ ]
773
+ },
774
+ {
775
+ "cell_type": "code",
776
+ "execution_count": null,
777
+ "metadata": {
778
+ "id": "-4HgVAacRs5v"
779
+ },
780
+ "outputs": [],
781
+ "source": [
782
+ "# Let's take a look to see how many layers are in the base model\n",
783
+ "print(\"Number of layers in the base model: \", len(base_model.layers))\n",
784
+ "\n",
785
+ "# Fine-tune from this layer onwards\n",
786
+ "fine_tune_at = 100\n",
787
+ "\n",
788
+ "# Freeze all the layers before the `fine_tune_at` layer\n",
789
+ "for layer in base_model.layers[:fine_tune_at]:\n",
790
+ " layer.trainable = False"
791
+ ]
792
+ },
793
+ {
794
+ "cell_type": "markdown",
795
+ "metadata": {
796
+ "id": "4Uk1dgsxT0IS"
797
+ },
798
+ "source": [
799
+ "### Compile the model\n",
800
+ "\n",
801
+ "As you are training a much larger model and want to readapt the pretrained weights, it is important to use a lower learning rate at this stage. Otherwise, your model could overfit very quickly."
802
+ ]
803
+ },
804
+ {
805
+ "cell_type": "code",
806
+ "execution_count": null,
807
+ "metadata": {
808
+ "id": "NtUnaz0WUDva"
809
+ },
810
+ "outputs": [],
811
+ "source": [
812
+ "model.compile(loss=tf.keras.losses.BinaryCrossentropy(),\n",
813
+ " optimizer = tf.keras.optimizers.RMSprop(learning_rate=base_learning_rate/10),\n",
814
+ " metrics=[tf.keras.metrics.BinaryAccuracy(threshold=0.5, name='accuracy')])"
815
+ ]
816
+ },
817
+ {
818
+ "cell_type": "code",
819
+ "execution_count": null,
820
+ "metadata": {
821
+ "id": "WwBWy7J2kZvA"
822
+ },
823
+ "outputs": [],
824
+ "source": [
825
+ "model.summary()"
826
+ ]
827
+ },
828
+ {
829
+ "cell_type": "code",
830
+ "execution_count": null,
831
+ "metadata": {
832
+ "id": "bNXelbMQtonr"
833
+ },
834
+ "outputs": [],
835
+ "source": [
836
+ "len(model.trainable_variables)"
837
+ ]
838
+ },
839
+ {
840
+ "cell_type": "markdown",
841
+ "metadata": {
842
+ "id": "4G5O4jd6TuAG"
843
+ },
844
+ "source": [
845
+ "### Continue training the model"
846
+ ]
847
+ },
848
+ {
849
+ "cell_type": "markdown",
850
+ "metadata": {
851
+ "id": "0foWUN-yDLo_"
852
+ },
853
+ "source": [
854
+ "If you trained to convergence earlier, this step will improve your accuracy by a few percentage points."
855
+ ]
856
+ },
857
+ {
858
+ "cell_type": "code",
859
+ "execution_count": null,
860
+ "metadata": {
861
+ "id": "ECQLkAsFTlun"
862
+ },
863
+ "outputs": [],
864
+ "source": [
865
+ "fine_tune_epochs = 10\n",
866
+ "total_epochs = initial_epochs + fine_tune_epochs\n",
867
+ "\n",
868
+ "history_fine = model.fit(train_dataset,\n",
869
+ " epochs=total_epochs,\n",
870
+ " initial_epoch=len(history.epoch),\n",
871
+ " validation_data=validation_dataset)"
872
+ ]
873
+ },
874
+ {
875
+ "cell_type": "markdown",
876
+ "metadata": {
877
+ "id": "TfXEmsxQf6eP"
878
+ },
879
+ "source": [
880
+ "Let's take a look at the learning curves of the training and validation accuracy/loss when fine-tuning the last few layers of the MobileNetV2 base model and training the classifier on top of it. The validation loss is much higher than the training loss, so you may get some overfitting.\n",
881
+ "\n",
882
+ "You may also get some overfitting as the new training set is relatively small and similar to the original MobileNetV2 datasets.\n"
883
+ ]
884
+ },
885
+ {
886
+ "cell_type": "markdown",
887
+ "metadata": {
888
+ "id": "DNtfNZKlInGT"
889
+ },
890
+ "source": [
891
+ "After fine tuning the model nearly reaches 98% accuracy on the validation set."
892
+ ]
893
+ },
894
+ {
895
+ "cell_type": "code",
896
+ "execution_count": null,
897
+ "metadata": {
898
+ "id": "PpA8PlpQKygw"
899
+ },
900
+ "outputs": [],
901
+ "source": [
902
+ "acc += history_fine.history['accuracy']\n",
903
+ "val_acc += history_fine.history['val_accuracy']\n",
904
+ "\n",
905
+ "loss += history_fine.history['loss']\n",
906
+ "val_loss += history_fine.history['val_loss']"
907
+ ]
908
+ },
909
+ {
910
+ "cell_type": "code",
911
+ "execution_count": null,
912
+ "metadata": {
913
+ "id": "chW103JUItdk"
914
+ },
915
+ "outputs": [],
916
+ "source": [
917
+ "plt.figure(figsize=(8, 8))\n",
918
+ "plt.subplot(2, 1, 1)\n",
919
+ "plt.plot(acc, label='Training Accuracy')\n",
920
+ "plt.plot(val_acc, label='Validation Accuracy')\n",
921
+ "plt.ylim([0.8, 1])\n",
922
+ "plt.plot([initial_epochs-1,initial_epochs-1],\n",
923
+ " plt.ylim(), label='Start Fine Tuning')\n",
924
+ "plt.legend(loc='lower right')\n",
925
+ "plt.title('Training and Validation Accuracy')\n",
926
+ "\n",
927
+ "plt.subplot(2, 1, 2)\n",
928
+ "plt.plot(loss, label='Training Loss')\n",
929
+ "plt.plot(val_loss, label='Validation Loss')\n",
930
+ "plt.ylim([0, 1.0])\n",
931
+ "plt.plot([initial_epochs-1,initial_epochs-1],\n",
932
+ " plt.ylim(), label='Start Fine Tuning')\n",
933
+ "plt.legend(loc='upper right')\n",
934
+ "plt.title('Training and Validation Loss')\n",
935
+ "plt.xlabel('epoch')\n",
936
+ "plt.show()"
937
+ ]
938
+ },
939
+ {
940
+ "cell_type": "markdown",
941
+ "metadata": {
942
+ "id": "R6cWgjgfrsn5"
943
+ },
944
+ "source": [
945
+ "### Evaluation and prediction"
946
+ ]
947
+ },
948
+ {
949
+ "cell_type": "markdown",
950
+ "metadata": {
951
+ "id": "PSXH7PRMxOi5"
952
+ },
953
+ "source": [
954
+ "Finally you can verify the performance of the model on new data using test set."
955
+ ]
956
+ },
957
+ {
958
+ "cell_type": "code",
959
+ "execution_count": null,
960
+ "metadata": {
961
+ "id": "2KyNhagHwfar"
962
+ },
963
+ "outputs": [],
964
+ "source": [
965
+ "loss, accuracy = model.evaluate(test_dataset)\n",
966
+ "print('Test accuracy :', accuracy)"
967
+ ]
968
+ },
969
+ {
970
+ "cell_type": "markdown",
971
+ "metadata": {
972
+ "id": "8UjS5ukZfOcR"
973
+ },
974
+ "source": [
975
+ "And now you are all set to use this model to predict if your pet is a cat or dog."
976
+ ]
977
+ },
978
+ {
979
+ "cell_type": "code",
980
+ "execution_count": null,
981
+ "metadata": {
982
+ "id": "RUNoQNgtfNgt"
983
+ },
984
+ "outputs": [],
985
+ "source": [
986
+ "# Retrieve a batch of images from the test set\n",
987
+ "image_batch, label_batch = test_dataset.as_numpy_iterator().next()\n",
988
+ "predictions = model.predict_on_batch(image_batch).flatten()\n",
989
+ "\n",
990
+ "# Apply a sigmoid since our model returns logits\n",
991
+ "predictions = tf.nn.sigmoid(predictions)\n",
992
+ "predictions = tf.where(predictions < 0.5, 0, 1)\n",
993
+ "\n",
994
+ "print('Predictions:\\n', predictions.numpy())\n",
995
+ "print('Labels:\\n', label_batch)\n",
996
+ "\n",
997
+ "plt.figure(figsize=(10, 10))\n",
998
+ "for i in range(9):\n",
999
+ " ax = plt.subplot(3, 3, i + 1)\n",
1000
+ " plt.imshow(image_batch[i].astype(\"uint8\"))\n",
1001
+ " plt.title(class_names[predictions[i]])\n",
1002
+ " plt.axis(\"off\")"
1003
+ ]
1004
+ },
1005
+ {
1006
+ "cell_type": "markdown",
1007
+ "metadata": {
1008
+ "id": "_TZTwG7nhm0C"
1009
+ },
1010
+ "source": [
1011
+ "## Summary\n",
1012
+ "\n",
1013
+ "* **Using a pre-trained model for feature extraction**: When working with a small dataset, it is a common practice to take advantage of features learned by a model trained on a larger dataset in the same domain. This is done by instantiating the pre-trained model and adding a fully-connected classifier on top. The pre-trained model is \"frozen\" and only the weights of the classifier get updated during training.\n",
1014
+ "In this case, the convolutional base extracted all the features associated with each image and you just trained a classifier that determines the image class given that set of extracted features.\n",
1015
+ "\n",
1016
+ "* **Fine-tuning a pre-trained model**: To further improve performance, one might want to repurpose the top-level layers of the pre-trained models to the new dataset via fine-tuning.\n",
1017
+ "In this case, you tuned your weights such that your model learned high-level features specific to the dataset. This technique is usually recommended when the training dataset is large and very similar to the original dataset that the pre-trained model was trained on.\n",
1018
+ "\n",
1019
+ "To learn more, visit the [Transfer learning guide](https://www.tensorflow.org/guide/keras/transfer_learning).\n"
1020
+ ]
1021
+ }
1022
+ ],
1023
+ "metadata": {
1024
+ "accelerator": "GPU",
1025
+ "colab": {
1026
+ "name": "transfer_learning.ipynb",
1027
+ "toc_visible": true
1028
+ },
1029
+ "kernelspec": {
1030
+ "display_name": "Python 3 (ipykernel)",
1031
+ "language": "python",
1032
+ "name": "python3"
1033
+ },
1034
+ "language_info": {
1035
+ "codemirror_mode": {
1036
+ "name": "ipython",
1037
+ "version": 3
1038
+ },
1039
+ "file_extension": ".py",
1040
+ "mimetype": "text/x-python",
1041
+ "name": "python",
1042
+ "nbconvert_exporter": "python",
1043
+ "pygments_lexer": "ipython3",
1044
+ "version": "3.11.8"
1045
+ }
1046
+ },
1047
+ "nbformat": 4,
1048
+ "nbformat_minor": 4
1049
+ }
notebooks_knowledge&presentation/Ppt.Slices/04. Custom CNN.png ADDED

Git LFS Details

  • SHA256: 89092022fcbabf001878d21d75736c0f1dd690af2042dd442c82dd273363d8e6
  • Pointer size: 132 Bytes
  • Size of remote file: 7.01 MB
notebooks_knowledge&presentation/Ppt.Slices/04.1 Cifar-10 Custom Cnn.png ADDED

Git LFS Details

  • SHA256: ae2bc40c50786814fdc1e8ed9371df27f64390b47438de87c16fb49ec9b27e21
  • Pointer size: 132 Bytes
  • Size of remote file: 7.62 MB
notebooks_knowledge&presentation/Ppt.Slices/04.2 cnn_table.sumary().png ADDED

Git LFS Details

  • SHA256: 7dbdc970edc18bd4521be6af0382cf45bbe007eab7da792efbd07178053ad90a
  • Pointer size: 132 Bytes
  • Size of remote file: 8.42 MB
notebooks_knowledge&presentation/Ppt.Slices/05 Training Custom CNN.png ADDED

Git LFS Details

  • SHA256: 8335bb5affffb53e511a36deee51cdf852b7a84b6749d05043faa61cb8795582
  • Pointer size: 132 Bytes
  • Size of remote file: 6.68 MB
notebooks_knowledge&presentation/Ppt.Slices/cnn.sumary().png ADDED

Git LFS Details

  • SHA256: 19eb34bd53fbfd6c44259bd19ac86ccb85f711c74720ae97e492d6f5ad792d90
  • Pointer size: 131 Bytes
  • Size of remote file: 152 kB
notebooks_knowledge&presentation/alternative_models_reference.md ADDED
@@ -0,0 +1,43 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Alternative Transfer Learning Models Reference
2
+
3
+ This document outlines the various pre-trained models considered for the CIFAR-10 image classification project, their key characteristics, and the reasons why they were NOT chosen over MobileNetV2.
4
+
5
+ ---
6
+
7
+ ## Models Presented in the Presentation (Slides 4 & 4.1)
8
+
9
+ ### MobileNetV2 (Chosen Model - Slide 4)
10
+ * **Characteristics:** Remarkably efficient (~3.4M parameters). Uses inverted residual blocks.
11
+ * **Why it WAS chosen:** It strikes the perfect balance between high-accuracy feature extraction and computational efficiency. It trains quickly on standard hardware without requiring massive resources.
12
+
13
+ ### VGG16 (Slide 4.1)
14
+ * **Characteristics:** Massive size (~138M parameters). Classical deep CNN architecture with uniform simple convolutional layers.
15
+ * **Why Not?** It is very slow to train on standard hardware and highly prone to overfitting on small 32x32 images like CIFAR-10.
16
+
17
+ ### ResNet50 (Slide 4.1)
18
+ * **Characteristics:** Powerful architecture (~25M parameters) that utilizes skip connections (residual blocks) to train very deep networks without vanishing gradients.
19
+ * **Why Not?** Its complexity is often overkill for CIFAR-10, leading to unnecessarily long training times without proportional accuracy gains for such small images.
20
+
21
+ ### InceptionV3 (Slide 4.1)
22
+ * **Characteristics:** Uses "Inception modules" capable of looking at the same image with different receptive fields (filter sizes) simultaneously. (~24M parameters)
23
+ * **Why Not?** Demands high computational resources and typically requires much larger input resolutions (default is 299x299) to be fully effective. Highly upscaling 32x32 to 299x299 is very resource-intensive.
24
+
25
+ ---
26
+
27
+ ## Other Notable Alternatives
28
+
29
+ ### EfficientNet (e.g., EfficientNetB0)
30
+ * **Characteristics:** Modern, highly optimized models that scale up the network's depth, width, and resolution evenly. (B0 has ~5.3M parameters)
31
+ * **Why Not?** While a very valid alternative, MobileNetV2 is slightly older but exceptionally well-documented for beginners, and tends to train slightly faster on basic setups.
32
+
33
+ ### DenseNet (e.g., DenseNet121)
34
+ * **Characteristics:** Connects each layer to every other layer in a feed-forward fashion, creating strong feature reuse. (~8M parameters)
35
+ * **Why Not?** They are heavier in memory usage and take much longer to train per epoch compared to MobileNetV2, despite often achieving higher accuracy.
36
+
37
+ ### Xception
38
+ * **Characteristics:** An extension of the Inception architecture that uses depthwise separable convolutions (similar to MobileNet). (~22M parameters)
39
+ * **Why Not?** It is quite large and heavily optimized for the massive ImageNet dataset. Using it for 32x32 pixel images is often inefficient.
40
+
41
+ ### NASNetMobile
42
+ * **Characteristics:** An architecture discovered by an AI (Neural Architecture Search) designed specifically for mobile and resource-constrained devices.
43
+ * **Why Not?** While an excellent alternative, MobileNet design (inverted residual blocks) is much simpler and more intuitive to explain in a presentation and learning environment.
notebooks_knowledge&presentation/app_documentation_simple.md ADDED
@@ -0,0 +1,46 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # How Our Magic Picture Guesser Works! 🪄🖼️
2
+
3
+ Hello! Welcome to our Magic Picture Guesser! This is a simple story about how your computer learns to look at a picture and tell you what it is.
4
+
5
+ ## Step 1: The Magic Brain (The Model) 🧠
6
+
7
+ Before the computer can guess pictures, it has to go to school!
8
+
9
+ We have a special file called a **Model**. Think of the Model as a big, smart brain that has looked at thousands of pictures of dogs, cats, airplanes, and cars.
10
+
11
+ Because it studied so hard, it remembers what they look like! We keep this brain in a safe place inside our project.
12
+
13
+ ## Step 2: The Doorway (The App) 🚪
14
+
15
+ To let you talk to the brain, we built a Doorway. In computer words, we built an **App** using a tool called Flask.
16
+
17
+ When you start the App, it opens a special website just for you on your computer. It looks super cool with colors and a big box where you can drop pictures.
18
+
19
+ ## Step 3: Dropping a Picture 📸
20
+
21
+ When you open the website, you can drag a picture of a car or a dog into the big box.
22
+
23
+ But wait! The magic brain is very picky. It only likes looking at pictures that are small square shapes (like a tiny 32x32 puzzle piece).
24
+
25
+ ## Step 4: The Shrinking Machine 🔬
26
+
27
+ Before the brain looks at it, our App uses a **Shrinking Machine** to resize your picture so it fits perfectly in the brain's tiny window. If your picture was huge, we just squish it down so the brain can read it.
28
+
29
+ ## Step 5: The Magic Guess ✨
30
+
31
+ Once the picture is small enough, the App hands it to the Magic Brain.
32
+
33
+ The Brain looks at the colors and shapes and says:
34
+ **"Aha! I am 99% sure this is an Automobile (a car)!"**
35
+
36
+ And then, the App shows you the answer on the screen!
37
+
38
+ ---
39
+
40
+ **Summary:**
41
+ 1. You drag a picture.
42
+ 2. We shrink it.
43
+ 3. The brain looks at it.
44
+ 4. It tells you what it sees!
45
+
46
+ It's just like showing a picture-book to a very smart friend!
notebooks_knowledge&presentation/fast_deployment_guide.md ADDED
@@ -0,0 +1,65 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Fast and Cheap Deployment Guide (Render.com)
2
+
3
+ This guide explains how to deploy your Flask application for free using [Render.com](https://render.com/), which is currently one of the easiest and most cost-effective ways to host a Python web app.
4
+
5
+ Later, you can move this to a VPS (Virtual Private Server) like DigitalOcean or AWS EC2, but Render is the perfect starting point to quickly fulfill the "+5 Bonus points" requirement.
6
+
7
+ ## Prerequisites
8
+
9
+ 1. Your project must be pushed to a **GitHub repository**.
10
+ 2. You need an account on [Render.com](https://render.com/) (you can sign up with GitHub).
11
+
12
+ ## Step-by-Step Instructions
13
+
14
+ ### Step 1: Update `requirements.txt`
15
+ Render needs a production web server to run your Flask app. We will use `gunicorn`.
16
+ Open your `requirements.txt` file and add this line at the very bottom:
17
+ ```text
18
+ gunicorn==21.2.0
19
+ ```
20
+
21
+ ### Step 2: Push to GitHub
22
+ Make sure all your latest changes, particularly the updated `requirements.txt` and your downloaded best model (in the `models/` folder), are committed and pushed to your GitHub repository.
23
+
24
+ ```bash
25
+ git add requirements.txt models/best_model.h5
26
+ git commit -m "Prepare for Render deployment"
27
+ git push origin main
28
+ ```
29
+ *(Note: If your model file is larger than 100MB, you might need to use Git LFS or upload it differently, but MobileNetV2 should be small enough).*
30
+
31
+ ### Step 3: Create a Web Service on Render
32
+ 1. Log into your Render dashboard.
33
+ 2. Click on **New +** and select **Web Service**.
34
+ 3. Connect your GitHub account and select your project repository.
35
+
36
+ ### Step 4: Configure the Web Service
37
+ Fill out the deployment form with the following details:
38
+ - **Name:** Choose a name for your app (e.g., `cifar10-classifier-sebastian`).
39
+ - **Region:** Choose the region closest to you (e.g., Frankfurt/EU).
40
+ - **Branch:** `main` (or whichever branch your code is on).
41
+ - **Runtime:** `Python 3`.
42
+ - **Build Command:**
43
+ ```bash
44
+ pip install -r requirements.txt
45
+ ```
46
+ - **Start Command:**
47
+ ```bash
48
+ gunicorn app.app:app
49
+ ```
50
+ *(Explanation: The first `app` is your `app` folder, the second `app` is the `app.py` script, and the third `:app` is the Flask instance named `app` inside that script).*
51
+
52
+ ### Step 5: Choose Instance Type & Deploy
53
+ - Select the **Free** instance type ($0/month).
54
+ - Click **Create Web Service**.
55
+
56
+ ### Step 6: Wait for Build
57
+ Render will now install your dependencies and launch your app. The console output will show you the progress. Once it says "Live", your app is running!
58
+
59
+ You can access your live app using the URL Render provides at the top of the dashboard (e.g., `https://cifar10-classifier-sebastian.onrender.com`).
60
+
61
+ ---
62
+ **Why this is the best first step:**
63
+ - It is 100% free.
64
+ - It automatically redeploys if you push new code to GitHub.
65
+ - It handles SSL certificates (`https://`) out of the box.
notebooks_knowledge&presentation/jupyter notebooks/1. CIFAR10_Image_Classification_CNN.ipynb ADDED
The diff for this file is too large to render. See raw diff
 
notebooks_knowledge&presentation/jupyter notebooks/2. CIFAR10_Image_Classification_CNN_Presentation 09.52.53.ipynb ADDED
@@ -0,0 +1,690 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "cells": [
3
+ {
4
+ "cell_type": "markdown",
5
+ "metadata": {},
6
+ "source": [
7
+ "# Guided Lab: Image Classification using CNN (CIFAR-10)\n",
8
+ "\n",
9
+ "At the end of this tutorial, you will get familiarized with:\n",
10
+ "\n",
11
+ "* Creating deep networks using Keras\n",
12
+ "* Steps necessary in training a neural network\n",
13
+ "* Prediction and performance analysis using neural networks\n",
14
+ "* Using Transfer Learning for complex images"
15
+ ]
16
+ },
17
+ {
18
+ "cell_type": "markdown",
19
+ "metadata": {},
20
+ "source": [
21
+ "# **Environment set up**\n",
22
+ "**Keras** is a Python library that provides, in a simple way, the creation of a wide range of Deep Learning models using as backend other libraries such as TensorFlow. \n",
23
+ "\n",
24
+ "Keras is used since its learning curve is very smooth compared to others, and at the moment it is one of the popular middleware to implement neural networks."
25
+ ]
26
+ },
27
+ {
28
+ "cell_type": "code",
29
+ "execution_count": 1,
30
+ "metadata": {},
31
+ "outputs": [
32
+ {
33
+ "name": "stderr",
34
+ "output_type": "stream",
35
+ "text": [
36
+ "2026-02-28 16:43:51.698653: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.\n",
37
+ "To enable the following instructions: SSE4.1 SSE4.2 AVX AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.\n"
38
+ ]
39
+ },
40
+ {
41
+ "name": "stdout",
42
+ "output_type": "stream",
43
+ "text": [
44
+ "3.6.0\n"
45
+ ]
46
+ }
47
+ ],
48
+ "source": [
49
+ "from tensorflow import keras\n",
50
+ "import tensorflow as tf\n",
51
+ "print(keras.__version__)"
52
+ ]
53
+ },
54
+ {
55
+ "cell_type": "markdown",
56
+ "metadata": {},
57
+ "source": [
58
+ "# **Understanding Data: CIFAR-10 Dataset**\n",
59
+ "The CIFAR-10 dataset consists of 60,000 32x32 color images in 10 classes, with 6,000 images per class. \n",
60
+ "\n",
61
+ "In Keras, the CIFAR-10 dataset is preloaded in the form of four Numpy arrays. `x_train` and `y_train` contain the training set, while `x_test` and `y_test` contain the test data. The images are encoded as Numpy arrays and their corresponding labels ranging from 0 to 9."
62
+ ]
63
+ },
64
+ {
65
+ "cell_type": "code",
66
+ "execution_count": 3,
67
+ "metadata": {},
68
+ "outputs": [
69
+ {
70
+ "name": "stdout",
71
+ "output_type": "stream",
72
+ "text": [
73
+ "Training data shape: (50000, 32, 32, 3)\n",
74
+ "Testing data shape: (10000, 32, 32, 3)\n"
75
+ ]
76
+ }
77
+ ],
78
+ "source": [
79
+ "from keras.datasets import cifar10\n",
80
+ "(x_train, y_train), (x_test, y_test) = cifar10.load_data()\n",
81
+ "\n",
82
+ "print('Training data shape:', x_train.shape)\n",
83
+ "print('Testing data shape:', x_test.shape)"
84
+ ]
85
+ },
86
+ {
87
+ "cell_type": "markdown",
88
+ "metadata": {},
89
+ "source": [
90
+ "## Visualizing the Data\n",
91
+ "\n",
92
+ "A plot of the first nine images in the dataset is created showing the natural image nature of the images to be classified.\n",
93
+ "\n",
94
+ "**Let us create a 3*3 subplot to visualize the first 9 images of the dataset.**"
95
+ ]
96
+ },
97
+ {
98
+ "cell_type": "code",
99
+ "execution_count": 5,
100
+ "metadata": {},
101
+ "outputs": [
102
+ {
103
+ "data": {
104
+ "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZYAAAGpCAYAAACu1BDuAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjAsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvlHJYcgAAAAlwSFlzAAAPYQAAD2EBqD+naQAAuedJREFUeJzs/XecZFd5Jo6/N1Wu6uocJ2tmNJJmlLNAEkKAQSIbfiRJtlmvbbzfjzG7i7ENiF2MjY13YVnMBi+wGDDBlrQC2RgUEAhJoJylmdHkzqm6ctVNvz+6+r6hNaNuVD0jjd9Hn/no3H5vPOfce+o873ue1wjDMASFQqFQKNoE80TfgEKhUChOLujAolAoFIq2QgcWhUKhULQVOrAoFAqFoq3QgUWhUCgUbYUOLAqFQqFoK3RgUSgUCkVboQOLQqFQKNoKHVgUCoVC0VacdAPLd77zHTj99NMhmUyCYRjw6KOPnuhbUpwg3HvvvXDjjTdCoVA4Idf/2te+BoZhwIMPPnhCrq94cWgfWRucVAPL9PQ0fOADH4AtW7bAD3/4Q7jvvvtg27ZtJ/q2FCcI9957L3zqU586YR8Nxcsf2kfWBvaJvoF2Yvfu3eC6Lrz//e+Hyy+//Kj7VatVSKVSx/HOFC931Go1SCaTJ/o2FC9jaB9ZOU6aGcsNN9wAl112GQAAvPvd7wbDMOCKK66AG264ATKZDDzxxBPwute9DrLZLFx11VUAADA3Nwe/93u/B8PDwxCLxWDz5s3wJ3/yJ9BoNNi5C4UC/NZv/RZ0dXVBJpOBN73pTbBv3z4wDANuvPHG4/2oihXgxhtvhP/wH/4DAABs2rQJDMMAwzDgJz/5CWzcuBGuueYauOmmm+Dss8+GRCIBn/rUp+DAgQNgGAZ87WtfW3a+F2rrZ599Ft7znvdAf38/xONxWL9+PVx33XXL+g/F+Pg4nHvuubB161bYs2dPOx9ZsUpoH1k7nDQzlo9//ONwwQUXwIc+9CH4zGc+A1deeSXkcjn4y7/8S2g2m/DmN78Z/u2//bfwR3/0R+B5HtTrdbjyyivh+eefh0996lOwa9cu+NnPfgZ//ud/Do8++ijcdtttAAAQBAFce+218OCDD8KNN94I55xzDtx3333whje84QQ/seJY+OAHPwhzc3PwxS9+EW666SYYHBwEAIDTTjsNAAAefvhheOaZZ+BP//RPYdOmTZBOp1d1/sceewwuu+wy6Onpgf/0n/4TbN26FcbHx+HWW2+FZrMJ8Xh82TFPPvkkvPGNb4SRkRG47777oKen56U/qOJXhvaRtcNJM7Bs2bIl6hBbt26Fiy66KLK5rguf+MQn4Dd+4zeiv/3P//k/4fHHH4fvfve78Ou//usAAHD11VdDJpOBj370o/DjH/8Yrr76avjhD38I99xzD3z5y1+G3/md34n2i8Vi8LGPfew4PqFiNRgZGYH169cDAMDZZ58NGzduZPapqSl4+umnmQ/uwIEDKz7/H/7hH4Jt2/DLX/4Sent7o7+/733ve8H9b7/9dnjHO94Br3vd6+Dv/u7vIJFIrPxhFGsC7SNrh5OGCnsxvOMd72Dbd955J6TTaXjnO9/J/n7DDTcAAMAdd9wBAAB33303AAC8613vYvu95z3vWaM7VRwP7Nq161cO7KhWq3D33XfDu971LvbBOBr+7//9v/DGN74RPvjBD8J3v/vdV/QH418TtI/86vhXMbCkUinI5XLsb7OzszAwMACGYbC/9/X1gW3bMDs7G+1n2zZ0dXWx/fr7+9f2phVriiXa41fB/Pw8+L4PIyMjK9r/29/+NiSTSfjgBz+4rL8pXr7QPvKr41/FwPJCDdXd3Q2Tk5MgE2hOTU2B53kRt9nd3Q2e58Hc3Bzbb2JiYu1uWLHmeKE+sfQrUTpWl35kLKGrqwssy4IjR46s6Frf/OY34dRTT4XLL79c11W9gqB95FfHv4qB5YVw1VVXQblchltuuYX9/etf/3pkB4AobPk73/kO2+/b3/722t+k4iVhyTlaq9VWtH9/fz8kEgl4/PHH2d//3//7f2w7mUzC5ZdfDt/73vdgZmbmRc/b1dUFt99+O+zYsQOuvPJKuP/++1f4BIq1hvaRtcFJ47xfLa677jr40pe+BNdffz0cOHAAdu7cCffccw985jOfgTe+8Y3w2te+FgAA3vCGN8Cll14KH/nIR6BYLMK5554L9913XzQAmea/2rH5ZY+dO3cCAMAXvvAFuP7668FxHNi+fftR9zcMA97//vfDV77yFdiyZQuceeaZ8Mtf/hK+9a1vLdv3v/yX/wKXXXYZXHjhhfBHf/RHcMopp8Dk5CTceuut8D//5/+EbDbL9s9ms/DDH/4Q3v72t8PVV18Nt956K1x55ZXtfWDFqqF9ZI0QnkS46667QgAIv/e970V/u/7668N0Ov2C+8/Ozoa/8zu/Ew4ODoa2bYcbNmwIP/axj4X1ep3tNzc3F/7Gb/xGmM/nw1QqFV599dXh/fffHwJA+IUvfGFNn0nx0vCxj30sHBoaCk3TDAEgvOuuu8INGzaEb3rTm15w/4WFhfCDH/xg2N/fH6bT6fDaa68NDxw4EAJA+MlPfpLt+/TTT4e//uu/HnZ3d4exWCxcv359eMMNN0T956tf/WoIAOEDDzwQHdNoNMJ3vOMdYSKRCG+77bY1e27FyqF9pP0wwlA4GRQrwre+9S143/veBz//+c/hkksuOdG3o1AoFC8b6MCyAvz93/89jI6Ows6dO8E0Tbj//vvhr/7qr+Dss8+OwpEVCoVCsYh/tT6W1SCbzcK3v/1t+PSnPw2VSgUGBwfhhhtugE9/+tMn+tYUCoXiZQedsSgUCoWirdCQJoVCoVC0FTqwKBQKhaKt0IFFoVAoFG3Fipz3QRDA2NgYZLPZk0LH5l8jwjCEUqkEQ0NDa7KoU/vIKx/aRxQvhpX2kRUNLGNjY7Bu3bq23ZzixOHw4cMrFsZbDbSPnDzQPqJ4MbxYH1nRwLIkPXDuBReCbS8esrAwH9njZsD274xhoNlIJ08B3NOF290dPHFOzHSishUXKUAti23OFxaisuvxwLZ8R0dUNn2X2RpNFI+r17mQXCLJE+/44EflWq3CbLkOIscQ+szWbOI1LVHFFnmOTDrDbGmRLtl2UDq73mgyW2iQXwsmv0azift6oREd//H/9s1lMhLtwtJ5P/t/vgWJ1nOM7X40ss8cfI7t7/t4z30jXJp8ZBNKauT7eedNJPG4vc/8gtkO7XuSbXtlbDPL53WUzaPatR3n9X7uRbjgdfMp/N7qxXm2/czTqBkVBLyNXK8elZ995mlmKy2gaCHtkwAAnot9ZH6Oa1iVq3W+r4/X7OnpZLZ8J75fQVjmx3lYrtcW3x/X9eDH//LTNe8jhw8fjtTGgyA41iEvX5BPjpx91SpVtj03j23d2ZlnNt/Ftpdpj60Yfo/Y+w4AAeA1+ZdxbVEsFmHDhg0v2kdWNLAsVZxt29HAQj+Qlskr1raw1mMOf+y4g5dMxBxmi1m4bce5DSx+qzVy7KISAyJBjjX5Nx8MIB054EZ5Pz5xQQXiw0SvASFvdJP0Ogv4OWm9JcUzJhMxtu04uC2Zg2MNLBbZd2lgwfOsDQWxdN5EKgXJ1OIHLU5ySsRi/NnowBIXuSeSZIBNicGXDiwJ8SLKjHwmHeBF+9F97QQ/LkUyBWbEC2QH/IdKKoX3EAS8rzddrOt4nD9/g/S1EPjH1SCfCtvm11t6/3Bn7MOOw20xcg0/5DbaDXzxw2yt+0gulzupBxZHfKtcDwd/mb7Db+IPhaT4YflyHFiW8GJ9ZFULJJ999hkwWrxagSh2domcNEY3/qHH5y+mkeyLypWAS9GXfWyt0OAvYrXOfw1WazjSuz7vnDPky5qw+UvjebivZR79Y7N4TfzF64lfo0a9OyqbomVdIqmdtHnllMnMY873mC2V4jM4g8zgDEsMtITfrNb5x8dzyQfVXnymhsuvtVYoFeaj5+/OYw6bsJfnrwltfMEG129mNp98vM2A//oLqvgc9XkuVR7W+K/54R7sa+vXncJs607ZEJWHhvmsqK8P79VxeJ/w8vzlXzcygDaP95F6HWcbhXk+Y5iZwb5vx+QLhB2qs5tfP5HmM5gFMoOKJ3h/DkKsK8fm5ykuFKJys7H4jnjHqY9QnIwiro3qAtueO7IvKh9+htsWiviNufQ1VzFbLkn7Ba8ngwwsx7MGV9peJ1+rKhQKheKEQgcWhUKhULQVq6LCErYB5pI/hcysN3TzqfzGfnSe9/XylL5JQvcsc3o1kMqou9yhGYp9Y5RfFxxxGOCxHV2cuvBc6v/hHL0v/DGU42w0Oc3ieng/qRinGew0njchbJ6BU18z5BSeB/wZqa8kk+bPUSYOQtfjVBh1eZWKi1Pvpisebq3gugAtv0CzgfdVrXKaaOO24ahcrvDAiKaLdd3V08FstoO/hbZu5Y71Sy46j20PE8d/RwfPO+7aWB8p4WOh7KnhcXqoVuGUVoPQjqkkb6POPFJxWzafxmzPPEOCGQwRYNLAtu3IcYe8wxliWChORuUQeB0HAT7I/Dyv41oV35ElUSfPP/5U2CtVUYret2nwZ5g4vJ9tP37fT6OyW+PUrpPB9q0VOU2WI+nQA/FtoD6X41mDK20vnbEoFAqFoq3QgUWhUCgUbYUOLAqFQqFoK1bnYzF8MI1Fv0A2i4duG+Y8cHcSwyWdgPsmynPIA/sBH9dqJJTUFFxyLs/XM9jEd1FYKHEbeaquLOe9SyS8r1kXvLMI2w0Jr5lJ81Bgt4lhn6ZYI+GQsGVfLNC0ieOk0eC2mCDQzQDro1HmC/OAhGbHRbizR9YGLFQWufSmd3zWC3j1Ongtf5jhoR8jHuP+rAUSrt49wMN915+OocF964aYja7tAeFbogsSAQCeHcdw5Oq+ab6vif3wuSceY7bzd6A/5NUXnM9skmMuEl780MExZos5dB0PX7/Q04s+pkOH9/DjEthny2JhbrE4w7ZtB/tTLsf7eo3w+dJ9QsPuozU2J8Dd8UqVdqFrj9wG95uMHT7ItnNkrVMqz5dfTM3jt2t2fJTZ+tetxw2xpoE2lWEevzpcaXvpjEWhUCgUbYUOLAqFQqFoK1ZFheXjFlitlZdJQvd0pDnN0ZsjUhJCNoVuWbbgcMiqzkZwbCkLm4Tq+g2+Gjm08DxTUwV+fRJ2W6ryKWzV5+GamSShLxr8OSwyFZbhhlYcKZBahdMzKYdoVAlapS7UBWpkJXQgeIpCGc9bqPK6KtPV6e5iXXj+8aHCGrUqGK22yZCVw7kuHu57zplnReV1m7cyW4mE+D637zCzFUmblQsFZpst8JX44xNIH+ZEuDGYGG77g+/8IzM578L+c/nFl3Gbw+t6YIBQdSGnqQqE5nj4kceZzSYr+tNZTpN5hOZslgvMZomfgr0knN8X/Xd2Du/HBKFDR96nfH4xpNt1+bMpEJICpe/89BzvdwcOHGLbDWLPCtmmarkYlZ997BFmG9i4JSrnB4aZDcj9yAjglwO9qDMWhUKhULQVOrAoFAqFoq3QgUWhUCgUbcWqfCw9HQmwWyRvlsjhJxLcV2IS2XyZY8AlIajLZApC5IibQqbFb3L+NwhxOxTccmgjj1lq8nBN38d7rQq/g/RDlCp4jdE5fh6H5KDJlflzuBPIbdcWuB9nfQ8Jpe3jYbZGlks6NIh6b7nMr79QQh/LzAL3MR04jOfxWxLewXGSzojHbYi30gG4FoZW1pI8XHx/Ee/50Xt+yWxzsyibMjo2yWwOCdd2RB6gxjJ1Ydwe7OVdfWoCQ0JzQtK+VEDee/d+Ls8xONjD74dI1Q+uG2C2IbJ9aIL7ip57Arf7Brn/58Ah4qtx+TMGTb7tE2kaKR8Ut9HXWatzHyGVb7dbysdhoL8zjw4hG0VyMI0eOcJs+w/x7cN7Ud24J8vfg5EeXMYwfoiHKT/x4ANR+bwr8syWyhGpoxPvUlkG7UkKhUKhaCt0YFEoFApFW7EqKmygJwWxVohwLoYhoZkUpxKMkNJWIjsdCRNuCKVPk8zpurNc1Tad5grKxQWkCzpEVrYSWUF/cJSHgJYbSIXFRATucEqENDtI1xyYLTBbIyTqAiLcuCOHFNAlp3HF3eI4TqHDqjiuhyfzalTxfspl/hsg7uC+6wb4al6aqGqyuEiZeX4Ah57kU/S1QDLZB8mWyu9UAfvI3sOcCnr6KUwjbIrMhz5RJKiVOAVoEfqr1igyW6HEt0tEifjAkWeYLZ3EOtu+ZTuzAaHUfv6znzDThk2b2Pa27aiw3N3N+yxNvNWRE9ktPaQrKw2pQIGh0LUCV5XwfR6+nkhiPygX+b45EsYcF3Q1TZ9dbYVwuycg0Rew7JnH4nRWwfeEtCgoYKEoTlNpGsf8nc2vHxBVDKkuXhLpo49MYlK3SVIGAPB9VMAe6ePXf/YBpIj7BgaZbdv5F5At/v6YMmssrQLxiHRXQ9bNC2El+yy/jEKhUCgULw06sCgUCoWirdCBRaFQKBRtxap8LJ2ZJMRbYcZ2sxD9PS448lQc5SMaNaFAS7jJfJ6rIlPZhKbPxzzXFdIoGQzbG5vm2SafP4j89XSJ88ZE7QQ2JDnv/NZXncW2RwbxGv/w0D5mu2/vRFT2Ah7mapv4HKUCV9WtlvFes1nuUwGfc6OJBNpjgiNPGWiTmf/WE0Xg7Nwi7950ffjpcfCx5Du7oyyhew/vjv4+foCH7aYcrIeFClduLhenorIRcE63UEK/SaHG+4Qd5/XZ04/8dVL47IY3nhmV14m63f/YfVHZMnjbuiLN6PQMhoTv3LmD2U7ZuhmvIUKKMxedHZUff1ZIgNTRn9hwRLgxcH9iEGLbT0wIdWUqu9TZBxzou6rVFn2JJ8bHsrIw+PBYPhZ5CiZ3IsKEgT8j86sIKRQDqP9FAv+yfuNGZkkJiZ5ihSwHMPh37cnD2NeTtshkSsLln7r3bmbrHkY/aufIZmYzPOnXxnuV9RiQb5W5gqZY6aoFnbEoFAqFoq3QgUWhUCgUbcWqqLDezi5IxBYPqc0hDWEa/DRlorZba/Kpp22Qle8upxXoKFdzOQWR7+TTyyZRgN13hFMAc0US0mvzUGiLyMPmEvz6fTYP10zMIV2zNcdXVY934XkmC1PM1qjivT+yezezmSTBkpvmzwQd/XzbJOGqHVydNhvg89eFKkHYxLDbjb3p1j7Hh+bYv/8hiCcWqZxnn98b/X1s/Hm2n0/CiLMdPIna9q0bo/IZO85gtvFppBUOTvNQ5N4BXn8btmBocLabU0GT83hsOMNpukMHkZqaForJJAcYAABcvQ3pr0qZKyBQYe+wyfvzU/cj3bZ1+1nM1j+cj8r3//KnzDYxyUOqKX1Vr/FrzBN15WQmz2wBCRutVBfrwvP4+3B8sLLftsYxKBhJdwF5N4KQP5Mr1BliMfw+GMsuQikkacLvWGcnV2O47NVXsO0nHn02Kh/Yz1fX+6TO91oTzJbYiJS2/xxPBvfE3T+Pyhdey2nWZIqv7qcMuxQ+ppveMWjJJVpwpT1EZywKhUKhaCt0YFEoFApFW6EDi0KhUCjaitVlkOzugWQrpLMzg6rFpsnDPAtFDB91iawGAIDpU3VjHkoZkrDlTIZLuLjAt5/Zh76LSoNz7YkEhu0t+YSWkEyjr6LT4n6Hh/ZyJV2vicc2OriPpbcT78cQIaCuh/6napPz7hUi49L0ROij8CtRAtQxhRK0SSRlRHZNr4G+obDliwr9FcYJvkQ88PO7wG61o92PUilbduxk+yWJSu+O03gGye3bUPXZr/NQ4NDE+qwAl+uxHd5HLCsflV2Ph3JWSiit0SH8TzSD46EpHgqdyIyy7Y4chsxv3rKR3yv53VYrcPmiZ3/xKO5X4+/BGa9/Q1TeuYuHktYe5D6W5/ceiMopwa135LvJFmfHi+QdbTQW7+2E+FiYpsix9pNhw7gtD/NICPaevdw3Uavxb8WpO9BHFo8LlfZjZGIMiKRTID6jl1z6KrZ9aD/2mb/9H3/L75X4xQ5NF5gtnsI+u7WLzwGe+9mDUblXhBufeukFbLtKQqwdoWAdI884VxXq6k38jiz5gkol7oc+GnTGolAoFIq2QgcWhUKhULQVOrAoFAqFoq1YlY8FTBug5U8xHOeou8WJFEkK+BoFm4xlpilkW4jPJZ7kEhwzE5zbq84gR7y5i3PrDaL0kUjz9R/btwzj9RtcEsSz+DNRHtq2OP+YjeFzdXduYbYtW9dH5f2HHmC2Z3cj3xqzuRRNGHJ/lOdh85hiPY4Tw3sNAin7QaQoWhIShnF8fkNMj86CZS3yz2ef+abo7/E4j7XvInT24BD3Uc0RqfjDe7nMeDNA3tk0uE/AskV2xZDUryel+dFXE4rMoZkOXJcwKzJ3mjHen3lmTrmegpwzwZ9x49C6qJyw+HEmYD/YeQaX6c/n82z71tqPovLEOPcHDfeRdRAG7+s082WxuOi3WVwTw9ddrTVo/cllJHR9Sihki1h3Fr6Qw6O4Dun7//QDZisW+Xt8yQyuQbvy8tcwW5xI4sgMrLTHyMyzmSxPY3HNW66Jynuf4/V7+z//GO9NSOo8O4rrWjoNnok3UccKuP+HP2I2u5v72sz+fFSuFPjzO2Sx1XiRSz4tlHDfen2x/9Sq3Gd8NOiMRaFQKBRthQ4sCoVCoWgrVkWF1eteFB5ouHRKxKdwlQqGRDZdPnZ5JtJW5Sqnt4pke3gdv7XQ4/tu6MHp75YhTmFV62gb3nYms8VCpATmF7gUSpKFZwLALPI160QGt0IFKZLNp/Jw2VxnipS54u38ND7H/IKYlgqaxQxxKu4GnPah7JcvptA0MnmJTlgme7FGSKY7wW6FPzvkkgUhexPvykflqsephDphbZKdnFaIB+Th6rxOQtGb6y6G+CaSIsseUS0OTG7LdCOFFAs5FWclhSJ3jISdGjyk2PCxPU2LX8NJI7WZzHCa02tgH5kd5SHw3WlOKb7lja+Pyg8+doDZyiSUtd7gKtuNGr6/+WweAHhWyeMH0oaCrp2fRzmdhXneDoaF/WBimvet+x7EzIsPPfUYsxXnCmy7QUL8T9/J5YP6epEStUT7FUvY1oUCP+fGkRG2PTSCckI3/Jv3M9vhUZQ6+sVjj/N7q2Df2nOEy72kBtA2++STzFa9iW3ClkvPicrzZeFSqOK3umEUmK3pIpUctGRy6jVO3x8NOmNRKBQKRVuhA4tCoVAo2godWBQKhULRVqzKx+IbPvgtHpSG/0n+PpnA0LhMlof7jhHZ8/1HOO9rE1I+Nsml8OuTfN+tfehXueoK7uN4fhT52Oww56R7ulGaZWqa89f5vPBxBCSDo8nlHqamMWzYThSYbbowHpVHx3kIseNgfeRz3LdQqwnZChvHfUNIugTE5yKlJwwSxn2clFwiDKzbAI4TW3Yf9TqXIpksYteL5bnsuOsRKXMR1l4rY326If9dZIsMfJ6F26kcD/ft6y5E5XCOh1A2ic/KEBIYySQP+6TdgmZzBADwiXyR6QhpGpK+oVzhvDfNmhkXIflF0WeTqa6o/OqLdzHbc8+jRPuTT3OOvlxEH2GsJYVz/DJINlr/eD+W2iwLRZTs+dm99zDbwTEMjZ0pFphtntSnmeb+q0SDv+NTs/QaP2O2jRsxJJyGHgMAjJJvlytSItSq/H7KJdwWyXZhx/kox/Lo3ieYrVnCl/dIgb8/qRjez0gHX26x/8GH2bYVJ0s8hrqYbcFDXxHvoQAQYt01WjJRjZVFG+uMRaFQKBTthQ4sCoVCoWgrVkWFdXSkIZlYnB55Nk6by2W+qjckmSHp6k0AgIOHcCpfLnOaKJnAcW58P5/69Sf4lHZ4eENUzg/x1clOiVBMCU6ljJyJyp+JCa5Um/Q43eYDPlelwp9xMIUUW1OsvDXSuPJ1JD3EbNk8UnGlWU5PTE3ybIWugfdeb4owPxOnyek4nwo3a1ivSyv0/WNKx7YPoWFB2MquR6mVqlBFjRNKqVQUq+vr+KzVIj/OIY+RTXN6oreTT/NzXUh79OY5heXbqOxQi3MKaG4DtlnDH2c2cHlIsU8yEgYBr2PfxH5hCCos34Vhy4EvzknqraOD33dMLE8vEJoldPn7dNYO7Gv5LK+rH/wAV2tPTy7SQcdL3fiZ556ATGbxHbFt7OOSUponYbyFMv+OHBrHd7ejjy8T6CJ11t3DqfDp53l7PvMk0k8/vv3HzNaRw/NYNm+/RpOolAsFjx/+i1A5ID/faegxAECqB5//zLNOZbZH7nkuKleFEvzuWfyOJn1O73V6PER/7/0PReVCL/9WzJE+6jS5zaPvb7WlgO2uLCRdZywKhUKhaCt0YFEoFApFW6EDi0KhUCjailX5WMoLc+DVFzlBu4nctyOVcwkdaVucm6wSrrQzy7nBfBo5vto897H0DXEedXjX5VH5ySOcm929F7cvGeS8e6GAtv4tXO7FBM51N4kMRj7kHGdxCv0hSSGFMdiF1yz4nNt2diG3Xitwvvfn/3Qr2z5yGK9vxaSaNPL5IkoZXKog3eJE68crlNRrRrdmB1jXIiIS1nXg/Z+6Oc9sGRKubom+VSGhpXWR8S6Z5u2wfSu2w7oNXGbDdNBHVxaSHOsGUb5n+34uF5ITStpdnRjGbAsF6oC0SyhiOanqtlcXkjzkOEeEG9eB+9q6e9CfV67y/lspoA9vuJf7Gt567eui8i233Q4Axy/c+BcP/RKSycV6rJGw53SCfw+uueYtUdkL+Xv00BPPRuWOLJfZqQXo4xjq62c2d5LHyy5UsM6qe55jtk4Sppvu4PeW6cT6TKT5t6Ejzxu7g4S653JceTiZwX5wxWsu5Pc2g/37ySf3MZvv4vtzqCCVq/m3wp7Adi3Ni2ypWZIJOMnD/kcP4/ep2GqnwF+ZH05nLAqFQqFoK3RgUSgUCkVbsSoqzDQAlkRFfRLSGopQVpOoHfsGnxbOE7aiWBQrzRtInQyKqef5V17Jtke2XxSVb/rqV5htgIT7Wk0+9R3dh2qiA5tPY7ZE9ylsOx0i3Ved45RIMsDpd7PGKYgZonya7+Wh0N0DG6NyrcxXg5t8E/wYTnHlynuXqLIaIkzUCHF7KVmYe5yW4F96wVmR8sLm05BqHBvlod3DZAXwtq08UdpAL4ZkWiF/7hIJr22I0F9ZR5k09qFMhlNYVgwpACcQK6crSEGec8YGZtu4bSPbdgPs0KH4neYFRJ3C4vdmkSXYbl0kkSKUlGnzcxoJETZO7A0RCmqTxHV+s8BsvYRCu+xV5wMAQK3egJtvvQvWGgcOHoB4a/nAwhQmJ9u6iStoJJPYfmNj/P07uB+TeWXSPCSb9gujyN//WkHQfaTPnLJlMzNt6cWQ9Gwnfzmnpgil38XbaHAd/3aVing/Mc6aQSLA72OOXA8A4Oo34DdvTrgGJo9gfcw0+ElTC8KNQKg4W4SrD2fxPUz3DzDb6IEDUbnZUp6XSQWPBp2xKBQKhaKt0IFFoVAoFG2FDiwKhUKhaCtW5WMxwsV/AAA+4XMNERJJaeGwxnlfg1B0Xd1c+XgghfznOedtY7Ydl1zEtuen0McT93jY6WaSwS0wOCc40IdhgjLMs1rgXHvTQ7tb41XlA3LUz48eYbYnnnwwKl9yET9n9wCGTRdLnDd2eHVAz0bkagNRx36T+FEa/BoL04Wo3CgtnrThHh+5jrNP3wbplm/j9LPRx1I7g/tR0h3I+0rWNiRqzabFQye70sgDC3HjZb+SKB/syVBa0n8bQrJ1yynro3JSZPWsVXhfC2n2SUNkPSV8diAUwH3yjEHAbU2S3dEPhOK2Lf2Z+NSlWe5zOrj/cFS+9LKzma3qov8w1fLbGMKftVaoFhfAayz6WKp1fNZ4ivvBqBzUwcMHmC1P+o8v5JYMIgk0PrGX2cbHZvi+Ju77rne8ndmCMkoN3XnPT5jt4OPoM+zu4GHmE3t4PQ4PYX9acLk6NTj4Dejq5qHRO7djRsvmW3nf+sr/+buoXCvx5x8rcGkfIGHwjSZ/28ozuGxiqIP7kWJJfPd6+vIAsKjYfeQQvCh0xqJQKBSKtkIHFoVCoVC0FTqwKBQKhaKtWJWPJfB8CFqZ72okdjqW5jIFVArbMjn/f8oArv9IJPm4tnEDZmw78zK+bmVwO8+O9+h9X43K69dxSYeB03fivfVybt9OYax4tc65yJqQaJ8cQ456fpL7UXwSK5/Mcm64h0hhHx57hNn6B4ejslfl1w9rXK7DqGCMvx9yPwDl75Nx7oeIDeB2Mb7I99abx4c/T6TTkGz5WDIJlOFIp0RXIzLkwsUABvWxiOyYAZHWCdxA2PiJqO/PE54cuuQlFLIxmTzG9nsiJYIfCG0WIpUfAvdjmfQivpDUJ+9ICKICiBS/EfBzxsX1HZ/IjtRFlkoiXzK9j3P7I9vRDzljtvqheXzWOjWbdYBWXVUbKOmydz/3h9x8yz9G5XvuvpvZqD9ossjfo+mD+N46woHnivqMDeD34Oc/5RkkGySD5dN7djNbZRJ9doVpfs58N/8eTBNJleJChdk6STqHps+v8ZOfYCbIZI5LWnX24FqvGZen26g2uD9xlPhgwjjvhylyP9Y09/nmu7FuLKu1Hs514bGHeKbLF4LOWBQKhULRVujAolAoFIq2YlVUmGPZ4LSmRPNEtsSv8+lVMkXUacX0uo+EGB8eLzDblnPeEJVHdr4BODjd5ZZwCteR5VIIvdvOisoVm6sbP/XIA1G5UePT0iJRzgUAmBnFuDrL55ReIoFVN7xpmNl2bUNpGM/i4aKOlcdyTEhw1HnYYPUghjQGQrbFIz8JykJBOtWN1+xvqULX6scn3DiT64RsKztgSEKFqyIkOmwg7dcQtkoZ26XpclujgXXmeZzncIWkCZW9qQrl32oFaU9PyFRku4iUR0ee2fJZrgCbiGEopy+kYcAg0izA6YksoU9np/hxdSKXFAS83xsgFJR9rMecyBK5YT2Gr9aqvK+HRG6mo6Uy7oh+tFbIdeYgHl98Dpf042KZS5E8/eijUXly/35mM8mnK2ULKtjEOgpFVkpTyE+NEGq6S6gkz1eRSty8cTuzHfSRpi7McSrKj+fZ9iQJh65W+XtYmEOK0hD1XzfINarPM5tJJIkCi/eJMCYU5QkN7It3Jk3Ok+ngz29Z2DhBSybKFUruR4POWBQKhULRVujAolAoFIq2QgcWhUKhULQVq/KxNOsNMFvheqk4HmokRAikSeTCfc4tJzO475vf/WZmu+TXrorKuR4ubzC57xm2bZFrFEpcZmP6AGaCGytxTvMnt9wSlTNJzs3WGzxscaAfufacyHa5/wiGNDZN/oxdQxuj8rad5zIbkIyScwUewlwVvqr5Gp7XCHlT1WvIlZZFmG1YRk53R761v6D/1wq3/dOPIZFY9B/4DoZvzs/zcNfyAoZyyihX6nOZnOTH+SQ2uYvI6wMAdPbwkMy4hXVWmSsw2+492J+KZd7u6zahVL4lsvHlsvwamzahXMfIOi47vmkz4e9FmGc2gecNhJQGEK7dFe+PJWT0LXLe/o3C/5PDvuaG/D2gtHxX1+L143GZpXRtkO7MQaIVim6T96o5y/1AM7vxHVuX4X5Ug/hRSjXum6yT99FI8tDfuEjjMT2Jsi0P/eIxZuvPZqPy7HyB2RaI7E5ZhDTXZriviGZ7tYU/JOlgf64Lf9A0yWzqm8KPaqNvREpqmQnpKyM3GHIfSaWCz1EUKQY6u/PkFK1nMFYWkq4zFoVCoVC0FTqwKBQKhaKtWN3K+7CJK5/JClZDhLB5ZLpliKlTIo7T/rPO5TRRnNAOTz/KV6zPj/Fwu0YDp7+l+TlmO7z36ahcDnl2OcfH4zI2nzLmEpzu6u3E6ff45ASzeSS0tVriVMrh/VT+8ylmK5cxzDVh87rx4pzamfWwrpJiSp/K4nMlbR5mWqriVHwpi6EXHJ9w47t+9otIeSE/giGaoc/r6JF7MVPhBqJGDQDQ04100+gRUe/kOVJdeWZrmrwfThK68qoLLma2s3adHpWrDU6lmCS74/5DB5lt9x7eD594EvtpvoMrULzjnW+LypeeztW6Y0SaeWRwHbM1CRUms2JKdQGXrPY3bbFKP499JinoksBC2mXprbNX9TX41RE4JgSxxfsJiSJBzOL36BBF7vU5vmzAI9RQqcYpHCuH7WDG+HtTm+S0eaOAYeilWa68MRPg/RQaPFx94zmoBDIxzcONC/P8GpkMflfqIuzbdfD+6mLFfI0oS5iiHyTIc4UGp7d8oTJhkYY1PZGtlITaTxFVdAAAusLBji1eX4b0Hw06Y1EoFApFW6EDi0KhUCjaCh1YFAqFQtFWrJJVDWApdC0gCqy2SH3oE3KuKaQs+olswL/c+gNm6+pHf0Sf5J2rnLd0HPQrZNI8XNMm/GtahIsO9CF/XyvNM1vS4r6K2WkMiXWbnL/OJogqqQhX3fMIZpAcf5YrljY8wgc73McjQwrTI8TnkxbSFHH0CyQCXsedgPe24/RNAABQrbkAwMMp1wJvfed7IJlc7A/xvq3R36sl7ivZ8wTey+AAb2uT+AOSCd62zQDrb9sZW5mtc5D7qKo92Neu+bXXMhv1UVWEj4UIFoMXcr667vF9p6bQv3dw/xi/RgrvfeII5+EPPLUnKptCymffBKrMXvC685htw8Yhtk3Dkc0ED2UFh/hBRR8BA22xVpbVmHN81I0XFspQb4WUN6rYr9NN3v97B/BZZw9y5d29B9D3Ne3y+uvqQn+MmeA+1krA33nfxcb2qlxdvN4gWVqFr3h6Ar8NlTL3v4Qu3zcVx+9jU4RGG3H85nh1fv1YGt//UKhs10lIfiDi9Zse/1bEHewXsQT/xmVS6I9KpriP0CXPsfROht7KVNJ1xqJQKBSKtkIHFoVCoVC0FasLNw4MCFo8QYyE6iZssfSUhMaFQt03IOqYMzOcHilP43bS5atXA+DT5K5OpLTyQ73M5hHF19Exfg2aVMk0+eM3PbHK2UAaLZ3gdB+NsLZEuDVdneo3OYVnEp6lWOXT8mach01mh/A5KskCs5WIkm69wn8fdOc2R+WeFvVXqRyfpfdxx4R4K5R097NPRn8vLoh2IGGzrlhxXCbqxoZI9JUgq8PdKg8PXZjmlMDkIQw3/ud/+Wdmmy/hsQtl3kbZHFJYHZ08zDWd41TCkSNIf/X1cJXrRA6puZ/dxq8/t+fxqOwLxdi9E6g2cKTCn3HrDk7/deSwX3Z08tXpyRSGpHakOSXskNXZqdTiMzVlP14r1B2AsHU/hP3xDE7lVcgrPy5WzI+Tey03xX3PYntajlC1FkrWIXkfa+L9D4laQczh9zZKaHKZDM4QCsrT8+Q9F/059PEaTpLTdjmqnC3Uzen7I9UYksDb2iRh3I54DoNcIxR1Y5DjTMNu3b6uvFcoFArFCYAOLAqFQqFoK3RgUSgUCkVbsSofi2nEI64tEUc+MBQhxekk8r5pkXGvSkIDu7Oc77PJeZoLXNU2MPm+VQf5wP7+TXxfwtlv38XlQu696w68Rsj5V0fwnzUSRpjL8rDXGJFJsAzOTZZJ+Oj+ce5HKRTwGRsGl3fo3cbH+eE8CWkO+fPPz+C9xeqcU00Pk5DqVsa6Wu34SLqU5ibBqy3e953/77bo74cnuJKz6aI/6fHHhRosaQfPk2GyWNc//sGdzBRzuP/jrLPPicrNWJbZikSiY98hHso6O4vKx806b9uxiQNse/8B3Pe8s7lE0f/3oT+Myr+8/z5m8xYw/LjY4GGmNeIH3PfgYWb72UPjbDtto3/GEZkDLRLKmhU+lpENG6PyW97x/wMAgGr1+IQb24YNdst/6RJfQbnG62GuiP1irsltHpHdCT2ReZGE9BoiO6krwsdNujRBqExbRFrHEno3RJGH+TvkcXJbSrNQpZ1AqhSz6/P79om0USjPKa5Pw/elzxIMkiVSyD7RV2/pPfTl+3gU6IxFoVAoFG2FDiwKhUKhaCtWRYU5tgGxVmhblUzfLaEKHJAV7FVXKI+S1b3xmFAedvA8sRQPnezI8WtMTCNVVh3mdFffulOi8ujUDLOdfv6lUbk8zVdK79vNlYgr5UJUti3+HB1k2mwINdHxUTzvoYMi3DiOz5Hr5yHMvV18Km4QSs2Y48/fOY9NN9zHQ2JH8lgfe59eDPOt1VemSvpSMdDXD6nU4r1u3YgUZSjqyCZKxJYhp/L4eycMOM0Qo33N4cq1Q0M83PeK178+KmdTvK47Ergq/+knuSLB7r2oYDwwvJHZ6iH/LWYR2vfJ3c8y29O7UXUhtXEHs42N4fU7853M1kdCQFMZ/o7MTXC15dnRvVF5eobTx3WfhHQHvI7HC9h/Lrlq0VarrWxV9UtFpVQBtxViXSwiHVwp83esUiH9X9xaLo/vSjzJKVAKmQQraXNK2YnhsZLCcgjdJqkwn4TmSioMgG9TsyXuhy1N8CUVRRImSlVrYvPF9Syh2m6Te5fnWUrKBwAQdwTdR6ixeItWNeX9HwU6Y1EoFApFW6EDi0KhUCjaCh1YFAqFQtFWrMrH0tdtQiqxOBa5sxguWROSBhUSRRuanDekfF8u181sMaJEXKvwENSk4P+gidsP3nsvM23eTiQxRAZCGu6XivMQTEuoGyeTyOdL/rdGstZ5Qk00QzjfS87mmQMTJGzZs3jonu/y8OfaYeSYzRL3J/SlMHz27G2nc1u+Pyo/NL4fAADqzZWFCb5UzM/MQz256H+76MJLor9fcvnlbL94HHlgW2QOpDxuIMJDLSLtIxWna01ef7NH9kflOeFjmptBVeJ9e3lWyLEp7DOZPq4mDHHeDkaMKNd6PCT2x3ffE5U3bNnJbOu60B+UENJCKRI23ahzSZd9Re4HzJD+5Ie8jSfmUXW7p2cjs1VJdsI77/4lAAC47vGR/ZmdmwMn1go3Jm1Yr/PrN8myASchJWnQV1ITGSSpj84UiuEgtsOQhLb7vP5MIpWSTPFvA/PdCL+FHxxdGkeG+0r5F4pqFfuz9L/YNNxahBtLvxK95nJ/EDlWmBJEGTrysUhn11GgMxaFQqFQtBU6sCgUCoWirdCBRaFQKBRtxap8LCMjMcgkF7nODgO55r2HObc9SeTLm77IWJbBS1ZEVkg/QE7YEmPe3DTPwFcqIx9ad/l5rBC3sxm+RmByArn1IxWROTDk/GF/L/qAjIBz9PMFlGqJp/kz5jvQ/xET/oMG9QvYnDeuNPi+zTKR7Q+47ZR1A1F5aID7qg4fQR/T7PRi2zTc4yPpkkrFIdXyMc0WsX4fefwhtl9fH7ZLfx+X/XFdrOv5+QK/AFnbY4s2Gd7E/SHrOrEdRndzKZRKGf0hff0DzJbqzkdlS2SwrIoMgIOD66PyxBiXrZkh8u2DQ1y+x6BSJg2xxsjG/uQKmY14kq9nihPOuzk7zc9jYv/pF+txmkTqZOlWltHvawTXa+L6DbIuyBbvA1GkgbiQlKeuAUN8xeh6FLEMCnzxjlPfhSX8LxaRyDEd/v7FyL1Kv4X0hyz3ayBo88o1Ivl8PirTdwIAoEH8T76Qspd+HHp9KZHkeeS8vlzrtnyNjbyPo0FnLAqFQqFoK3RgUSgUCkVbsSoqLJd3IJNanALWppH+6uwTIX1pDMGcmeQhmHUyhbNjnGagiQQDQd24Pj/PQg2pqLSQdKhXka6o1bmkS5Oc13fllJU/R7lI1I1zIrtbDiVnajVOBc7M4r1lMpy6oKGAhifkSmx+DRrZGhPKtRtP2YjXF6q0P/3p01H58d2Lyr0yy91aIW4HEG8pTzfqhejv9957B9svJCrXuRR/btclNKcIJbXJb6ENG9cx2xkXnca2t6xHaqxwmNNUE/PYL2Ki/2zpRmpserrMbDu3n8G2T9+5PSp/+xtfF/eKIbGuoF2bTdwORXZASODzU4ViAICNmzaz7anDz+GGoHKShKLdsYOHvder+FzrBhczXTYa/B7XCl1dXRBrydaYJNuh70vZEuyzku6p17FfGJYIt2WKvbzfN8V7YAXi20VtjFIT3yNyb8cKGV68HywHgpvzSNsH/tGlWSSFRSVd3ECESQtpmmOFGzPlZTg6pbdUj55SYQqFQqE4EdCBRaFQKBRthQ4sCoVCoWgrVuVjsRI22IlWBskc8sddGT4+2SQTnJPknGaRyL2DLyStE31ockTGtEaBbcdSeB5HSGFbFvp4GkISpOnSMEvBzYqowJDw4L6gnx0aGhnjPHhhHn0stSbnJDuI3LctM8aJ56iSjJqTM1zaY56EW5cqPNz69p+gfPtky/0jud21QrVew1BQ8nyv/7Vr2H5BE8NvLZdzxAHhwUOZjY/UUSLNpfAnCtwfUyqgbP1cjV/DIHLhzz26j9lm78Ow3c2btjPb+adsZdtNEn6cFP0gJHy0DFM2Ley/QtEeasQvYAuZkQ0j3MdSL2MY/mkitcQvH3okKo8dfI7ZakR3Kawu9tfmcZJ0yWazkURI4FNJERmaj/VXrHJfl+2Q7IoO7yMs3Fe4rxzxznmkrgMRJsz8KoaQSaHfjhd5twLi1wiEjyckv+2lfFGzhu0hw3wD6g8Rki7ybqifKRTWFHkPYkJun8q3LElxudbRfVLs2BXtpVAoFArFCqEDi0KhUCjailVRYZWyDUbQooCsTPT3TJpP850kTrfSQg22owOnZeUipy7KRVwxXq6K8L46387GcLV5wuErdj2S3dK2xYpZsunEZVge3zdFVAKEAC1TQo0luTGXR4pmbo5TWCUy3c118RXzVaGSvOcA0hzPPnGY2fpJtsn+EU4JAcnO2NNSAfCDAA7Or304aTrtQCq1SFd1kFl3tpeHuzZIGyXE75uYgXRXKFZcx1NoC+qcHimVuCK2lcI66tuSZ7YtKQw33rOfqxuDgf3CEaq2o+OH2HZ3T+cLlgEAmjWkmxoNTlfS7IgNQfO4DQxftxO8bfuHetn2wXF8ZyYP8eeol/Gazz/1KL/vbjxP2LmYgTR0j09IugEmGK02Nwj/3HTF0oQGfh+k8jINqZWUckjopqYI022I0G6D0EhSFZhSQXJVfECWChxDL3hxX3pvYlU8y0Rp8Po3bdzXsfg3jkIw+i+gBECouGXJLvGapvj+UZvXWprha7ixQqFQKE4EdGBRKBQKRVuhA4tCoVAo2opV+VjGDgOkWi6TRgF9J9lezmMmksjDdWSYCbq68JLlCpdCKRRwe36Wh97Oc3FjJsUQHEtdVKjD0pHUEGF6ls2ro0bCoUVyPnCIsq5XnWM2n0i8+EKxtVBGm0iACHPC53RgLz50YZar4zYrePBAB1fn3bEBsxMundL1A3j4AL/PtUC1vBfAb/UNosjsGLwjTE4i/7/n6QPMliDSNrGOPLP1EFXkoZ4OZpNce3cH+rCkok2dSAL19XFpoeGhrqg8PsEzkO7e/Qzb3tjcFJWp3wgAoFTCZ6xWJ5mtuID+IOlj8ZvYD6w4DyF+6kmuBE1Vivv6+plteBfKz/T1cltPL/aZROsa9eMk6RIEQRQC22jQkFqZQRLvhz4nAECThKjLMF0qsWKJ8NiEkMgxSYitL/wx1FchpWEMIp8jJV2kPyZ2jBDdOlHrlrItFjmPfA56b7LfVav8O0IlXRIJ7vOm1/Ca/DzU55JILNabEa7MD6czFoVCoVC0FTqwKBQKhaKtWBUV5jvd4DuLUyI3dl7090YgplAehnImOvg0Md+LU7FOk0/9uqo4zSrM8TDTwgyfCtYqeOu+x2kzuoI38PjUrU5WQC8prC7BEitPS3U8tlYWIdUhTs2zZpbZAhNpDtflVRxP4xQ24YgEYTE+3d8M+ai880xOiWzfdWZU3njKKcx2wUVItx0ZW6RZGk0P4OEDsNYImw1YYilN8rvFdnnd5oiywkP3381sE5PYfwxRRxdccG5Uvuzi85htYYGH9D7+8C+icqXO22/3IQzf3nfgALPVqlh/Up0hkePhvsUihpOX5rmSdqWIdJsMQbWJIm9HlocUD21Ceq2ze5DZ+oY47Tl09s6o3CVW3lMKRlIpNKR66X2RibbWCp7rRXQRpb8kFUQzj9mCpgZGRXEwxV4Ziizob6oSLK9BKXUDpCow1pVpymULR0+0FQpKjX6D5L0eiyZzyBIL2bbyPPQ55HliCXy/UnHeD+lTLD2TPPfRoDMWhUKhULQVOrAoFAqFoq1YERW2NJWr1nHaWiNlwxECaSTxjFnl00K7QvY1eVhUpRaQMj9ntc6nkLU6idgQM2g6Xi6jwkh+cZn/2hIidLUGXqMuxCTDELdtQenVm7jdkPdGVhpbIrFYQ4gxNskqYUfYaFuURRKpGnnGRutels59rPzbLwVL563VkRZ1STt44lnrZD9f0AM0yk9GoVDqoi6iYRoicojmBW+K9qOUgIz4oXSFpMICEWUYEJVDSXMcq66pSV7/WNSFjJyiEUH1Bq/jwFwdFbYUFbbWfaTJ2uXoVJhL2swVSfE82kfEdajQ43IqjO/rkndMUlh+QNuWX5+KZ5riO7YaKswnNhmVdqykWvQKvvhuyXqk4pqeFOF0cdsVdN8LUWFLYpgv1keMcAW96MiRI7Bu3boX203xCsDhw4dhZGSk7efVPnLyQPuI4sXwYn1kRQNLEAQwNjYG2Wx22WiseGUgDEMolUowNDS0YgfcaqB95JUP7SOKF8NK+8iKBhaFQqFQKFaKk955f+ONN+qvo38F+Na3vgWf//znT/RtrAg33HADZDKZF98RADZu3Ag33HBDtH3gwAEwDAO+9rWvrc3NncTQPnL8sKp1LArFyxXf+ta34Mknn4Q/+IM/ONG30lbcfPPNkMvlXnxHxYtC+8jxgw4sa4BqtQqpVOrFd1QoXgRnn332ib4FxcscL8c+clJRYbfddhucddZZEI/HYdOmTfC5z31u2T5hGMLf/M3fwFlnnQXJZBI6Ozvhne98J+zbt2/ZvrfffjtcddVVkMvlIJVKwaWXXgp33HEH22eJanv44Yfhne98J3R2dsKWLVvW7BlPBuzduxd+4zd+A7Zu3QqpVAqGh4fh2muvhSeeeILt97WvfQ0Mw4ADYmX8T37yEzAMA37yk58AAMAVV1wBt912Gxw8eBAMw4j+LWFubg5+7/d+D4aHhyEWi8HmzZvhT/7kT5aJ9xmGAb//+78PX/3qV2H79u2QTCbhvPPOg/vvvx/CMIS/+qu/gk2bNkEmk4HXvOY1sHfv3mXP9pWvfAXOPPNMSCQS0NXVBW9729vgmWeeWbYfAMBTTz0FV111FaTTaejt7YXf//3fh2qVC7NKmuNo2LNnD7z3ve+Fvr4+iMfjsGPHDvjSl770ose9XKF9ZBGv2D4SniS4/fbbQ8uywssuuyy86aabwu9973vh+eefH65fvz6kj/lv/s2/CR3HCT/ykY+EP/zhD8Nvfetb4amnnhr29/eHExMT0X5/93d/FxqGEb71rW8Nb7rppvD73/9+eM0114SWZYW33357tN8nP/nJEADCDRs2hB/96EfDH//4x+Ett9xyXJ/9lYa77747/MhHPhL+wz/8Q3j33XeHN998c/jWt741TCaT4bPPPhvt99WvfjUEgHD//v3s+LvuuisEgPCuu+4KwzAMn3rqqfDSSy8NBwYGwvvuuy/6F4ZhWKvVwl27doXpdDr83Oc+F/7oRz8KP/7xj4e2bYdvfOMb2XmX2vGSSy4Jb7rppvDmm28Ot23bFnZ1dYUf/vCHw7e85S3hD37wg/Cb3/xm2N/fH+7atSsMgiA6/jOf+UwIAOF73vOe8Lbbbgu//vWvh5s3bw47OjrC3bt3R/tdf/31YSwWC9evXx/+2Z/9WfijH/0ovPHGG0PbtsNrrrmG3dOGDRvC66+/Ptrev39/CADhV7/61ehvTz31VNjR0RHu3Lkz/PrXvx7+6Ec/Cj/ykY+EpmmGN95446/SRCcc2kde2X3kpBlYLrzwwnBoaCis1WrR34rFYtjV1RUNLPfdd18IAOFf//Vfs2MPHz4cJpPJ8D/+x/8YhmEYViqVsKurK7z22mvZfr7vh2eeeWZ4wQUXRH9bGlg+8YlPrNWjnfTwPC9sNpvh1q1bww9/+MPR31f60QjDMHzTm94UbtiwYdm5/8f/+B8hAITf/e532d8/+9nPhgAQ/uhHP4r+BgDhwMBAWC6Xo7/dcsstIQCEZ511FvtAfP7znw8BIHz88cfDMAzD+fn5MJlMLvsQHTp0KIzH4+F73/ve6G/XX399CADhF77wBbbvn/3Zn4UAEN5zzz3R31by0Xj9618fjoyMhAsLC+x8v//7vx8mEolwbm5uWb280qB9ZBGvlD5yUlBhlUoFHnjgAXj729/O8g1ks1m49tpro+0f/OAHYBgGvP/97wfP86J/AwMDcOaZZ0bT5nvvvRfm5ubg+uuvZ/sFQQBveMMb4IEHHoBKhedHecc73nFcnvVkgOd58JnPfAZOO+00iMViYNs2xGIx2LNnz1EpgV8Vd955J6TTaXjnO9/J/r5EHUhq88orr4R0GsUcd+zYAQAAv/Zrv8aok6W/Hzx4EAAA7rvvPqjVassoiXXr1sFrXvOaZdcBAHjf+97Htt/73vcCAMBdd9210seDer0Od9xxB7ztbW+DVCrF+usb3/hGqNfrcP/996/4fC8XaB9ZxCu1j5wUzvv5+XkIggAGBgaW2ejfJicnIQxD6O/vX7YfAMDmzZuj/QBgWUejmJubY51rcHDwqPsqOP7wD/8QvvSlL8FHP/pRuPzyy6GzsxNM04QPfvCDUKvVXvwEq8Ds7CwMDAwsCznv6+sD27ZhdpZnkOvq6mLbS+qzR/v7kgLt0nleqB8MDQ3Bj3/8Y/Y327ahu7ub/W2pr8p7OhZmZ2fB8zz44he/CF/84hdfcJ+ZmZkX/PvLGdpHXtl95KQYWDo7O8EwDJgQ2f4AgP2tp6cHDMOAn/3sZxAXmeQAIPpbT89ilr4vfvGLcNFFF73gNeXgpGtlVo5vfOMbcN1118FnPvMZ9veZmRnI5/PR9tLsUzpQV/MSdHd3wy9+8QsIw5C10dTUFHieF7X1S8XSB2B8fHyZbWxsbNl1PM+D2dlZ9uFY6qvyY3IsdHZ2gmVZ8IEPfAA+9KEPveA+m4gM/ysF2kde2X3kpKDC0uk0XHDBBXDTTTexHAalUgm+//3vR9vXXHMNhGEIo6OjcN555y37t3PnYm6LSy+9FPL5PDz99NMvuN955523LJeLYuUwDGPZwH7bbbfB6Ogo+9vGjRsBAODxxx9nf7/11luXnTMej7/gL9mrrroKyuUy3HLLLezvX//61yN7O3DxxRdDMpmEb3zjG+zvR44cgTvvvPMFr/PNb36TbX/rW98CgMUIppUilUrBlVdeCY888gjs2rXrBfvqaj5CLxdoH1nEK7WPnBQzFgCA//yf/zO84Q1vgKuvvho+8pGPgO/78NnPfhbS6TTMzS3mer/00kvht3/7t+E3fuM34MEHH4RXv/rVkE6nYXx8HO655x7YuXMn/O7v/i5kMhn44he/CNdffz3Mzc3BO9/5Tujr64Pp6Wl47LHHYHp6Gr785S+f4Cd+5eKaa66Br33ta3DqqafCrl274KGHHoK/+qu/WiZqd/7558P27dvh3//7fw+e50FnZyfcfPPNcM899yw7586dO+Gmm26CL3/5y3DuueeCaZpw3nnnwXXXXQdf+tKX4Prrr4cDBw7Azp074Z577oHPfOYz8MY3vhFe+9rXtuWZ8vk8fPzjH4c//uM/huuuuw7e8573wOzsLHzqU5+CRCIBn/zkJ9n+sVgM/vqv/xrK5TKcf/75cO+998KnP/1p+LVf+zW47LLLVnXtL3zhC3DZZZfBq171Kvjd3/1d2LhxI5RKJdi7dy98//vfhzvvvLMtz3g8oX3kFd5H2hoKcIJx6623hrt27YrC9P7iL/4iitqi+MpXvhJeeOGFYTqdDpPJZLhly5bwuuuuCx988EG239133x2+6U1vCru6ukLHccLh4eHwTW96U/i9730v2mfp/NPT08flGU8GzM/Ph7/1W78V9vX1halUKrzsssvCn/3sZ+Hll18eXn755Wzf3bt3h6973evCXC4X9vb2hv/u3/278LbbblsW8TM3Nxe+853vDPP5fGgYBmvz2dnZ8Hd+53fCwcHB0LbtcMOGDeHHPvaxsF6vs2sBQPihD32I/W0pwuav/uqv2N+Xoo5oXwjDMPzbv/3bqA92dHSEb3nLW8KnnnqK7XP99deH6XQ6fPzxx8MrrrgiTCaTYVdXV/i7v/u7LNooDFcW8bP099/8zd8Mh4eHQ8dxwt7e3vCSSy4JP/3pTy+r/1cCtI+8svuIilAqFAqFoq04KXwsCoVCoXj5QAcWhUKhULQVOrAoFAqFoq3QgUWhUCgUbYUOLAqFQqFoK3RgUSgUCkVbsaIFkkEQwNjYGGSzWZUueYUiDEMolUowNDQEptn+3xPaR1750D6ieDGstI+saGAZGxuDdevWte3mFCcOhw8fXrZ6uR3QPnLyQPuI4sXwYn1kRQNLNpsFAICB3gSY5uIvjUQS5enlrw/bsKKyHNW8wMcNcdxCsRSVEybX4kqZFtsuN1ATzExxTaFEzInKVIEYACCX64jKhcI8szWrXMiOrhx1my6zAbl1y+b3FrPxmXPpBLMN9OSj8tjUFLNVmz7bzmZxX8/j61irlWJUHhrKMpvjYLPa1mLZ9Xy47Y5norZsN5bO++Wv3QLJ1GKdB0EQ2ZNCW80h6Q1Ci7efF2Ll2sDr1iRV5ATAIdb6hqQdXOPo64ANX9hC7D++y22+ydsIjvHDm649XrYOmRwXBOIaxCjvWp4nINuBL+6NHie2PXZvixVZq1bgP/zWVWveR/76ve+J+kOt2ozslsW/FcYIKpMvJPl7dHoO+9ORp7hO2D//ErcXGh6zWRZvMPrtcuL8Gp09qJ+VTfB72zKCgpGXXXQus/ku/1bMFjHFhk3eaQCA3fsOReWf/OyXzAak/8Ydfv2cjX00ZvN2b4rrex555pC/NHHy7tXCJrPN17GPmK1Ter4Pdzz06Iv2kRUNLEuVb5pGNLBYZMCQAwu1yYElpC+4OG7p3LIsz/mi+5IOKjurTQYBaZPXoC9jIKd9dGARNnpeW1zDIdeXNsvirz+9V/nRpNdwxMDGriFsa0VBLJ03mUpD6oUGFiEoGCMDS7BsYCH1JwYWq00DC90yjzGweC/3gYXUsf8SB5bo1ta6j8Ri+EODfAMtWwwspM80Evyjn07iwJIkPyQBAByLvmP82ZYNXuRZbUu+R/h5jIkPeyKO18yk+L15Lj9PzcXBzREDZII8I73e4g3Rd5xfP0Z+PMZs2V68tU04+sASs/A8nrA5NhlYZPd9kT6yKhFKx7Kij7jvYY8IfNExya/Thid+MdAPnbi5fDYVlXNiptEs8cRaQQ1H15STZLaOFG6nRENmSCecqfEZShDy7UQCG723l0taz8/jbCchrjE02BeVLdHIfX2Ys0F2sv2Hx9h2zMH6yed5fWTIZndHB7MZpCNVqq168+VXeG0QGIv/AABs8vI1A/7Rqyzg7NRJix8GtD1DbgvIs3lisPDr/JdafQGVbGMJPnj5gPVRrpWZzTRw30ya120IvB7pLEG+bPTu5IBAb10OLPQZ5XhEBxJ5Xjmw0PsJRD9kM53WOY81MLUThbGDUG99GG3SL+mHDABglLyPe2q8bXft2ByVgyZ/b/uJ/HxSHCc/urSOqkJ6f2EO3/GyweumUce+deY5FzKbW62z7ZlZPE9/gn+rgiYyD8m47AdYN33ZDLOdsfmUqDw9xRWfa7US2y6XSf82+SAct/H7PDTA+7obw+/Y3qcPLP7NW5nvTaPCFAqFQtFW6MCiUCgUirZidVSYbYLVosIMA8ck6uQCAKjUqniMz/lGj1BjhpjnDw7g1Gugl59z/97n2XaPjdO2gSGektgk0zVT0BM5Qj91d3AHVGgJSo1QTKl0itksE5+jt5/TZDR4oFRcYDYvxKl5R55PPYeFg57Qn2A73EadboFw+ueyuagcuovT6SYcH5qjVClH/L1LnIgz0zyV6pFRDFywEoLmy3ZG5bjJKSzKjDU9TnMELqddqyWkAJKOyBhqIs1QanLqoNnEi2zetJXZTtmygW0nqa9I0FRsW1DSIflDIAMLyKak0FYjRk5pHhMkpXh8qNEXwsFGDGL+4jtSreH7ETM4hQQ+vh+mwYM/Zg5ORuWHxo4w27NTSD2Fwnkv6coEaT/XE+8I8Z0mkrz/FGpYf798Yg+zDXbz97pBneeCiouTd9xxRCchTbR9yxZm2rge+yF1IQAATIwf4KdxsV4znTwtsk9o51ScU8JDPUi/HbYWr2GEvD6PBp2xKBQKhaKt0IFFoVAoFG2FDiwKhUKhaCtW5WPpyGaiOHAaYtvX18f2m5pFPj0h1i8szBeicn9PL7PF4+iPSSZ5WNzwOu5HoQsf3Sbn/WKAfGw8xq9frWGY4Lohft+hWBgRi+N5mk2+eKiH8Ki2yY9rNDA0Opvj/GetgdcvLfAFmo0G53i7e9AHlEzzprJJ+KPd5PxzvYLX8BqLfghf8sdrhF888EuItRaalSvI2ZrA27PWQK657nP/ixPDbSvgv318QkPXBd/rC19FOoZ9NGnw+kuQvuabvG0rFfTdPPj4I8w2NcNDwjdv2hSVe3q4ry2ZwrYP5VoVEtobyHUk9JlfQoLXkPh4QhkK/QLhxtJHtFaoWQb4rYWKc2RdkOHzcN9usq4jk+tktnoFfTOFEj+uSMLOQ7HuSIZUW2RfW/7OJmuYKiKkOUPq75eP8QWa2045hW2fumU9XiPGvwcbN6LvpBLwd2RyfDoqF0s1ZgPilzzv1buY6dEH7mbbNeLXLrn8+rMVrNeuGvdxDVvoe6yXF9vL9YQf6CjQGYtCoVAo2godWBQKhULRVqyKCuvq7orkQui0uVnnU6h+EjacEitN40Q2YbCXU2Gui2HKszNcRyub46HBNpFYCJpSioDKvXAqoVbFla4yBNRM8NDoRrNGynwqHCcUX7nIw1XTGZxuyqn3LFnNG3d4mK1USWiSa5bKYnU4uflmUWgFEV2zTIsyXBZKuUZYqNTBaVEIIYkNNkSYpU1CslOCprJMIlcBguYjYdOe+F1UqnJ1hloFt+MGb9tMiO1nibfAiWOfrZd5337+MF/lfHB8IirnczzMdB0R6esVIfn5TqQgbKGDZxFq7MXCi6kaTQBHp7ukbAtfeR+y/6814sY8xFptPphCmiYv6NKuTmyH/aF4x5L4PHFBgdL+5KY5Fe6KEPU6WW3vi/5EqcxYnN/bwDoM2x0a4cKaM6LPTBTxO3LhhRcw29wk9p+3v+NSZvunH/xLVL7v3vuZbf0Z50Tl1+ziWmXPj+5j2/t//kBUXmjy72jZw3rccf45zFZz8VvV07NIKzddThsfDTpjUSgUCkVboQOLQqFQKNoKHVgUCoVC0VasysdiQhBx+02SD8UX/gePhN826lVmo1LxxcIcsxmEPw+Fb2J0fJxtd2SQK0zZnIcvNjAUUXLUsQThXwXf6ornMIikQyB8FAHRb48L2W7qTqgKBeVYnPC2QmYkleAceZyEOy8UCsy2UMBnzCSEujHxY6VavH/TXZkUw0tFvRmA19KioHlhpEMr9ElIKPB2MEjdSrWTJpGncEXvzaa4AmypiH2v2OThmg3iI4yJXDHZGF7Usrit4vH2pOHQjRku31MooF8sneG+xsHBoai8ZdNmZsuQEPm4uDdX5NpwieskFCkGgmP4aujmkp/GD1f1OfiV4aTsSPZ9cxb9sZvE9TtIuDgscNmWVB7rqBLj35jAwf5z3lncb9Avlkbs27s3Kh8+xP1npoXvdehxv0mChDFffCG/xjS/Hfjl3T+Jys89t57ZfCJ/BWkeUl2oYF8ru3wOsHccQ/IrAW/3ilAgnirgeRoJ/o5s3YB9L98/xGzTZNnIa15zOgAsLtf4P7d9HV4MOmNRKBQKRVuhA4tCoVAo2opVzX0NCKOw0VgMD5XTbI/QHDQhDgBAZxJDbB2R+dEmSWjqTZHuV6QNbTYw7K1Z5GGmMUI7SJrDcMiKa0FrJEVoNE1HnM3lmY2qohoiCRANDXaF8rBB6K+EyIoHguZokFTJflNkkLNxSpvr6mI2l9BexcriVPt4hRvXmnWwW/2hQabvx1KVXZYlkWZXFFwY3a5UeAh2IimoRNrWLrfViQKCZ4jkWeQaMREKvPynGO4rs3XS85Sq/F4X9jwTlWdmZ5gtS6jNkWGeV7yzk9MlsTjts0LBmKy49sSiehqq7YeLfaMRyqRYa4NK0wG3RXt1WERBY4YrURwuIDV12ZmnMlutie/8sHi2RArr/SKRIO80kbCvSkKsZ4RKSJUoY/giytYmitgbDu1ntmSB085dvfmo7D7JlRwo3Xbf088w23NjqPJQF9+q0UNIDU7NTjPbBWdfxLY35DEc+r996xZma9Yw3PmhB3g/nJxERflzrlqsf1skQzsadMaiUCgUirZCBxaFQqFQtBU6sCgUCoWirVhduLFpgtkKwaVqrck0903UCWcdS3OO0ychdCCkPAb6+6OyNyuYd4+TnGkSktkocf66YwB9DtWqiP0j6OnnkjKNMr+GZSD/6YjQ4ASV/ajx68djaDNjPLxvgTy/6wqlVZ9zs3WivAoipJBmLrSFH6lOZBemZxb5V88/Psq1zTCMwlwNcs1l2RXNY6ikxtEWWvy3T0Ayd9qi97oipDhmYx1lkryOqk0MH/WA1zsRXoaGyOoZN4X8DAnxDcXvNDcgPg6RwdMkoewTc1y+aKyBYZ57Dx5itl7hIxgaQv48k+FyHQnilwyFr8gNiY+lFdrfENJMa4UeKw7xlm9hmNRfTsg2PTqPfoT5Bg/l3jCAkirvnNrEbA7xuXbv4WHK8ef5sgU/wHdso+iSDpHSNm3uD/XJt6Hxy4eZrUP4Q4Ie/Ab60tlF5JhyFv9WNIgkUZdw9aVC7OvFiYPMNrxjG9vOpvHeL9gyzGxTC/itmCjzb2W1istB9u1ZzJJZa67MD6czFoVCoVC0FTqwKBQKhaKtWBUVNj5TjBJ90RDjdINP7zIdOPWri3DbjIXTsuFBHjoZT+HU0+KRh9CZ4lRGPoXnyQ5weqBBFI13T/DETPl8Dver8IvUq5wScci9ukVBU5Gwu0Ao51okzLVc5qqsHmFrmj6nWXrzPAlPF0lutKfEFUu7SdipuDzkCDUZuIv0wvEKN/bDACBcTrv5gag/Ui+24LRoMi9bJOGiociOI8LVZXem9JsIW84QtQSxUBloni1XUHieiDs1Ddw5FDSHT+gv3xLULmkOKWBsEJrFc/k5i2O8zx4cPxCV4zFO16SIOq8Mbacr+h1n8XrNhkgmtUbYlklBslX/aRJqbYmEeduIOnRpkofU0o4wLNWNY+Q7UhXqHkLBmbZmwxQdgdDtjmgkm7S1Ywo1hKwIOyfLBrwGP49PQsT7RV9/DVma0TT4988fQrdB4sABZqvyXQEIxXj6qTwJ2WAVrzko1Dm2bcGV+Kf0LNJ0lVoNAL4HLwadsSgUCoWirdCBRaFQKBRthQ4sCoVCoWgrVuVjaXgBLFHFc3PIXaaqPEyxi4S7OuISiQzxv9BsjgBQpj4OEfpneSIktIS8ZW+Wh+k9twclFjIJ7rfIJNH/0BCccucgl0YxfMJ1V3kIIRFJhlKd+y/iJMxzYpL7eCDA62c68sxUr/FwP49IvCRFdstsGonUORFuXSfK09nMYt3I0Oa1QsNtRgGkVMZFZiekPjpPtEOtgfXgCOVoi/g04ja3hUKaxQixzmS4cxhgfcjEiVUS9t0Efpwpwnab5BmdUCg4E5+Ba/L6p5S9aQknmYHtJ2l/KX8TEIdQU4S9FyvkmlKTpIH7LrWTv8LsgC8V8xMHoNaSv2l4WGc1i9d1tQPf62RVZH58BuVGfIvXrZfGl9O0+DPFhR/MAHxXPdF+PukzoSP62lHKAAB2H1erzhawjepCxam5AX2lnR5vv3Qd790TMjHlKQy/ro79nNnGH3yMbedOx/Dj2Qnuq2qm8JvnCRdbdRb9eUVn8V6qKwxJ1xmLQqFQKNoKHVgUCoVC0VbowKJQKBSKtmJVPpbezkwkDe7VkQ/MZrjcSUjkVyybj11JIq0h4/erNSKFLxYXxBP8Vndsx3jsiYlJZmuQWPGeXi7bQiX9A+C8aSrD5WeaVeRYLSHJbhHOvDLH5SYWqrjdkcsxW7mK90blJAAA4oLHdYlfaXj9OmYLiBNqvsi5WepPyHctPr95nDJI1up1MFtSLjZ1EARHX2NSq/D2i5EMjl39XDY+Seh0U0jgWEK2JSTrCxbmZ5mtVkb/3oZN25mt5GI/mJ/nbRuPc5+dS/wShpBtCWgHF9VPbWI5E8RIRk3T4gd6rvQDkDo2+DsTNlASJCgcZrbZUbIuqiXvIv1Qa4W5ygLEW+vhDleItI5Y6xQzBqJyqpOvVZut4TqoAYt/f5J1IldTFKkopCRJD543vY2v8agTn0d5hvuD4wFZKyOk5BvTfO0axMmaszz3B9tkDU5Q5P6L5OnEVyOkoVJT6BCpjPLMl4Vn97Lt4BC+X9kuLpszl8c2n53g35HxKZTD2RRblNCpNVbmh9MZi0KhUCjaCh1YFAqFQtFWrIoKS8ctcFpU2I4t66O/J1OcHjAtPO3EYa4m6hHlz3Smj9kKZZwKWkLCwBDxx6UFnG5OT/HMZzwRI6eXyiS7YyAy5lWrPBNlmUxNcyk+hWwSuiI0BCVDKKBclh+XTGHdyIyD2SyPRbTMo4fL7j+E1IZh87qKkfDVUisU3D1OVJjv+1gfhOLpjHMF7Fwa+0wtJbqhQcLVyzwGMkEo0r4+3n/qSZFl1KPh2ryPWim8n5SgK/NpVM4d6BFKtaId6oTSqgrbxDRSEG6lwGwO6Xu2xykQK8Dnd11Oq9gWf46AhMsGQnkZCF1UHDvATI15vLdyefEZZSbYtUKhXodYiwqbIJk1XZEJlqqPh+t4W8c78b2KC7klewxDaptCsbcswsd9km3W2bCe2WySGTad5+dxd6PqtCvotbqQeMm++rSoXC3wbxU89yyWpbbQOO7bCArM5Ayg3MrA5TxjZDzJvytzuzE0O1/lto4NSCMeEi6FJJEhcpzFb4wrY/OPAp2xKBQKhaKt0IFFoVAoFG2FDiwKhUKhaCtW5WPJOBY4LUn4dApDMqXsRkceZQJElC7Mz2LY51PP7GY2j4ROxkV4XVeaS+yPkRC72RnOW9Y95J2LCyL0j8qci+jKQoFLklOFi6YIs0ulkKvs6u7glyDXaAi5epp5s1bn/oMQOJ/vkXDjhghp9IkkSTLFw6Qp7BY3KrMbrhm8Jiw5VzqIXyov/Cij48hR12I8XLRBwogNkR1vUzdy7X3reDa8Z8e4fE5IQkJTFV7XHSSr3hOHuQRGZgC5/kyc9+39u59m2z7pl/mtu/h5hjB8tXLwGWazSLhzLuRhntVyAcslnl0y5vD3oljHfpjM89D6bvLylUGE2ZL30ljyCYYhgL/20j/Dw0OQcBb7g7kf3+OkkBTxm/iuxA3eDvMVrL97D/MskUN1fOdPBX5SGW5cI9+R5sO8bWvESWgM875W34ah0FWP+712bTmNbVdMbLOa8HXFFki4dY77SpuHiB9nkvufnD7sF9V+7n9yuvj3qPOqc6JyQfi88z3Yf87JbGC2H9+D38N4q2/5KumiUCgUihMBHVgUCoVC0Vasigob6u+FeGzxEErFdOY5TWWRlIZOD7cN9HZH5TvuupvZggCPy2c5hzYxzqdg/Z1IZeQ7OD1QIKtSZ6YmmC3fiaGl6TSfenZ08rDTbBopvWwHn16mM0T5uMan2/v2In1jiVDgKqHUmk1OrzUbnIZYytYJAGCIMMlkAukjX9AELom3dhtL4cbHR93Y9F0wW003kMF2mZznlI5L2tcWIdkm6T+ey+nJDeecHpXnRZ00O0VIsUFUbnM8FLlQRLqkJCjJoFqIyo06D2XtEOc5TMLXK9N8df+GfD4qD23nNFnhaezPlVFO981P4naxws/pi5DUhRrWY7KTU2HZdbjtCSXxeg2p1SXF5uMUbQz9g31RBsnSKNLYqU7Bmxskg6PJbeMzWC9/+9hTzLa9G/vd/5fgNHFKqkVXsP3mnuBU2FwvvvP7GpyKahKabGjbELOt7+TfiuY4hvFmBBVlkNByKPFnjJsYCl0Uyuf+PlROCMf4N24+y6nl9HZUrxjatIXZ6iTEuFdQ6mefgVTuuk2L5yhXV5ZlVGcsCoVCoWgrdGBRKBQKRVuhA4tCoVAo2opV+VjCMICwFaMbJyHG1BcAAOBWkI+MWyKrnoPbTJkVAEwTz7lsxBNKwBs2bIrKUsF4ZBx507gIF811II9oiXubmuIqoZdceEFUHhjiPKoXIkdenOVZ2eZn0C8wW+DcrE1kEnp7OBcrsywGJPSzI8P9SPMkjDoU/HOzhvfmt6RcfO/4+Fg6s1mwWirNPRn0nRTmuFxEVwLbJe7w+/eI/EzfFq48vHkQVZ6fOrSP2fJx7s/ySLx430Ce2cwerM+KUOA2s3ie+WnOX2/o42rL1RheY97nbT03j/3CHORyISOnoQzH6JFnmY1mEnXk+yOkkC3yXjQK3I81DdhHvCrn6E3yzh6HCGOGBb8ATX/x02OHqB7t2Pxz1CTvSkGkN5yrkQykIT+u6KBvYtThfrd8yH1mTRO3w5CH9C8EWGdHpnjb5kz0tc1ztSK4dfRWtr2dhCpv6eI+uu44hi1XDvDvj1/Da4ZCyXue9C3ZJ5oJ7mNxF9CP1Xx8D7OliK+okeDfyg2noT/THVv0+3kabqxQKBSKEwEdWBQKhULRVqyKCjsyOhqpG2fSSCmVSnyaSCmJpljx69s43UqJMNNmjVAgvTxMOW7yqfCWzTi9jAsKxCRT4ZigwpJJQrcJCims8VX6DZJAy+3g1+8eRBrLFNP0DeuQLokneJhnkajcxmK8+m2Db3skbNgSSsg+CVu2REhlSBSkM62Q6WbTAwC++nstsK6/E5zYYnu8/ddeE/394L6NbL8SSRTXqPOwa6+B/WDjEKeQqHJB2DPAbAsuP0+FKOeO9PDVyR6RXShX+PQ+JFRCJhSh9AHnjfo7sK9VpjglWh5FKsVt8NDoNElgNnT6q5gtcJEemhp7ntmqZaEkQe4nl+Z9xCarzgVbBG4Vjwtby/CPl7pxLAwg1qp/m1B5PSZ/V5skyZkt2rZax/ocllT4JqRLR4U6toypjhH6x/AEFRfgezTYzRON2YSZKgq6NJzjtOPYLH4fF1L8W7W+QZK6zXAqDMj30BRh5jUPz1n1ed2EJqfbUiQkfXyUqxSkDLRVPE635cl72LNrGwAABJroS6FQKBQnAjqwKBQKhaKt0IFFoVAoFG3Fqnws1VoTnFZoZkDkUZsilLWrF6VQgoDzdvU6corr1q1jtqeffC4qOzb3fwwOcB61l/hgLIPz1w6hamNx/oipFPKPMtwYapyzrxXRPzI3zUM5QxN5+WSCn4deI5flnG6xOofn8Ln/KZngcYs0M6QrOOZcEsMofVFXOcLjOku0O6ff1wxZqw4xa7E9Lj4H/SMXnM7VYUtV5K/dUISreySUVEhI1Op43KYmP2dVSOKUiaKx4/B+ME/aNrGJ8941oiQd5jm3PjrBJTn27EcF2tM6uR/n0DS2NQTCR5ZA/2JmwznM9qotG6Py3GHuY3nu4YfY9tQEvjNpg8vfAJEhqfv8+gbJdmk7S5IuITREn1wLJOspSLbCjcc89FX2mdzX1Vkr4D1OiUy0JXzWHadtYrb127dG5bnHnmO2QUO8CA7Jkij6YbKM9WcDf49TJAPp7ucPMFtPhZ9n80b8Hh6J8fqd3IvPlSzNMZtB3gNDtF+d+J+aJr9es8K/uXM++uVSKS5bVWpiX680+DPOjeISAXv94rex2lxZ/9AZi0KhUCjaCh1YFAqFQtFW6MCiUCgUirZiVT4W07LBtBa5Prr2IC6k4RuEt4snhFyGi9yu3+T8eWm+EJWrZb7+Y9N6LvecjKNfIZPi62E6OpH/dD2xjobEfEspmp4efp4pIr8/Ps35z4eefDwqn3IKX2sxNY33PjbO1zZ4JEtkPsev5wgZ+HgcfTWeWMfSINIKgXAVpbryUbnYknX3zeOzRqEyX4Bmy8l1ZP+T0d9HhjkPPjzYH5Vt0X4BWc9TFNlBaZbP7q5uZqvUeFtXa9jWlTJfa1UqI7e/fctmfh4iSVQXKRF6k1wuwyHrEM698BJmm6ui7cDEArM1yVoDvyZkMoj8/dAuXm+9u65m29488uBzz/yC2fY/+UBUnnmeZ2s1Y/iMpr3Y78IwBFghh/5SsFBxodnybfxkAf0BHm9OuJRIyidF+ouEi2tFzj73Ncw2tA7l3r//yyf4tRu8rn2bpJgQ/pdkiC9W/Qi/vtWFfpPNndwPV/d5W9skPceuyy5gtjmiIjP3EPfjNsiarcDm/a5G7i2dFhWX5OvaajF8rqCbr8uqE+frhPjGLRTw3Zt/dlEKRmbEPRp0xqJQKBSKtkIHFoVCoVC0Fauiwvq7+yHWCtuMOzgmpYSkSjKF0zRPyA04ZHqXS/CwuC3DSI/kUzz0dqgvz7YzcZzC5dJcwqBOMq/FAn5vRTL1TqT5NZwUl5SYmEZJkMNCpuG5vUhBTEzx6XVxgUjBuGVmO23HID6DUBP1q1xdlYaoSrmNBFGXlsrFhoXN6rVUUT2hjrpW6EikINaSdCnNIn0wHnCar2cA+0iHxbthOpsnJ+Q0mWUgdZEVqrIdGb5vaGLbeyJc+5mnUVG4V0iCpFJIbVYFhXbmRh7ifPl5GCpc83gbVUmVb13H22hyFim2sQlOQUzsPxyVDwnl2rqgDZN5lIbJn/EGZjtr+8VReXj/48z2+L3/FJWnJ/YDALSUy4VkzBrALY1HEkV7Z/E9qrn8Xc2PIMV0psMpuizRVNkkli3kMkhTNcT3p1Hl2zEH26UeChvpP7Emf39qc9hmplBlDizeZpPkPZh/hmepTCXwHS8luIJ5iSwpaIi+TenaVE8Xs801+feoRL4Ppsup3fEJ/D6ZQhqqSN6ZdHGR3muuUApbZywKhUKhaCt0YFEoFApFW6EDi0KhUCjaitVlkDRNCFvyAQnC/zkiA58Tx+16ifsNXJdkRcxyeYGzzkJONelwntJxOP9qkxBnX/D3QKQh4kKaPpNB30QsLmTzA76vQ6QSnn6WS0NUSCgpiMyBDRKCGrO4H8U0MWwwNPj1A5Pzl0US6lqqct7UtkhqAsH/eiSkstmSJ2m6xydN4EBnR5TGwCChq3OTPJTyscf3RuVHnuR12z+MnPmrLn81sw33YphwfZ77vSxbOF0IR24LHnz9EIZdJoWvKx7Dds/FeAZCyPJ+6Pp4npIId6752L7P7DnAbPMNDEM/ZzP38ZT78F73j/Mw12cO8myTj+3DeizF88zWk8N7P62f+4bOezWGLT9y348BAMD3PSgt8PDutcBr1qUh0wqBnZ5Dv8ID+3l7/vgAhu0mN3P+P5XB9yhr8TZySySDqsH7fUWEGyeIf88Xyw/AwO1AyKbMVdA3Edb5+xcTaRjcAvoqwucPMVuK/LZvCrmVJ0j6iwMz/P1JkE9eLOB+EyfB+7rhkrDpAvfnVUL03dgZ4fMlmV03dOYXj9dwY4VCoVCcCOjAolAoFIq2YlVUWNPFKV+pgtNWM8unorUChizKle+pJE69LJPTCoVZnPo2BBW2IDLBUQoibPCpKFVGdky+mrbqE2pOzOqaNU7bpYgy8oRQtW2EGOLcsPgzxghNZyXE9UnmPq/JwxvjMV4fC3V85olZrlwbUrnikFNqBpn+J1vPYB2fhffw5OMPgdOincLZg9HfO7o53fPQU0jpPCtookuvvCoqf+Obf8ds1151WVTuTIgQ7CQPybQd7Je1OqdZertRiTiIc5plviHCvgkMQZe45LeZ4fCw970HMVvff/0v/5XZZqaQkrjwosuY7Zpf/0BU7hOq3mmRrXTIw7Z/qsAp4cDE92Lq0EFm27oeQ/s3bz8NABZDsp9/mqsnrwVOGbQh11ou8JsktHtdnGdQvPM5pJvuOMDfsbM2DEXl8vP7ma1A2sQSNHmhKfoBCd/2Q/6uuiS75XTIzzOTQgqvbvPvT1Zkgk2TkPlA0NYwiyodcdEPj5D3f1aEnQ8QCfdUmocpZ9P8PCFRdpgRaie2hfVhiSUVZ4T4PcqUFuvCUipMoVAoFCcCOrAoFAqFoq3QgUWhUCgUbcWqfCyzhQVwWlIMQ32oqEn9LQAAXoCcXlc3lxsoFXFfz+PHNYjPIRA+gWf3ch7VJFkjY4L3Xr8R+Vczw1VB6xXkCH3h4/AE/xgn5y3Mc8XS3aPIWW/qHWS2riyGxNpdPISwUkHedt4TKqgiNLpEuNF5oYAbkGx3hmhGx0Aet1I9vuHGMws1sFsK2M86GFJrTc2y/Q6No8/q1VddwWx//Kd/EpW/+N//htlu+/6tUfnUYa7q6sQ4R54m4ey+kKLo6sB+2dvVz2w0NDkm/F6m4M/LRCqnKcLuv/w/vhqVn36Wq+zGSfj8zbd+j9lGtu+Myju3bmO2ZJz7cXIhXn+IU+3gkfup+CK0niiQbxhe9HM0j+FbaicazSo0WkrCXST76sXbuErwTAXf8YdG+bvyzCT6HLfW+XvbJO9RGPA2KdX5M4YNkm1VhOmG9CMkPki0HUqhkHRaz/tT9+mnRmVLrIx44l/ujsrrxL2NEJVraPBvVcLGEy0ImZbKLP+uDhB/0FAPf2diJj6zM8freEMJfVzr8nkAAKh6vB8dDTpjUSgUCkVboQOLQqFQKNqKVVFhoxMTYJmLUyHHQdpBUkjr1g1E5YpQ7C2WKRXGp5cWCQ2uekKNdu8+fuNk37HDPBS4pwtDkTs68sy2Zw+uVA6BX//Nb7qYbcdDpFI680JVtoiU1myhwGxBE6eptJ4AAIplDIGtNPiK/aqoRzOGNF7d5XNoqmAciJDK+TJOaXtaEsB+uLIp7EvF0PrN4LRCIX2ilOu6nC6IkRDJwXV8VXhoYLusGxphttv/3z9G5dIET1qUEkm44km6Ep8/f9zGcM1MinNIKaIqEROKD4kYX90fJvCa0zWuDPwUUbJ97WuvYrYzzzozKv/vv/0qs93303+OypsH8swWS/H+NDOBK/Mf28OTeTlEvbs/x8/j10hIektpIDAET7NGMCwbjBZdapDV5YN5TvNdsgkp5aJQ7D1QwO9I1eJt20fUji2hnFAX35x6CdvMFnRxzMH66wAObxJp3pxQDm8UORU1R97dfCfvs3myut8RIfHDJGw4JuYARhr7neHw8GKzzL+d/TbWQUJ8BswGPnO1xPtvBwlF3rJ+sW3KMlz6KNAZi0KhUCjaCh1YFAqFQtFW6MCiUCgUirZiVT4WLwxhKZHh7ALy+LkU50apH8WS2dWIFEmlxjlFKiAaCsXObJJzy1NEfuDRJ7hcRTqJ/GejzqUgAEiYspBbeWYPP09/CsMfs2mu/DkwgLbZg1yB1iCSMlPT08w2MoLhfn7ACc+GzEBYQc7TE/v6pH6yOe4jaJLQyErL3+N6x4c/98AHo/V7xSf3EYtzrjtNorBpfwEAmJzCOpuZ41I2RyYwbDkUckGJOPd/UCVtqWgTd7BfpuO8bZeyGwIAJBO8bycS/DkCwu8fmp5kNiq189a3vY2ZLrnkkqh8+PARZrv51u9H5Uce28Bsfp3z5/OT+B42Z7kkiu2jX7Dq8Uym++YxS+VSBljPle/K2iAMDQhbdRMG2EaxgPtjT+vCNpoe5H28QkKjPRGK30PkgxIZ7h0piLBhlyhwe03+/A0Lz2sa/FuRI98q3kMAmkUetgt1PE84wVWKR4jvz7GENEwNz9Nn8b49T3xM8Sz32wQuny941UJULjbEEg/iVgqEz3fwNJQ92rR+sU6Ly76nLwydsSgUCoWirdCBRaFQKBRtxaqosHxXV7SqOpfDELeEw08zV0QKJ5kUSXiaOPdqCqVM28FxLhbnYZ5Nn0/BpubwGnWPj49d2XxUHtnMV/O6RKG5WCow24EjnLaK9SJFYoZ8mppJ4f0ZfXwqmksiz1MuFJntwMEDUXnLtvXM1hQhwU2fTPEFk0VpsvVidX8ygffWqC1SJ354fFbezy7MRSvXXQ/v3xaJkkLS9o88/iSz7TzzXGLjK9apmnBTJPZqupyuGB/HpFV1keApRihaERHOApOdGKfJHNHXfaJ6WxYrwLt6cAV2Tzdf8VwqYr8YGBxgtrl57Ic/+tE/MVu9zOmK2VmkuCoGr2ObhF9bom919iNd1Ne/eH3fW1ko6UtFYJgQtO7VpyrdgtrsIJTy2ev4ezxbQnXo5iRfbuBWsI5iad5H6qKOXKJgYQb8+j6hUg2hXOCR8zQdGcrP69Egfd23+HcNTDxW1n9IKLSEz/th6CIlOpEoMJsrvp0BicJ3BKVfreJ5YkLBuXc99stES7G9aa9s2YLOWBQKhULRVujAolAoFIq2QgcWhUKhULQVq/KxlKs1sFqKvwHhI4f6+9h+MeJXqQpVznQK/QGGzXl/g6Q5dGJCwkT4UapEkiKW5AF/mW4MTXRNzlt6JNtbIi9CR23OP5ZIGOzWzTzs05tAbturcG59oYz879ZTtjLbkcN78N6Ej0mqFJeJNEQgfgNkUilS5pxqhahNW60MeYF7fPhz3wjAaEmDGIRPLld5mGOtjPU3Mc2Vjz//xf8elQ/u5SHgZeKj2zvKfWKhCCWlisauL/oTySRqSbkM4mUxaryNQkPw58woFHDTeI3ZWf6MNFtocYH74RokI+qBAzwU2RB9hir9hCIUmt6NlKZJx/EdqbYUv6UC9FohlkxBbCmzKbnnZoGHRFMfx5B4V3cuoP/hmQIP854YOxSVizVet2Uhf1Qnvj9H9B+P+CXNkL+bFQNbvir8V7boT0EjIGXu6zOIj0UqKNfJ9zEQ/pcK2bceF6rUJm/HhINOlsAX32MS4n1KP5et6ozhNaqzhcX/N1TSRaFQKBQnADqwKBQKhaKt0IFFoVAoFG3FqnwsyVQS7JbchU9k7RtCCsImCwMcwe1aFl00wMc1k7g4bOfYEiQN4uMxbL4QIdWB1ywJKegkkVKfnp5jNtsWHGMS7y+V52tFMgn0q/T3ctmImRBlSFIp7rfpo5k3i5z/bQqKm9KvOSH/n83hcxQXCvz6M7h+IzQXuXTPOz78eWdXZySbD2SNQk2sv2gQ2XxTrC0ozBeicncv9991dOH6C09w0kEoMoK6yB/LNQJU7iVwj+6baQgfYSD8KEBi/03RnwukfX9+78+Z7corr4zKTz39jLg+lpviGS3gfT0gdSf9SH6DvJdNfp7DB1HSxYov9nsqr7KmMCyAVtoLw8D3QyxLgrqJ9+/E+P2vH0Sfy/4jvI2aRJrED7itINJxzJCMoFmL161B2towuB9lgVT1hHhxZX+Wa4jYvqTsiLadJN+4BeDXKJPrD5v8/Hkh/2+RNX/9NvdHn0tSnGxZxxsgVUOfV6Plm2mqbL5CoVAoTgR0YFEoFApFW7EqKiyRjEVUmGkg3VRr8nC3eEDUYYW8gEHkDmJSS4MoxeY6upipLhRDmzZOae04pwBqJNucJSQUCDsCzRqfXo/XZ9h21zBmNnTHuSppkmQ5TGT5c/R2IH0zM3uI2bo6CKVmcpqs7PF63D44FJWDkF+jWsVpcrXCqcguQpstRRl73vHJIOlDAGZLf4ZmtrRFP4gTtWNbKGB3dhL5DkHhBYQaMgV14TV5SDMNrfQFTUTvTbJbHgnNLld4CGyjwdvIJdIavrhXuu8PbruN2Z58GrNLPvjQw8xmkH7hi8yXnrhZKikTCgXrgGQ2lASGSTKwJsLF/hOGx6af24bQBAgWf9M2iMK5pIxoKG4olIczJLtiT47TW3PT+K6WhJrwgsV/S99L6KZO0Q9yhKZLCyrMNXHnosxKCXIZAcIS0kYx0odTIN9RtNkiu2eKXF8uJWgK+ZkkuZ+OjOgJLtK15Xl+jWIOn99oye2U3JXRpTpjUSgUCkVboQOLQqFQKNoKHVgUCoVC0VasyscSs0ywWxxlikiKSCkIi3B6lsX5Pp/I33si9C8k/GepxPnzmgjNpddIJPhjNAnn6NY4p1hdQN47JuIbs115tg0xlEJwq1y2xSLhj1LiPyTS6jQsGAAgTkKj8yR0FgAgLPLwZ4NIM9RLPFy3ViXPn+JyFyw0ssXJu8dJ0sUwLDBa2fYckgbBEP0ACA+M4cktEMo6FNx2nPpVhC0merNBcvt54vl9Ku0h/BbUd9Pdw319sh6pX2K5HwfbiMrsAABMTKIMycaNm5itRHxm1RrvdzIXJvW5+MJHEpJnlP4ok3D9ZsuXEQQB1Eo8Y+dawA/CKLsoleExhP8jZuN7FdZE5kJSDX1p/v49/ASmYZgd47I/nsE7yTTxaxTF9yhF2jMlum+c3GsY49c3hR+Fvo+2kI2ibVYUqUHoEgHp/4rRS4g+GYh6NG3ijwF+jUK5EJUtkRokbuLyCyNYrLey+lgUCoVCcSKgA4tCoVAo2opVUWEpJw5OK0TYJlNIOTolEkhBlMs8XJOuvI/F48yWTKeObhMXqZHV5v19PBMjDffLp/lKU6eXTK9FdKULPJTUI+GayUya2RyqKCymyS6Z+vb0ZpgtFmCVW2JaHI/zew1DvJ9Uip8nSa8vaI4aoU+Wyu4Kp7AvFWFoQdgKjQ4DohIsKomyWIFQnGXUmFBVoLSCKagwua91DOVal6hFLFP1pUyiXPlu8DajfUQ0Azjk+kmS1RQAYHg9tl8grlEjK7kl9SbritJHoaD06L6WuDmuLrDYzzzPg/HDXE16LWDaDpgtutght2yIcF/DIp8n0UY+CQMfzHIquNvBfR2R1TMX8D5DM0rKFfOejfVXEfXOVioICssS4ce075uS/idtFoqQYpbJ1ODt55C6SYr7zohvZdog9bHsM0D6QY3T7TTSPmUu1nHTXVlIus5YFAqFQtFW6MCiUCgUirZCBxaFQqFQtBWr8rE4EILT4gRNwi3HLH4aximK0DvK+8ZEmKlHFGiDgHPLCXGejixVx+X3mYgh5xoI5dFUBm2uUK6t13hIaIOE+6VELKtDQpErIjtiIouyLTWhBloj13RC/vyWKUJCLfS5+OInQLWG9Vgo8BBRWo+xViikIQnsNYJb9yFs3Sz1h4gISOZ/kH4Di0i8SOXqEGjopOCyBddsEn+Ik+R1HVrIi8flzfGz8uOEH4PWtduUSsjBC+4HAFBt0jBl3kfrHlHuln4kEbYdkmNDUY8xEgYrZXMolpYOeEKlfK1g2haYrfuxQlL30unJfCy8/Wzy0mcMXu+vPh2lkBaq3PbIIS7bNEMyItaFr6tB2j4Q3zia0dUXx5mG7JfEZh79PbRE/yVRwpA0+fVTRPYna/M+kTV5PXaTQ1OiPzlUYkvcW0i+8fWWr6ruqY9FoVAoFCcAOrAoFAqFoq1YnbqxY0OsFSbosym4WHlv4TQtl+MJsijtIaf5lNIJBRXWkeQr2DOEmgoDEW7bwPsxZDIoF6fC2TQP4ZUqt/SpKkLB2XHxGWs1EaZsYojjzAJPNFaeRQWBfL6H2WYrnNJKJGkoKW+q+Tmk30qCiqPJzJbKxyvRVxgaEEYqtdi+UvkXSAhkXISW81BgTs84Max3SaHZIFY1k1BdEQHKKC1JqZmEZpF91BCUrBPHvmeJpHb0WEl30Xt3Pf6MJun7gTjOkyoXRBE48KQqQPiCZYklulo+25ohlgCI2hHv2ZD3SOg7T9RRQD5dlLIBACA5wOCaM4eZrV8kENw7ie/jpFAJnyeK4HXxjWmQW/UMUe+SkiWh3jLsm4UUi28ViXaGtKDi4uQacRGmnLN4H+kkVFla0L4JohIiWGf2HlZb72tNqTCFQqFQnAjowKJQKBSKtmJFVFgYCRniFIsK7slJNk08I9M9UwpARoy53tHptaZYOd6kK7BFNEOTRJpIKswgN9QQyYOaUqiRTLFNkbynQaK95L0Fx7DRZ5TXcwVdZLmUyhAJn7wXbgtpWyov/f9YlMhLwdJ5vaNENBkhf7aQRM4sE+0j2zIajiKQoouyJ5JV1jJiitaDvD6lwkDQGsuoMaYgIKOBCE11LCpM0H0u6fvyuGWUGukX4SqoMJ/su1TfS2231n2kVCdUZ/NYVBhuew3+bD6N9hR9KyD0WllEhUoqp0HarCnVGSjdJaMBj2GTtWfSdpB9hJblecimu6xvhS9YBuDPBABQJ4/s+OLuyLczFO+aT867lESu2jr+xfqIEa6gFx05cgTWrVv3YrspXgE4fPgwjIyMtP282kdOHmgfUbwYXqyPrGhgCYIAxsbGIJvNLo+rV7wiEIYhlEolGBoaWvYLvR3QPvLKh/YRxYthpX1kRQOLQqFQKBQrxUnnvL/xxhvBMAyYmZk55n5XXHEFXHHFFW25luLlh5X2A4VitbjiiivgjDPOeNH9Dhw4AIZhwNe+9rW1v6mXGVa1juVkwt/8zd+c6FtQKBQnMQYHB+G+++6DLVu2nOhbOe74VzuwnHbaaS+6j+/74HnesgV8CgXAYq6bpFi4q1AsIR6Pw0UXXXSib+OE4KSjwpZw+PBhePvb3w65XA46Ojrg/e9/P0xPY/5rSYUtTVv/8i//Ej796U/Dpk2bIB6Pw1133QUAALfddhucddZZEI/HYdOmTfC5z33ueD+S4lfA5OQkvOc974GOjg7o7++H3/zN34SFhYXIXq/X4WMf+xhs2rQJYrEYDA8Pw4c+9CEoFArsPBs3boRrrrkGbrrpJjj77LMhkUjApz71KQAA+N73vgcXXnghdHR0QCqVgs2bN8Nv/uZvsuOLxSL8+3//79l1/uAP/gAqFZ5cSXHiMT09Db/9278N69atg3g8Dr29vXDppZfC7bffzvZ74IEH4FWvelXU5n/xF3/BwshfiApbomgfeeSRY36fXuk4aWcsb3vb2+Bd73oX/M7v/A489dRT8PGPfxyefvpp+MUvfsEzFAr8t//232Dbtm3wuc99DnK5HGzduhXuuOMOeMtb3gIXX3wxfPvb3wbf9+Ev//IvYXJy8jg+keJXwTve8Q5497vfDb/1W78FTzzxBHzsYx8DAICvfOUrEIYhvPWtb4U77rgDPvaxj8GrXvUqePzxx+GTn/wk3HfffXDfffex2erDDz8MzzzzDPzpn/4pbNq0CdLpNNx3333w7ne/G9797nfDjTfeCIlEAg4ePAh33nlndFy1WoXLL78cjhw5An/8x38Mu3btgqeeego+8YlPwBNPPAG33367+upeRvjABz4ADz/8MPzZn/0ZbNu2DQqFAjz88MMwOzsb7TMxMQHve9/74CMf+Qh88pOfhJtvvhk+9rGPwdDQEFx33XUveo1f9fv0ikF4kuGTn/xkCADhhz/8Yfb3b37zmyEAhN/4xjfCMAzDyy+/PLz88ssj+/79+0MACLds2RI2m0127IUXXhgODQ2FtVot+luxWAy7urrCk7AKTwos9YO//Mu/ZH//vd/7vTCRSIRBEIQ//OEPX3Cf73znOyEAhP/rf/2v6G8bNmwILcsKn3vuObbv5z73uRAAwkKhcNR7+fM///PQNM3wgQceYH//h3/4hxAAwn/6p3/6VR9TsQbIZDLhH/zBHxzVfvnll4cAEP7iF79gfz/ttNPC17/+9dH20jflq1/9avS3lX6fXuk4aamw973vfWz7Xe96F9i2HVFbR8Ob3/xm9ouhUqnAAw88AG9/+9shkcD8KNlsFq699tr23rSi7Xjzm9/Mtnft2gX1eh2mpqaiWcUNN9zA9vn1X/91SKfTcMcddyw7dtu2bexv559/PgAs9q/vfve7MDo6uuwefvCDH8AZZ5wBZ511FnieF/17/etfD4ZhwE9+8pOX+JSKduKCCy6Ar33ta/DpT38a7r//fibGuISBgQG44IIL2N927doFBw8eXNE1ftXv0ysFJ+3AMjAwwLZt24bu7m42nX0hDA4Osu35+XkIgmDZ+V7oGoqXH7q7u9n2ErVVq9VgdnYWbNuG3t5eto9hGDAwMLCsr8i+AQDw6le/Gm655RbwPA+uu+46GBkZgTPOOAP+/u//PtpncnISHn/8cXAch/3LZrMQhqGGRL/M8J3vfAeuv/56+Nu//Vu4+OKLoaurC6677jqYmJiI9pH9CmCxb9VqtWV/fyH8qt+nVwpOWh/LxMQEDA+jZLbneTA7O/uCHYJCct2dnZ1gGAbrVPQailcuuru7wfM8mJ6eZoNLGIYwMTERzUaWcDQ/yFve8hZ4y1veAo1GA+6//3748z//c3jve98LGzduhIsvvhh6enogmUzCV77ylRc8vqen5wX/rjgx6Onpgc9//vPw+c9/Hg4dOgS33nor/NEf/RFMTU3BD3/4w7Zc41f9Pr1ScNLOWL75zW+y7e9+97vged6qF0Wm02m44IIL4KabboJ6vR79vVQqwfe///123KriBOGqq64CAIBvfOMb7O//+I//CJVKJbKvFPF4HC6//HL47Gc/CwAAjzzyCAAAXHPNNfD8889Dd3c3nHfeecv+bdy48aU/jGJNsH79evj93/99uPrqq+Hhhx9u23nb9X16ueKknbHcdNNNYNs2XH311VHUxZlnngnvete7Vn2u//yf/zO84Q1vgKuvvho+8pGPgO/78NnPfhbS6TTMzc2twd0rjgeuvvpqeP3rXw8f/ehHoVgswqWXXhpFhZ199tnwgQ984EXP8YlPfAKOHDkCV111FYyMjEChUIAvfOEL4DgOXH755QAA8Ad/8Afwj//4j/DqV78aPvzhD8OuXbsgCAI4dOgQ/OhHP4KPfOQjcOGFF6714ypWgIWFBbjyyivhve99L5x66qmQzWbhgQcegB/+8Ifw9re/vW3Xaef36eWIk3pgufHGG+HLX/4yGIYB1157LXz+85+HWCz24gcLXH311XDLLbfAn/7pn8K73/1uGBgYgN/7vd+DWq0WrWVQvPJgGAbccsstcOONN8JXv/pV+LM/+zPo6emBD3zgA/CZz3xmRQtjL7zwQnjwwQfhox/9KExPT0M+n4fzzjsP7rzzTjj99NMBYHHW+7Of/Qz+4i/+Av7X//pfsH//fkgmk7B+/Xp47WtfqzOWlxESiQRceOGF8Hd/93dw4MABcF0X1q9fDx/96EfhP/7H/9i267Tz+/RyhIpQKhQKxXHCjTfeCJ/61Kdgenr6pPatnbQ+FoVCoVCcGOjAolAoFIq2QqkwhUKhULQVOmNRKBQKRVuhA4tCoVAo2godWBQKhULRVqxoHUsQBDA2NgbZbFblvV+hCMMQSqUSDA0NgWm2//eE9pFXPrSPKF4MK+0jKxpYxsbGYN26dW27OcWJw+HDh2FkZKTt59U+cvJA+4jixfBifWRFA0s2mwUAgC/88xshmV6UlP/FT6cieybOpcRTqWxUdgx+iXQKJem7c1zhM59CUbZ8LsdsE7NH2PaBmSfw/oZ4Fr6uQdx24lxttFbB7IGJBE+oYxl5th34XlT2/TK/19xQVI7HeHpaC3DfYqnJbHNTVlRuVDqYrdpIs+0QMGCvMM8FL2s1PG+pvMBsIfjkuMV7cRs+3PxfHo7ast1YOu/Ahs3RLxkzxPq1khbbf3grtr388Xpo/3hUDgLefzK5DCknuC3Gf0H1D/RH5YUyb7+5hUJU7uziwn9uAftMeYpL9uSzGbbdvw77QcWrM1uRyP2Uy1Vms8ir5zZ8ZiuWilE5mefP6JI+CQBM0t0P+XnCALdjNq/HJEkB0Wwu9iXf9+HZh55Z8z7y0W/eC/HUYj365B59EaBK386Y6CSGhSvUmwG3lV1sB0v+qK7zdsgmUV0hm+FKCx6p6rLL+69J7scFXu9BKO41bP/sjAbzhhBII98Eun2MezlWfHDreRvVMvzX6y550T6yooFladqaTDuQyiw2dyyBh8YTXIYgQRpLDixJMrCk0vyDnE6lsJzhH9lUne+bqOI1kmmeLyGVxW0nzl9Ew8T7WT6w8O3Ax0bwfTFAZvGZpfSHDfjR90Vr1at4HhN4vYU2Pw/tEPUG3zcgtobP7zskncep8fteKwpi6bymaYJpLr6EZogvo2nxF9N2bHIsPxfb1+DHWeQDSc+xuM33pfIYTswR+9pHtYUO9hlbfJAdm+9Lr9E0+Qtuk7w+8jx0YAl9XgEWeX5LHBcYvD8FIbmm/L6Q01q2rEdyjYDb1rqPxFMZSKQXP0xsYAn4A7CBRdAudGAxxcDiNrHOlg0s4jyJFP2O8EGcDizeMQYW62U+sAThSx9YZJ94sT6yKq0wK774DwAg3YO/AB9/6F6237qBc6JyVgwe9SY2UK3En6SWx5v1DP7LonOI3+rWdbhdS/AUwaWgEJWDIv8gx30csMI4v77r82vaFv467cpx+YVUDI91K3z0LlYwb0dptshsh3ZjIiArLjqEwwfII6M4S8lm+HOUS9iZPU/qC+G9Lb2rwXFarRS6IYTm4sXoR6Pm82ebGJ+Pyn09/EdEwsaX3zR4/3HIR7AxL/pIb4ptj/TjTCSd5P2nWiQzkQafzezYgTPngUtOZbZMkg/+cfIrtxHw2WmjgVRBsVBiNvqDa3qM5zrffxD7RayLz9ytBP/A+QZeMylmcIk49otsgtexQwasoNU5GvUmPPXLJ2GtEVoOhNbisBHQD50YBGoN/LLXxeAbIx3aMLnNJj8ejYD/sJQXoYNApc5nnJaB9WeY/AcF9S+Y8mMtXmvjWB/zFUK+vvQpLPH8phjoXBe3XXFv7BrHus2lgcRYme9No8IUCoVC0VbowKJQKBSKtmJVVNj49BwkWj6CoU2d0d8ti1NBXZnNZItTIKP790Xl/aPjzDY8hNRGJeTn7LTn2baXezYqmxmezrPh4rS1VOBT4S4b6ZJYjE8wcx3cMZtNIpXREHmvmx6huDw+v1yYxGyE8/t4Fe9+8NGonF7H7234lD62nUjjc1CHLgBAo06OFb6hmVmkVpotR6Z0EK8V4jE78rGEzEclJvMeUjp9nZxmrM9hP6iVeR0lLKTGUilOfe3Yfgrb3rptY1ReKAsqKkF+U5n83k7bicdt2jjEbM0GDxQJTbw/k7NUzMcSNAU9UUEKq1nhQSwX1XdEZcPh9JaZElRYzCU2fn3TwfqPiT5CfQRLfH21XIf//glYc7heAFbrnQlJv5BMjEkq1BXvWBCQ55ZEEXWs+Py4WEw46C3crrq8ryUdQnfZ/DzUjwnCN7RcJcs4ShlW5DBfvEQgTGgzBT0lr099tccS8DqWuteSLfQltfjC0BmLQqFQKNoKHVgUCoVC0VbowKJQKBSKtmJVPpa9e8sQay1027gZ/Qibtq9n++3bszcqV6o8lDOdRSK4VOML+558Dhc9Zoa2Mlt3lodyemTNwJF93McCIV6jM8Y58hCQI0zEepmtq6OfbZcXMNzw2Wc4t9iZRl48m+Pjs9uN3HBllPPnE5P5qLxphPPlqQw/jxfgczTrvB5tshhwfo77D6oVDJtcWgbiHx8XC6Q67Ggdhh3gPWZ97itIxnHb4E0LKRtt9Tr3LVXLM1E5TPH6mhrj13iEhI/Xmw1m6+5Df9bgCG+jwSH0+cgFijKwm0T0QiLG25P6D9wKvz4k8cCGWNgZNrBvm2L9FMQ5R5/sw0W2XpJz5A1SsaFc/0I4+2gtzBrIuLwgwhA5+xVm7TCMY/gxxBopapPrLdwGXzAdI2vOYjZva+6V4nBJTPEyj8qxwnZXtfPRQdvPFXUozxiEtF2PHm98rLUpIe60ovvTGYtCoVAo2godWBQKhULRVqyKCjtyxAenFZ0XAk4pi92H2X5NEyku3+ZhuvnOrqi8dfsmZpucwuMqLl8F+/hTnO7yTOR28j2cNoMQqSEnzs/T2YXXz6R4mGupyKd5M5NIXwRNXlWJHIZDF5udzPZEHcOtG0KHyuzDlfepBH+m+QLXpRofw+fwGkIjqoHPVa5wusjzKN232GCBdXzUZNef2gdObLGu4nWcdntCZWF0tBCVn3uc14MZYl03inx1veFhvzMbfFq//0FOrR6K4Xm8kO/b049U2LygwtLBrqjcl9vBbAODfN8UUW+IC7qpWSKaY03efs0iUjDlA3zlfXEKQ+ubJd5/ayJ8v2cbijqanUL2qA/D5428kG0hq7WdVlivc5yoMBfCKETYCI8ebky3TUlpkdBgy5LPhs/hi1XoUuIlRUKyhUgIeFXsew0Ry90AEVt+lPsGAAhZ3zv6cb8qlocXH9v+q8EQ/z82dMaiUCgUirZCBxaFQqFQtBU6sCgUCoWirViVj8VvOGC0OMLCFHLEbpXLrcTTyOl1DnQxW0hk7PtO4RIqxQBDass1HoOaBH6e2VnknrMxntdkaCSP9wZTzLYQ4HGVuRlmS1j8PGUSmZjNCSmIGD7zVIVLsfzTzXjvQTjGbFtiuK8Vcr51Zoz7Spp1rEfL5txmnUjMhIJ/zmTxOSLJbnNlUgwvFa+95mJIphbDNisHsO7v++f72X4WkUapFjkP7vv4eycpGOMOknYh7fDjui3Og+dTpD2FbDwQGXRzlNf7oz/4eVQ++OjTzHbF6y5h22ecupHcj5DtX8A6N2b4vc4eQn9a/VkubVSZQJ9LXYTHjhULbPvgHvRv2t28/6bWo+/vtKt3MptD5OLdluyJ2ziG9G0bERqopEuFeS3B31O13WWyJcQWij5CUyJI5WHLkormREFZyP6Ux7BderadwY8jv8mF2kykFv1C92oIiX/q/jiWj0mCXmGZj+WYui3HOOmxdl46Z7iyPqIzFoVCoVC0FTqwKBQKhaKtWBUVFjPsKEGRW0NKqXOAh2COTmLirWJ9lNlCc3dUPvMMntL44tfjedIxrm7sVvn27t0k3Hmeh2smSTImP8YpiCPFQ1G5O8tDN4c6+brqbBfGH8bEGFzxcJr4/JGDzLbvHgx7bZaeZzZjHdqqU5yCGdzAqZxkntyPycNOTZJBL5Xia4SbhEZ0lsIkTf6sa4XTdg5BOrtYb3trGK69IJJydZP01Z5Qjp4pIU00mOdtckoej7NFKKnMVtpJEl/FkjzRlU/aM5EQmUzTSEIsiNTEz/3gLradnyChyZ08KZdXJ5RokxMbTo2EKQvqpFogFK1gHvwFXo+FGaRvUtNcedklycUaZ29mNmsj1tVSDjb/+DBhMHbwSNQeFllR7wi60iCZPQ0RJxx3aAZJ0Q8auG8gMnAmZNi9RxLmhSIp4MDGqDxf5coJFULN2RY/bpnKAaGPDPEdoQnDlmXjO0bmR6quvCy8GI4OmRSNnTcUdCNNGGgsdhJ/2fEvDJ2xKBQKhaKt0IFFoVAoFG2FDiwKhUKhaCtW5WMpFypgtxRccz3Iv80WebhkIoM8XLkipEgIp/ns0/uZbXwU/R/ZLFca7e9fx7b7NiKvWT3IueXD0+jXSGY5cdzdizx4Z074LcwjbNuOEY7e5KGcXhPlYAJX8I4BhiLv2MllRk7dhNvZFOdtO3v5vVar6BdoNjmPW5pFP5bf5MclY8RXs6Sw67ZD1uHFkcs5kMktcuMzMyjV4pjcx5GxsG7nAx5SCyG2Syzkdbs+i+dJxjkn3xQ/kxpNPG9J+CZiSfTVhA6/RsrAe+vr4bI/MVv4Qw5PROXxKe7r83z0sZim0Ashoea2UCymvr1GkfeRVJy/F3Nl4rOb5P6gjiyeJ2PwcHmfhJ83oy5yfELSHzsyAVa81UdD/B6YQlLGoX4M4WOwbYfsx9uERn3XxavZ18H9YBu7cHsgwT+HmRT2tVqdfyuMAC8yX+TveK3J9/WJxJLl8PeYZrSUYdMW8Q816rwfGKQ+pNxNo8mXatDr06ymAABJ4l80hY+S3o1nLt2HeFePAp2xKBQKhaKt0IFFoVAoFG3FqqgwIzCilaMmWQlerhXYfv1EOdYCTiGNjWFoaTHk0/riPE7h7ASnFWYrfLsji6uKExlOM+S6R6JyMs4fsb9zkNik0igPe3Vdn5S5Am/o4JhcnOcJw3Jktn3F1VzdOE6UAAYHuPJATNzP7ieQ4poT4br1Ik5JQ4+HW3b04Hn9JVtwfGJJk7EYJFvTe4PcV2m+wPYzCRVmG7zeQ4+savZ4HbkuWXmf4s/kiJDUUgkp0pgIKc5m8PqOSNBVqZCkaiLRVleeU3r1BlIUMpma2yCJxiqcpiqV0JZKc3qkM4PPPFXktEYiwUPSwwBDiutNXo+HDyFNt+kwf3/6NuI74geN1v+PDxVmpDrASLTqkYTUSrK2Qf4gcsGBz1aF8/tOkbBd1+d1kq5ymirMIBWV7+JtPZjFb5yV5/1wZgH71vNT/N3cO8upeYOpLwu1bkLjxS1OUy2pTgMANBuCiqOr+YFDUmEuCeeXdGOCUWEyYRq+X0uviFvnz3Y06IxFoVAoFG2FDiwKhUKhaCt0YFEoFApFW7EqH0ulXAar5VuwKjgmZR1+GpdkXjMFp5iMIydtGtzHku3MR2Xf4rxprck54uok8oibhk9nto4k8XmIMFt3AbnSzjTnq8Hh3GSV8ok2v5/Awmfet5dzo539yNuecy73sSQBs126fpnZ6hXOlnouhhQ3a1x5NW7hNZJpHkpKKV3DXORJgxWqkr5kuN7iPwCg4sOO+A2T78Bw31TA+8HhItZ7Q/g4SnU8qePwvmXHhQK1i+05sm6E2Tq6US17Zpb7z1xynCfeEFfw11RapF7jIaF+De+vKsKGi3Mo5xN63P+T6UX/Ic2UCABQrnCfQbVB/IAe7+t1IveyfzfP8tpz8VBUtlvxubbT/uyGL4Sw2YSwFdoaEn+IIcJmAzi6pAlzMgjdG4/IxCRC7vgyhR9pYgF9lYGwHSiQDJIBr5sCaYeFKj+u6vN2KJI2NMV7QJ/fNqWXifhGxHEGeZ+XiRkLaZogwE4cinujkjahqCt64qXq9hsr88PpjEWhUCgUbYUOLAqFQqFoK3RgUSgUCkVbsSofixkzwIotjkW1OvJ/5YOc/2/MIG/ZN8Q5vTSRtF8Q61+yNvLQXf2c05yeFn4En6zVaPB962XkRuMGX3dgWvmoPDcjOPq0yPJXwvuplbk/BGw8z+FREf8+ghIPiQyXxreJlHqtJtYkNPJse2QY9+0Q/qAJImOTzojzmHic0XL/NOrHRza/OFeAwF2st8osStt0pnjagwSRsmg2+L0FNrZD1eASEvNEEj2bE3H/gqPPpdF3ke/gdZTNIA+9UBDtTiQ6LODrF3q7+HNQ1IXsRqSVAgBNIbtTLuO6hHKF9614HO9NypTPlPi7Nk+uWXf5Neou2sZGebZUWudBS6YmOE6SLr7nAUQ+VMLji2cN6Nor4UgwyHoMQ/hfPLI2JGvytk2In9Iz5FtRd3l/Mgu4c7XJr0/l9wPR79Limk2yHs73+XeM+h5DkQYioNcQPtKQ+JGWuU+FDBKtuuBY2SWNZc6aZdfwV7geTmcsCoVCoWgrdGBRKBQKRVuxOkkX8MFoTaVCEvbZm+MKsFYNp9ReiU8vAyKx0qzzaf3MDNI7UnE27XBKq7cPwyX7uvn1e/MoKQMup8kcku3NtTgFURSyMUcmUX154sgks82RTa+xi9myeTzPxMzTzNZhICWTip3GbH1DPKPm0DDSLobHQ3JLO5DmaXr8OXyDhLk2FqmkWqUBAP8Ea43A9SBoSYu4RLakK8MppIUCUoTTNa4O27MBw20707z/TBxBmZJcfZDZ4jbft7srH5UzKV5/toVT+lyO28YOIU1VqRyDngGAMqVSqpxaDUhk8nyRS3IUSiS7ZMhDmO0JpK1iWd7vyyIkdoEo1zYEBdIIcLsuwmU9Eubqt8KrfVcKp6wNTNMAs0V7sRBjQSlRWyipsGNomtAsjb7Iihg3RfvZ+B4VBZWYTuKJ7Ri/fpwssVioCdkYEbadieG+B+bFkgZyr47Ibkmfw5BTAFofUtNFMlrEvvw0lO4S4cYvhBUuW9AZi0KhUCjaCh1YFAqFQtFW6MCiUCgUirZiVT4WcOuwNBbFCJ+diYkQOiLD4TU5b2fEkWtOJfhxs1PIVfqckoYdm3kGyeHuTVHZtrmEQb1CsssBl8swCI9ZFiGEz+0/xLbHC7htuiIUsIDX6Ap5SOy2TiL7LmS6mzby+ZbLQ0ANIWkdS+Kx/T1bma0ntz4qFyvzzNYgYaZpe1FSplLm/P9awQYT7FYfcUhGuqaQOymW0C9UCzlHfdnVl0Tl00/jfpR7vol+oplRXu+DIjtgRxZDhZsiq1+D+CYCIa3eaBAe3Oec8uwcl7+HAJ9LctSVMh5bWBBZBUlGR1P4hiZm0f80mOfPBCnen0tENr8R8P7jERl0K8XDpn3m2gjZ/9ceBiwR/+ExOHvpVzmqLeD7+cT/Uhft55X5OxcamNbDifM66s/hdyUpUjJsIJlFN/XxUPa0iGkm7jz42d4JZvvJHryfuSZ3llg0FFv4nzwi3yOrSe7LUhOE0iGDCI4ViXz0w14QOmNRKBQKRVuhA4tCoVAo2opVUWG5XArs1sr7BFnVHNoiNJhkW/N8ToF4HoYUlxc4PWOVSTY1m0/5oeaIbZyKGjbP4OiTrINxR2QgJLTHAmeQICzuYNtJFxVwkyG/ftwajsoThQeZbaON4c4jiTP49U28fq3Kw4QXmuNsO5jDMFwj4Cv482ncDkxOKZaKSMnE0ouhu27j+Kgbx8MkxMPFthvo3RL9/SGfh2vPE9XrodP7mO2SKzAM+9QdQ8zWncIu+8O/v4PZigVen9UKhurOzfD6axK6MLRF5skGpUs5TdYpKL04WS3tezwUuEDCrZtCediJISVad/k15uvYVo6ga2sW7881wPepKVR+qyQM3cryPpJK4/X9FlXie8enj7i+G9GPtOZNEVN7LCqMcTNiPxphLMSxwQHeR87LY72cee55zNaXw4MDEbYcI9kd1/WKFfuCEvU83Nfe3s9sxRru+y/PF5iNqg0bgtKzCc0ZCgo9XEaFkWN93kd9cq/LQpFZlk5j6Y8rgs5YFAqFQtFW6MCiUCgUirZCBxaFQqFQtBWrUzduhGAt8bEGcnWuCBetEh6uWq4wm0OkEXIGD9OLE94y5vEwy7S1gW1bDeTvgxrnLZNOHjd8kXnNR05xMMvPOZC/iG3XfAzlrMzx0Nb9Uwejcqf9FLN1hPhc6/u2MNszE89HZdPoZDbH4PXYJNkB6zXO29Yyv4jKfoz7o4p15M9LhUW/zaKky9qjWnLBbGWsM+PYhg3hMhvagOHjb3g3r/dTtqP/LJbkpO7pl6H/RWZ3vOd/f59tP/r8vqhsNPjOzJ8Q4xIcc8SP0tUppGCSPLS9VsQ+Ulrg/H2FRC1bFr9+w0PjQp2HIlfJe/DMKJcZOjTDJUFKhHuXyrUNouWR6+lgtgxRy55rvaM+HB8fS+gHELbum/oDwmUZFMkx0o9CpHWkunFInsOyeftZ2Y1s20jh96FR4dJCczb66LJCEmjPNPrsHni2wGyV2TG2nRrApRGmz+/VreI7nxFyM3UiyRMaov/SDfH9XaZATOou8ISSONnXXiYpQ0/Rur5KuigUCoXiREAHFoVCoVC0FauiwsKZEJMCJXFK1DT5VD5G6IKY081sJkl4FHp8Wh8QbqNv6Cxmc/ztbHt6DLkVx+aP4SWRpvObnAKq1fCaiSSf3pqiNjryuOo7lhN0SS8+R0wk4SrWMY55svYks2UGcCxP+JwKa9R5KKnlY6htKKb7E3OPROW4w5WDu7pQbdl0F89ZTa5OZOFXxdjcFKTqiyGc9z5xb/T33i2cinnXb789Km8+jatTGzbSjo0Gp1KbRMnhjHN5ePjBh59n27d/586oHGtylWCX0IwywVVHAut63eAws8lkSGXSv2iYMABAoUFW1/OzgOPgeUoO76NOHvvT4SOzzDZR4vv2rMdQ7bEjnDbziLK3aXAKrziPFF7dWzxnvX581I0tMMCKVt4TmkZQOEx5V1JhZPtYK82NgIcCH67y7WcXkBp6evYws3WQpG6Bz69fWMA+6h7hCub2/AG2/db3IRU2Pcppsi0d2C/NBH+P7z2I3xFLsIQdRDE5G+ffpniMt7Vhob0hwudrVXyOhTqn26cby78ZwTFW7lPojEWhUCgUbYUOLAqFQqFoK3RgUSgUCkVbsSriffvQWRCPLXKUfgr5Y9/hvOVgHjnzhFCcNUgI3fQ0VxOeqyDXbSVOYbZ6Pc+2ay76dRJJHiZIlWxrFS4bU6kgZ+/7nFP0hdxBLoucZzLD42VHp1Hltm5xH8s4yUSZmeXkqNWJ53GLB5gtZXJutDO5MSrbMaFu2sB903HuxxoZQCVkBxZ9BOXS8VE37t80BOlWXXkZ5OzPOu9Mtt8pZw5EZT/kYboukbZuCuVhICGRsQzvvut3cgXo8s13RWXb5e1QJOHXMSHpctapm6Pyxk2bmW2hIkKKp5Cjnqjye52skrBXi/c1y0YfR2aAc+SXvhHVnSe//0tmG3M5R/+W9702Kv/0zvuY7f67MSR+VPhf3AaqYxsteRAjOD6/M60Qly0ERCMkJkKyPRLa2hByOdznIhwQRH7F4IG5LKsmAMAs8YvFRLhttk6+FfzykKmjKnE95HJBrgjJ9eZRqmni8HPcRvx7F1/5BmbrIT7gvgz/xq7rJt8mhz9/Is6/IzbxQctQZK+B78H+iQKz/e09B6LyeMv/IsOVjwadsSgUCoWirdCBRaFQKBRthQ4sCoVCoWgrVuVjOeOMyyCZXOTPzQ7k+MwMXyOQT6DPwYpzuW4LkCt86jkuNz97CKXV909wn4Bj87UyyQyRf3FLzBa6yDFWFrgUixcSbj3GectqmZ9n3wFcF5FJcN7SD7Dqyi6P/58u4dqDLe5GZpsbRY7y0IFnmM1pcq49n8H6GNrI14EseOjjCfLcx9PlEB9PfLGdPJHlcq3Q0d8Jmdzi/XzwwzdEf48l+W8Y18S6NgUPbpJumUzy2H4qJe4FfE3H0IYBtr1tB/pcjjzBfQwhSedgOdx/RrN8Pvr8QWabKnB/3sQ0+lymF3g/KBJpc9Pi9Z9JYD+48MpXMdsFv3ZhVL7vsf3MVt3L11qk89gvr337q5lt91M3R+VHH+Trqa64FutmYOPieirD5318rRBzbLCcxTY2TGzPjiT/VlRJqgEqnQPAfxEfS10/JjI/yvVgNvGHrM/x65/Wn4/Kc/MFZlsgPktXyORPFbkf7id33x2VzzjvYmaLx7Gvd2b4e7yuH9OB9AofS574uE2D+01S4ltlkjpoinUsBZJZ9rnD3H/nEz+2ESz54VTSRaFQKBQnADqwKBQKhaKtWBUVtvmMcyCdXpQICR2Sgc7m0yvbwjA9y+eyKUYS6YHqk3wKOXoYKaS5OpeyyGa43Ik3gddMxbmtrwtlLrpznEIqV0nGvSan11whaVEuYBhhPeDxhmaA+5brnJ4ok32LAZ/CG0TB1TG4KvPTe7kkSUcPHjtvc0rISePzlwUVODuPU/FN/YtZ8arl46NuXGmWwWjJpaS7sO0D4H2EZccTdIVHsl2Gofztg/XXdHn75ft5HV37jl+Lyt+euJXZqgU6pecU5KyJbdvTJ/qPx6mwBpFNsYW0T9LCftDXy9v6wotRpfmi157LbEYen3loUxezBUKiZO9epMqufdMFzLZ9O0oSPfQwD3M9cgBDYDecsigdJNWi1wqpVBLsFl1uEa2SOZHStUqyZ/pCUgVI1sTlki7YtjKboy/e43NG8lH51VtFXTdw3wVRNz6Ro6qWeJ/IiG8OzUx53kWX8X0JpdVs8O+PyeSFxTOSzZhwN7giI+mRA0ei8k8ffIzZHhzHb8czBV5XC0QGyWxlCQ5NlXRRKBQKxQmADiwKhUKhaCt0YFEoFApFW7EqVjWZ64BUy9fhEfkHX9JuDnKTQcjDhhMkTNit8BDQyT0oPx2KEObegdPZ9t7nMDSuZvBwUYPIddjDnJs1CEc/fugAs1WqXJqhWkVfhSXkX4yQyLknCswWEombwxPc/9JJZLLXrR9htoZIs1hr4vWbDR7CmO3Ca9QbPASwWUTONw6Lfpt6ZWVSDC8VvtcEz1tsY6YQEvL6s4lvwpOS6KRbRpnrWnA99KuEIuOeJ+Tn1+3aGJWTA1xaaOGZ0ahs2EIu40KUOX/zu17HbOOT42x7aqoQlUuijj2SZXV4kKcGWE/k7pvCRzlfQ//iyAbO+9smfy/27cbnSP86r4/zzkFZpEce3sNsNZLe0ncD9v+1RqlUAquV/oBesykzQRI/SuwYX6pQSLrQbmeJNAen9PP6e9/l+F1ZqHCf3fxCISp3xvkNjJbxHdt1xmnMduFlr2HbnV2YHiMp+lqcZH/szHF/dII8dMzkvqHZGfx2PvUs95/97L772fbPf/bzqDxv55mt65JronLV4/cWGOSdbfmmAuGzOhp0xqJQKBSKtkIHFoVCoVC0Fauiwkxr8R8AQEjkPl2x8twj6rRBjNMTQQmnfkaZhxR7ZVxp3tm7idka05NsuzKFFJMnFEvdMlJas+I4i2Rbq9V4mG6txqmwUhXvz5LpJS18xpFN3NY3iLRLikcCMlXWijvBbJs2rmfbto/ZC6vNp5jNtDGEsOlzCi2dQYotcPn/1xpG6z8AAI+EPdo2D+mlC3irVd5HOP3F6RmfqKs6CT51b4qfSck8XjMzlGe2iQq2fYdQ4O7bgtRFx0Yeyp4Y2sC2TzFw262JcPU6Plcg5HFNsuLcEGq4cQs7TU8vV67OCrok5iC1k8qKMNcLcHV95813MxvtD8kWzRM0j0+8cdP3I2o5JM9u2/w9NojasGBSwSO/iWMi3Dj0cOf+DF+F/rYLuFr1CFEuqIoV8/15DF/vFFkae9K4gn7Hdp7JNNfB6csmyTIaFyrXJqHC5qY4zXqQKH/88sGHme2BhzFseO/z+5itVObfMZ+E03de+FZmq5HlIIbI6OvQZQBLYf/Lwv9fGDpjUSgUCkVboQOLQqFQKNoKHVgUCoVC0VasilStN+tgtXjYZs0nf+fKrT5R0vWICi8AgAdECmFBKJbGkSu10/zWCjOcN5wZJz6GkIcJej6GOGfyg9xWR74xaPJQ6GqNhz/X/amobAglZJtkbesZ4dc4ZRv6hyZmuY8nRuh8w+S2ZoXX1UDnTtwwh5gtzGB9PPcsl8IYJPIh6fiidEbN4vzpWqHWDMFsSXFYFg0XFdkBSYhoVUhZ1OpE+diUv33wuLTF/R++wfc1TewX+cFOZvMsbE/T4Y6wLhIe6grfSFNI05ge8ueGsAHxozSFH9IgEh0yXDZmIe+fyXEfS2ePyNY6jP3CF6HI3evxvOu38POEZI2A3fJRWFIaZY1gQEjC/rF+DRFaHiN+zY4U95U0SGiyJ7JLWi7W+0iG94ntoh/UiIyT4XNfXzqB9blhE/etmZvR/xmP8f7ji+9haQZ9qQ/t3ctsTz2FvtNHHuNyK8/vQ99JqST8JuSZA7EUwhLqN4lu/B5ke4eZLaTnEXI3IZM6aoWk+xpurFAoFIoTAB1YFAqFQtFWrIoK8wMD/FZob0CmW4kYV5V1G0RBuMBD6ObcQlROdeeZ7fLXYcKjsSqndw7PjbLt3i04/QwEBeK7SHE1gYcQpnNIHUwd5vdWb3IqbOtZJGwwyeeXswsYipzv4+G+YCBdUStzeqGrF6fXXsifsaefh4v29uJzmSZfuV2ooZJub54/f9xC29TY4rS8Xj0+VFjdA7BajJBJYopd4Nd3XUIhidXRsTjSHr7Hp94B6Xh1QaHVmzxs1yW9O9vBaTMrhtN8J8HbL+5gXTeqYnW/KcLnG9jX7ECEVJNblwmmPBdph2qNU7INE59/bq7CbDVB36bSeO8zc1xl1yOUUFqEIlcqaKtWFxusVjs+MelxywFriYokVbZtqI/tt2UQE11t6OJh1oUy1stCmddRjKgzZF3+jjXrvD81iIJxNsvVqVNx3Ba5tCCdxvuZn59itrvu+hnbvvfeX0TlZ57lCuYzs3h/TY/3LZ/G5Et1Z0KfWhb/jFsx/hxONy5jMISNqrQb4jw0FDwMvdb/V9ZHdMaiUCgUirZCBxaFQqFQtBU6sCgUCoWirViVj6XpBuC0eGyDHGoEYnzy0eYkeChegsgkZCrcN1PahzIt553ey2xbTuf8NZgYQtes8es/8FM8z8wMD89MZvGa1Rr3v3R08X13nY8hhvunuIIoZJEzH1o/wEydnRh+nEnzMOGahyHGJSFlEoT8+kdmnozKXXnuY2lUkTPvSPIQSpeEgjdasiKNxvHhz6tND6C5yMd6JMTWdngblUqFqJxNc/68txtDY0NHKB8TSZyayPhZq4qwdws5Ypk50Ixh+xWEBMbB/ch7dw7yPmoleZ8JfazXwOV9tFTH+6k3pWwNPofM+OeRZz4k/IALIuzUJPVaLPN7M0P01dTqvB737EWf5UJx8frVMq+jtcKlp2+BeGrR55VP4X1t6eXSOmkS2tph83tziURQLc3fG6+CPpdGVXybZPg68e+lYtzmkGyv5ZkxZiuPYTvc8YtHmO0b/3Ab256ZQt9tIHw1AfltHxi8/1C5l1BIGxkkRD4W536TmFwa0UdCjG3+rlFHYAC8j7LMnJGmzsr6iM5YFAqFQtFW6MCiUCgUirZideHGTR98p5Wgp44hfbYtkmnZSAFkczyU068VovLooWeYbc+TuCo1mziV2epdXAm4RmiW7iRXBTYDvLfezm3MFk9iuG9DJDbq6MmzbZeofZZKM8w2PIJUneHzlf9334nhhU6KX6NvPU49YxanCSfGeLhz08eQ5rkyp9S6Eji97chwCsGz8feC15p708ROa4lypQI+LD5jzEEqJi4SHMXIamXT4N3QINvNJq/bahXDbV1XrAIOj77phmJ1dgLrqFDgIam3/dPtUTnX/UZm27hZrPYnq+09sUq/WkNqoSRoKrpa3BHUhRng9vgkVwBvivBrmySgkjaf0G+e4GDGDiG1Mzu7eG81kehqrfD2czdAutVnY3FspYPjvP/fezeG7Z4uQvoN0reaIlz9+eeQQj5lK3//TUHlFEYx/Lcyz8O1J8YxjHjP8zxM+PAMtouX4lR41zBXZg/Je+43+fU98tO+IVXiq6hAkXR4uLpJ5J7rVR5u7Sc4bZ7sxDBuSt0CAHiECgtBJDMkVJjf6tuBoHSPBp2xKBQKhaKt0IFFoVAoFG2FDiwKhUKhaCtW5WNxHA8cZ5Gjc8tEyiLGw+TqPvojxiYfZ7ZnH3wiKmeFOm3axVC4Z37yKLPFN3KOcZb4eFJb8sy2cQTD745MCpkEwnHaMa6Y2r9eyIeEyIsHVb5vykTedP9ze5jt3l+g8vLIabyKgyyO5Y7HFWe9Ir9GVy8ee2A/53ifXUAl5Ndd+SpmGxhBPrriLXLBNhwfH0siFoNkS5IlkcDniYlw40QnhkvHbf7ctRq27UJhQdiIcrXwLYUBbz/qj5E/odId2EfOPv8cZjtwGNvzf3/p75jt8ldfwLZP3bUuKnf0c59ZGOJ7YVs8zNMgfLYnePfphUJU3vv8AX7jMrKf+I58kUm11sQ2TwqVX6eEfavSynxZqx+fkPRaaIPZUjKeI36dZ8e52vnPn3w6Kh8RvsruDPbxDofXX44sKUgKKZsj49xXuucg+koeepRnadxzBP1QpbqIE7axrV9z9mnM9MYdPEslcedBQighj06hH+fIFL+3Yhl91bufepLZnnvo3qgs1Y1jg1vZdkB9PFWuoA4kxNl0+HvIfSz+C17raNAZi0KhUCjaCh1YFAqFQtFW6MCiUCgUirZiVT6WgnsEmu4iN91sIP9X4UreMFlAP8rY/N3MNjNRiMoDzunM1k34viJZ7wIA4ExwPj1WQ171iL+b2ba/BqVYZgN+nvkxfOTeQc4X7jpf+AGI1MjMDF8rMz2NXGU6w2U/duwYicq5EV45oY/15ru8+idGeTx6ZY6sUWjwNQaFMvoeRnfwuPV0FuPWx2cWfVyN6sq40ZcKB3xwWv4D00eOP2HxdQg0a2Io1lgERCI8Hue+iRjxiyWTPGNiqcTXivgkk2gixc/jkfUMW7bz7IDbdqJc0G3f4f335m/9nG2/roL+mfOu4ucJSAZEz5VrBLCvhSH3jUxNIe9fKvN2X7eB98NSGf0SE1N8HYhNMzB2875mOthHyi0JlHp1ZWsUXioeGC9AIr1YH0uSQwAA45Pcx5IiSiVzVW7bP4G+iaEs99W+/a3oczxt55nMFkvyd7V7EH1kfaduZ7Yrie+rr4v7avJJUrdJLqkST/C+libbjpCUKTfw+edEaovxArb9T3v5O14j6SPGZvlap1CkkKzOoa/IF0lCkymsu9DkvnLqY1mSIKJSRMeCzlgUCoVC0VbowKJQKBSKtmJ1VFhlEhrh4rSuUkSJFb/GKZxCGUNjgzpXnO0gaqbVhb3Mlu4ioW8ilNRJ8OluzsWpqdnPp6KdvTj1zHXwud+h5wpR2QA+9Zub5ONsw8Pwv/6BEWY7PIo0y+wMf/7QwSltnxATjcfxfph6KAA0GpwSGt+NCqpph59o21koG1Eu85DcmXmsYyfekuBZYZjgS4XXrMOSEo7XxPuwhTh1KoXUmCPCHC1C4cSEjU7FKY0CABA0+TOaPkqjeA1uoxks5+Y5lXDxq3dE5QsvO4/Z7r/7Kba9/yCGlg8c5qGk8Qz22Y6OLmZrEvmOYpH3nxIJ5d962hZmy+e5fEiuEyu2sMCVjy1CbazfOsxsdaL6W20uXl9KiqwVCvMFiLcyNxJlGzCE3EjMILItJq/bgS7sByOnnMVsm888Pypn85z6MgUVlcvgO9jfzakwIoANZijUhQmVa4jsoL6ki3zsa02Pn8ck9H9KSPv0d+B7cOF5vB/GM/mo/IM772C2Q2MH+eUD/AZ74jtiWnhNG/i7ZpL+s/StkrT10aAzFoVCoVC0FTqwKBQKhaKt0IFFoVAoFG3FqnwstdIkgL/IdRoWhjY6WR4S2ZFCzrGxj/s/sr3Io7o9XF7AcJCHHuo6g9mOjHLZ/IU96Fc4bZhLKmQyyHGuG+G88ewYXnPf05wLrRW5I8BKIdcdS3JfUf8Q3uvEES7F0AgIZy74VoNkgsvlOW+8aQvPBDm9FzNhei7nRotzyNtOjHNuveEXonJ3KxWA73HZi7VCteZBaLZkfzzS1h7/DdNsYh9JJXkdMX+QCMW1LOyyvvCpuDXO0dOMiJOj3I/ST8I3Ozvy/Djif9mwk2cyna/z7RhJUSASUYJr4vVjSV7/vkf8TyIDYP8w+vM2buZ9pCnkX0jUMjRFBsuFIr4j6QwP904myPVTizy7Byvjz18qBnIpSKQXQ8Vd0taukWf7xdO4fUhEQsc6sP1e9epzma2LhB+7wqcRhLzPlImZtiUAQJa7HBhs0i9Nix9nmSKmlzaSkB0Kg+UhvfgHLOZz3Fe0fQv6WJ9+bpDZRke5j4VK41sipJiFusvLE3/Kkun/39659LZtRFF4+NDLiozISuPYsQM7SVEU6Qsoirar9nd33V2766JAgQZF/UijuFJsS+Kb7EKy7z1XcSwDjIMU59uY9FDkDDnUaM7cOUNLF0IIIe8FNiyEEEJq5UZSWDz+w7l43m0OWtI3TTzsbjZ7IttsPcOVD/Wqf3kL27XyVEKMz4Y4Y33yGvejFyJN/fYLzrwfrEux/AaGKX/3o8gOe/ubkLbxEfa31++LDNEZmDA9X8I+T45wxbjhSMKoy9bfkOYyFVJYYl+7uWbcRZUK0rtjuvSlzESeTFAeyZUE027PJZDbmnl/eha5JF+WVIoCJclZJPnxSsx/okLUtfTlHM5qbhqn2MkMJdlMyU29DZQSvv9B5JNHeygl+Mott7eBs/u/+gZl17Wm1Kf1dQyRT5wqh4/l8JTs0jLyhJYkYrOCZpah3NfuiMTV62EZmy25P0ETr5+qGd8Xx5XF7fzO3Busu7WFW0VRSr14HWK9mSmJ8uM+ysRPvpYZ9Q8fohtBqu5REKAstTRvXP2jLDEV3amN3KV+k3tW+nL2PNVVSUBpHShUflomXn9dOUk8fYTl//P5c9g/HIlGW4Xme8yT7yM7/cFX5aoWebElvQr2WAghhNQKGxZCCCG1woaFEEJIrdxojGWzE7rOwtVzpqxJQoe6XaX042Yfw3TTsejAsyEkufHvEhLanBgLl8SstqhWJEwq1O/LQnTv8UvUqM+VbcXjfXQMTTLU+kcHkh9/gpltqxX59vfRQXXzoeje4xjHAV69krGRMsX7FjRRwfzy2z1JK8aQVjoZc4pyvMeeeh6eX8Hfd03pmq5cWEM0QjWe5KNdxWSqXJ5THNuaTiRcOzAhoP27gUpb8suB3faaXPOBGWPo3hMn5E4Pr1GUsh+WeM6wj+XotmQMphHiNbJIyuUbW1ntdnx2jpY8ibofnil/aMqhnUZabZPXhuR1apyLfWWRMjmfvyNJdDvuxoM7bdftzd+RLJXyTGb4/q19JuNgu/dw/OqTxxL23TS/j/2GnLNhBgUaZjhLD11Ya5bQk3dmKYJY270YmxjfHFypsZLKmXBjVeSswvJX6jyBsZ/qduT5ffH5p5CWmIGcn37+9XJ7eIrfh74qSODZfsYb7Ke81ZoM9lgIIYTUChsWQgghtXIjKWwjv+u6+VzmSLakazo8fA3HDQ9fXm7na9i9DlPlSnyE3cL2SHUFjZupy7Er3H0qctfgCXb9AnUNN8S8/fNc8laMMYT5/j4u5uOX0v3sJBiSOjoVuaZRYEjxYFPCmB9sYHhqER9dbh8cvYS0zh3r0iz3II9R9gl1H/8Ey5+cqtnMcb74ezvhxmlWOT+b5ydXYZ+RmRU/VavDtay7cdhV23j+SrnBJjmWKSkwXDNL5RlZCaKlQtJzD+WBVN2rwjhOJ1Osz2kg0ipIf865k5HIpxv9u5BWqhDUkxe4QFecyjnvbaGbcWFCQkdnWiLFeuCrm/fi2EipSp4pFjOz0/h23I2rInFVPr+PsQp77jTwd+6zpxJGu93H+t/x5Rn5JqQ48K4O7/WtE4ba9c299dSxZsK+K5W0bGfM5yZsWztJZAUeO02lfk2MW3ek6l5R4YsQqbpfBFjvtnZwwblB/6/L7X/PDiBN3yvPOjiD68Viu6K7MSGEkPcAGxZCCCG1spIUdtHVm6mucqqiTKIIu9BxLLJH7mGkQ6gUkTwxM661qaBv+p4mYitRa5Z7sZHCGnIRa9qXqS5kmpqFvWIzg13NCPcic32V93iGMk80lfuRm4i1WEW+pBGW0Q+MUWQm+SuMlFXocsTWaE/2L2bcJ4trrbpm9U25OK+OLPLfMk9XH1dldqayMs1biuKRf9iZ0rExyNMGh1YKc2rfN9Ew+rlYKSw1kVOVkmEKM3M8Ue9LbD6npbA0xvqjZ44n5t0KjJmnTrdr1vuBriMmcvKNUtj8uu+6jswmEhk5U/mfmcXY9AJw0xDzXwRaCjPmsep5muXfXWr+oT9qpbDgLdPkdZ3xTP2pnI0yvFoKmykpbGqlsPRqKSxW7/8sw6jQeDaB/TwRqbfM8BqeimT0THExSs5bfH5+ruvqiFetUIsODw/d7u7udYeRD4CDgwO3s7Nz/YE3hHXk/wPrCLmO6+rISg1LWZbu+PjY9Xq9JT8Z8mFQVZU7Pz9329vbS3H3dcA68uHDOkKuY9U6slLDQgghhKwKB+8JIYTUChsWQgghtcKGhRBCSK2wYSGEEFIrbFgIIYTUChsWQgghtcKGhRBCSK38B0IkX+lihZnxAAAAAElFTkSuQmCC",
105
+ "text/plain": [
106
+ "<Figure size 500x500 with 9 Axes>"
107
+ ]
108
+ },
109
+ "metadata": {},
110
+ "output_type": "display_data"
111
+ }
112
+ ],
113
+ "source": [
114
+ "import matplotlib.pyplot as plt\n",
115
+ "\n",
116
+ "class_names = ['airplane', 'automobile', 'bird', 'cat', 'deer', \n",
117
+ " 'dog', 'frog', 'horse', 'ship', 'truck']\n",
118
+ "\n",
119
+ "plt.figure(figsize=(5, 5))\n",
120
+ "for i in range(9):\n",
121
+ " plt.subplot(3, 3, i + 1)\n",
122
+ " plt.xticks([])\n",
123
+ " plt.yticks([])\n",
124
+ " plt.grid(False)\n",
125
+ " plt.imshow(x_train[i])\n",
126
+ " plt.title(class_names[y_train[i][0]])\n",
127
+ "plt.show()"
128
+ ]
129
+ },
130
+ {
131
+ "cell_type": "markdown",
132
+ "metadata": {},
133
+ "source": [
134
+ "## Data normalization\n",
135
+ "\n",
136
+ "The pixel values for each image in the dataset are unsigned integers in the range between 0 and 255.\n",
137
+ "\n",
138
+ "**Let us normalize the pixel values of the RGB images, e.g. rescale them to the range [0, 1].**"
139
+ ]
140
+ },
141
+ {
142
+ "cell_type": "code",
143
+ "execution_count": 6,
144
+ "metadata": {},
145
+ "outputs": [],
146
+ "source": [
147
+ "x_train = x_train.astype('float32') / 255.0\n",
148
+ "x_test = x_test.astype('float32') / 255.0"
149
+ ]
150
+ },
151
+ {
152
+ "cell_type": "markdown",
153
+ "metadata": {},
154
+ "source": [
155
+ "## Preparing labels\n",
156
+ "\n",
157
+ "The labels for each input data indicate which object represents the image.\n",
158
+ "\n",
159
+ "In this case, we will represent this label with a vector of 10 positions. This is known as **one-hot encoding.**\n",
160
+ "\n",
161
+ "**Let us convert the labels into the equivalent one-hot encoded form.**"
162
+ ]
163
+ },
164
+ {
165
+ "cell_type": "code",
166
+ "execution_count": 7,
167
+ "metadata": {},
168
+ "outputs": [
169
+ {
170
+ "name": "stdout",
171
+ "output_type": "stream",
172
+ "text": [
173
+ "Training labels shape: (50000, 10)\n"
174
+ ]
175
+ }
176
+ ],
177
+ "source": [
178
+ "from keras.utils import to_categorical\n",
179
+ "\n",
180
+ "y_train = to_categorical(y_train, num_classes=10)\n",
181
+ "y_test = to_categorical(y_test, num_classes=10)\n",
182
+ "\n",
183
+ "print('Training labels shape:', y_train.shape)"
184
+ ]
185
+ },
186
+ {
187
+ "cell_type": "markdown",
188
+ "metadata": {},
189
+ "source": [
190
+ "# **Defining Model (Custom CNN)**\n",
191
+ "\n",
192
+ "The Sequential class of the Keras library is a wrapper for the sequential neural network model.\n",
193
+ "\n",
194
+ "**Define the following model:**\n",
195
+ "\n",
196
+ "**For the convolutional front-end, start with convolutional layers with a small filter size (3,3) followed by max pooling layers.**"
197
+ ]
198
+ },
199
+ {
200
+ "cell_type": "code",
201
+ "execution_count": 8,
202
+ "metadata": {},
203
+ "outputs": [
204
+ {
205
+ "name": "stderr",
206
+ "output_type": "stream",
207
+ "text": [
208
+ "/opt/miniconda3/envs/ironhack.nn/lib/python3.10/site-packages/keras/src/layers/convolutional/base_conv.py:107: UserWarning: Do not pass an `input_shape`/`input_dim` argument to a layer. When using Sequential models, prefer using an `Input(shape)` object as the first layer in the model instead.\n",
209
+ " super().__init__(activity_regularizer=activity_regularizer, **kwargs)\n"
210
+ ]
211
+ }
212
+ ],
213
+ "source": [
214
+ "from keras.models import Sequential\n",
215
+ "from keras.layers import Conv2D, MaxPooling2D, Flatten, Dense, Dropout\n",
216
+ "\n",
217
+ "model = Sequential()\n",
218
+ "model.add(Conv2D(32, (3, 3), activation='relu', padding='same', input_shape=(32, 32, 3)))\n",
219
+ "model.add(MaxPooling2D((2, 2)))\n",
220
+ "\n",
221
+ "model.add(Conv2D(64, (3, 3), activation='relu', padding='same'))\n",
222
+ "model.add(MaxPooling2D((2, 2)))\n",
223
+ "\n",
224
+ "model.add(Flatten())\n",
225
+ "model.add(Dense(128, activation='relu'))\n",
226
+ "model.add(Dropout(0.5))\n",
227
+ "model.add(Dense(10, activation='softmax'))"
228
+ ]
229
+ },
230
+ {
231
+ "cell_type": "markdown",
232
+ "metadata": {},
233
+ "source": [
234
+ "**Let us check the architecture using summary().**"
235
+ ]
236
+ },
237
+ {
238
+ "cell_type": "code",
239
+ "execution_count": 9,
240
+ "metadata": {},
241
+ "outputs": [
242
+ {
243
+ "data": {
244
+ "text/html": [
245
+ "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\">Model: \"sequential\"</span>\n",
246
+ "</pre>\n"
247
+ ],
248
+ "text/plain": [
249
+ "\u001b[1mModel: \"sequential\"\u001b[0m\n"
250
+ ]
251
+ },
252
+ "metadata": {},
253
+ "output_type": "display_data"
254
+ },
255
+ {
256
+ "data": {
257
+ "text/html": [
258
+ "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓\n",
259
+ "┃<span style=\"font-weight: bold\"> Layer (type) </span>┃<span style=\"font-weight: bold\"> Output Shape </span>┃<span style=\"font-weight: bold\"> Param # </span>┃\n",
260
+ "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩\n",
261
+ "│ conv2d (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">Conv2D</span>) │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">32</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">32</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">32</span>) │ <span style=\"color: #00af00; text-decoration-color: #00af00\">896</span> │\n",
262
+ "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
263
+ "│ max_pooling2d (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">MaxPooling2D</span>) │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">16</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">16</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">32</span>) │ <span style=\"color: #00af00; text-decoration-color: #00af00\">0</span> │\n",
264
+ "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
265
+ "│ conv2d_1 (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">Conv2D</span>) │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">16</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">16</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">64</span>) │ <span style=\"color: #00af00; text-decoration-color: #00af00\">18,496</span> │\n",
266
+ "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
267
+ "│ max_pooling2d_1 (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">MaxPooling2D</span>) │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">8</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">8</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">64</span>) │ <span style=\"color: #00af00; text-decoration-color: #00af00\">0</span> │\n",
268
+ "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
269
+ "│ flatten (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">Flatten</span>) │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">4096</span>) │ <span style=\"color: #00af00; text-decoration-color: #00af00\">0</span> │\n",
270
+ "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
271
+ "│ dense (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">Dense</span>) │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">128</span>) │ <span style=\"color: #00af00; text-decoration-color: #00af00\">524,416</span> │\n",
272
+ "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
273
+ "│ dropout (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">Dropout</span>) │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">128</span>) │ <span style=\"color: #00af00; text-decoration-color: #00af00\">0</span> │\n",
274
+ "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
275
+ "│ dense_1 (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">Dense</span>) │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">10</span>) │ <span style=\"color: #00af00; text-decoration-color: #00af00\">1,290</span> │\n",
276
+ "└─────────────────────────────────┴────────────────────────┴───────────────┘\n",
277
+ "</pre>\n"
278
+ ],
279
+ "text/plain": [
280
+ "┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓\n",
281
+ "┃\u001b[1m \u001b[0m\u001b[1mLayer (type) \u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1mOutput Shape \u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1m Param #\u001b[0m\u001b[1m \u001b[0m┃\n",
282
+ "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩\n",
283
+ "│ conv2d (\u001b[38;5;33mConv2D\u001b[0m) │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m32\u001b[0m, \u001b[38;5;34m32\u001b[0m, \u001b[38;5;34m32\u001b[0m) │ \u001b[38;5;34m896\u001b[0m │\n",
284
+ "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
285
+ "│ max_pooling2d (\u001b[38;5;33mMaxPooling2D\u001b[0m) │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m16\u001b[0m, \u001b[38;5;34m16\u001b[0m, \u001b[38;5;34m32\u001b[0m) │ \u001b[38;5;34m0\u001b[0m │\n",
286
+ "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
287
+ "│ conv2d_1 (\u001b[38;5;33mConv2D\u001b[0m) │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m16\u001b[0m, \u001b[38;5;34m16\u001b[0m, \u001b[38;5;34m64\u001b[0m) │ \u001b[38;5;34m18,496\u001b[0m │\n",
288
+ "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
289
+ "│ max_pooling2d_1 (\u001b[38;5;33mMaxPooling2D\u001b[0m) │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m8\u001b[0m, \u001b[38;5;34m8\u001b[0m, \u001b[38;5;34m64\u001b[0m) │ \u001b[38;5;34m0\u001b[0m │\n",
290
+ "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
291
+ "│ flatten (\u001b[38;5;33mFlatten\u001b[0m) │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m4096\u001b[0m) │ \u001b[38;5;34m0\u001b[0m │\n",
292
+ "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
293
+ "│ dense (\u001b[38;5;33mDense\u001b[0m) │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m128\u001b[0m) │ \u001b[38;5;34m524,416\u001b[0m │\n",
294
+ "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
295
+ "│ dropout (\u001b[38;5;33mDropout\u001b[0m) │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m128\u001b[0m) │ \u001b[38;5;34m0\u001b[0m │\n",
296
+ "├─────────────────────────────────┼──────────��─────────────┼───────────────┤\n",
297
+ "│ dense_1 (\u001b[38;5;33mDense\u001b[0m) │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m10\u001b[0m) │ \u001b[38;5;34m1,290\u001b[0m │\n",
298
+ "└─────────────────────────────────┴────────────────────────┴───────────────┘\n"
299
+ ]
300
+ },
301
+ "metadata": {},
302
+ "output_type": "display_data"
303
+ },
304
+ {
305
+ "data": {
306
+ "text/html": [
307
+ "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\"> Total params: </span><span style=\"color: #00af00; text-decoration-color: #00af00\">545,098</span> (2.08 MB)\n",
308
+ "</pre>\n"
309
+ ],
310
+ "text/plain": [
311
+ "\u001b[1m Total params: \u001b[0m\u001b[38;5;34m545,098\u001b[0m (2.08 MB)\n"
312
+ ]
313
+ },
314
+ "metadata": {},
315
+ "output_type": "display_data"
316
+ },
317
+ {
318
+ "data": {
319
+ "text/html": [
320
+ "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\"> Trainable params: </span><span style=\"color: #00af00; text-decoration-color: #00af00\">545,098</span> (2.08 MB)\n",
321
+ "</pre>\n"
322
+ ],
323
+ "text/plain": [
324
+ "\u001b[1m Trainable params: \u001b[0m\u001b[38;5;34m545,098\u001b[0m (2.08 MB)\n"
325
+ ]
326
+ },
327
+ "metadata": {},
328
+ "output_type": "display_data"
329
+ },
330
+ {
331
+ "data": {
332
+ "text/html": [
333
+ "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\"> Non-trainable params: </span><span style=\"color: #00af00; text-decoration-color: #00af00\">0</span> (0.00 B)\n",
334
+ "</pre>\n"
335
+ ],
336
+ "text/plain": [
337
+ "\u001b[1m Non-trainable params: \u001b[0m\u001b[38;5;34m0\u001b[0m (0.00 B)\n"
338
+ ]
339
+ },
340
+ "metadata": {},
341
+ "output_type": "display_data"
342
+ }
343
+ ],
344
+ "source": [
345
+ "model.summary()"
346
+ ]
347
+ },
348
+ {
349
+ "cell_type": "markdown",
350
+ "metadata": {},
351
+ "source": [
352
+ "**Compile the model.**"
353
+ ]
354
+ },
355
+ {
356
+ "cell_type": "code",
357
+ "execution_count": null,
358
+ "metadata": {},
359
+ "outputs": [],
360
+ "source": [
361
+ "model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])"
362
+ ]
363
+ },
364
+ {
365
+ "cell_type": "markdown",
366
+ "metadata": {},
367
+ "source": [
368
+ "**Train the model for 10 epochs with a batch size of 64.**"
369
+ ]
370
+ },
371
+ {
372
+ "cell_type": "code",
373
+ "execution_count": null,
374
+ "metadata": {},
375
+ "outputs": [],
376
+ "source": [
377
+ "history = model.fit(x_train, y_train, batch_size=64, epochs=10, validation_split=0.2)"
378
+ ]
379
+ },
380
+ {
381
+ "cell_type": "markdown",
382
+ "metadata": {},
383
+ "source": [
384
+ "**Plot the loss function and the accuracy curves.**"
385
+ ]
386
+ },
387
+ {
388
+ "cell_type": "code",
389
+ "execution_count": null,
390
+ "metadata": {},
391
+ "outputs": [],
392
+ "source": [
393
+ "plt.figure(figsize=(10, 4))\n",
394
+ "\n",
395
+ "plt.subplot(1, 2, 1)\n",
396
+ "plt.plot(history.history['accuracy'], label='Train Accuracy')\n",
397
+ "plt.plot(history.history['val_accuracy'], label='Val Accuracy')\n",
398
+ "plt.title('Accuracy')\n",
399
+ "plt.legend()\n",
400
+ "\n",
401
+ "plt.subplot(1, 2, 2)\n",
402
+ "plt.plot(history.history['loss'], label='Train Loss')\n",
403
+ "plt.plot(history.history['val_loss'], label='Val Loss')\n",
404
+ "plt.title('Loss')\n",
405
+ "plt.legend()\n",
406
+ "\n",
407
+ "plt.show()"
408
+ ]
409
+ },
410
+ {
411
+ "cell_type": "markdown",
412
+ "metadata": {},
413
+ "source": [
414
+ "# **Model evaluation**\n",
415
+ "\n",
416
+ "## Accuracy\n",
417
+ "\n",
418
+ "After training the model, the network has to be evaluated with new test data. This can be achieved by using the `evaluate()` method.\n",
419
+ "\n",
420
+ "**Let us print the test accuracy of the network.**"
421
+ ]
422
+ },
423
+ {
424
+ "cell_type": "code",
425
+ "execution_count": null,
426
+ "metadata": {},
427
+ "outputs": [],
428
+ "source": [
429
+ "test_loss, test_acc = model.evaluate(x_test, y_test)\n",
430
+ "print('Test loss:', test_loss)\n",
431
+ "print('Test accuracy:', test_acc)"
432
+ ]
433
+ },
434
+ {
435
+ "cell_type": "markdown",
436
+ "metadata": {},
437
+ "source": [
438
+ "---\n",
439
+ "\n",
440
+ "# **Defining Model (Transfer Learning)**\n",
441
+ "\n",
442
+ "We can use a pre-trained model like MobileNetV2 to extract features. However, running this on a local CPU can be very slow. \n",
443
+ "\n",
444
+ "To solve this for local CPUs, we **resize the images once before training**, instead of resizing them on-the-fly inside the model during every epoch.\n",
445
+ "\n",
446
+ "## CPU Optimization: Pre-resizing the data"
447
+ ]
448
+ },
449
+ {
450
+ "cell_type": "code",
451
+ "execution_count": null,
452
+ "metadata": {},
453
+ "outputs": [],
454
+ "source": [
455
+ "from tensorflow.keras.applications.mobilenet_v2 import preprocess_input\n",
456
+ "\n",
457
+ "print('Resizing data for MobileNetV2... This takes a minute but saves HOURS later!')\n",
458
+ "\n",
459
+ "# MobileNetV2 expects values in [-1, 1], not [0, 1]. \n",
460
+ "# We also need to resize to at least 96x96 for MobileNetV2.\n",
461
+ "\n",
462
+ "# Note: We only use a subset of the training data (e.g., 20% = 10k images) \n",
463
+ "# to keep RAM usage and computation time reasonable on a standard CPU.\n",
464
+ "SUBSET_SIZE = 10000\n",
465
+ "\n",
466
+ "x_subset = x_train[:SUBSET_SIZE] * 255.0 # Scale back to 0-255 first\n",
467
+ "y_subset = y_train[:SUBSET_SIZE]\n",
468
+ "\n",
469
+ "x_train_resized = tf.image.resize(x_subset, (96, 96))\n",
470
+ "x_train_preprocessed = preprocess_input(x_train_resized)\n",
471
+ "\n",
472
+ "x_test_resized = tf.image.resize(x_test[:2000] * 255.0, (96, 96))\n",
473
+ "x_test_preprocessed = preprocess_input(x_test_resized)\n",
474
+ "\n",
475
+ "print('Resized training data shape:', x_train_preprocessed.shape)"
476
+ ]
477
+ },
478
+ {
479
+ "cell_type": "markdown",
480
+ "metadata": {},
481
+ "source": [
482
+ "## Building the Transfer Learning Model\n",
483
+ "We load MobileNetV2 without its top layer, and add our own dense layer for CIFAR-10 classification."
484
+ ]
485
+ },
486
+ {
487
+ "cell_type": "code",
488
+ "execution_count": null,
489
+ "metadata": {},
490
+ "outputs": [],
491
+ "source": [
492
+ "from tensorflow.keras.applications import MobileNetV2\n",
493
+ "from tensorflow.keras.layers import GlobalAveragePooling2D\n",
494
+ "\n",
495
+ "base_model = MobileNetV2(input_shape=(96, 96, 3), include_top=False, weights='imagenet')\n",
496
+ "base_model.trainable = False # Freeze the base model!\n",
497
+ "\n",
498
+ "tl_model = Sequential([\n",
499
+ " base_model,\n",
500
+ " GlobalAveragePooling2D(),\n",
501
+ " Dense(10, activation='softmax')\n",
502
+ "])\n",
503
+ "\n",
504
+ "tl_model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])\n",
505
+ "tl_model.summary()"
506
+ ]
507
+ },
508
+ {
509
+ "cell_type": "markdown",
510
+ "metadata": {},
511
+ "source": [
512
+ "## Training the Transfer Learning Model (CPU Optimized)\n",
513
+ "Because we pre-resized the images and froze the base model, this should run reasonably fast (1-2 minutes per epoch on CPU)."
514
+ ]
515
+ },
516
+ {
517
+ "cell_type": "code",
518
+ "execution_count": null,
519
+ "metadata": {},
520
+ "outputs": [],
521
+ "source": [
522
+ "print('Training Transfer Learning model on CPU...')\n",
523
+ "tl_history = tl_model.fit(\n",
524
+ " x_train_preprocessed, \n",
525
+ " y_subset, \n",
526
+ " epochs=5, \n",
527
+ " batch_size=64, \n",
528
+ " validation_split=0.2\n",
529
+ ")"
530
+ ]
531
+ },
532
+ {
533
+ "cell_type": "markdown",
534
+ "metadata": {},
535
+ "source": [
536
+ "## \"\"\" Optional \"\"\" - Cloud GPU Pipeline Code\n",
537
+ "If you decide to run this in **Google Colab** (Option 1), you can use the much simpler, fully-integrated code below, since the GPU is fast enough to handle the resizing on-the-fly during training."
538
+ ]
539
+ },
540
+ {
541
+ "cell_type": "code",
542
+ "execution_count": null,
543
+ "metadata": {},
544
+ "outputs": [],
545
+ "source": [
546
+ "\"\"\"\n",
547
+ "# OPTIONAL: Run this ONLY if you are using Google Colab or Kaggle (GPU enabled)\n",
548
+ "from tensorflow.keras.layers import Resizing, Input\n",
549
+ "from tensorflow.keras.applications.mobilenet_v2 import MobileNetV2, preprocess_input\n",
550
+ "from tensorflow.keras.models import Model\n",
551
+ "\n",
552
+ "# Reload original raw data (0-255)\n",
553
+ "(x_train_raw, y_train_raw), (x_test_raw, y_test_raw) = cifar10.load_data()\n",
554
+ "y_train_cat = to_categorical(y_train_raw, 10)\n",
555
+ "y_test_cat = to_categorical(y_test_raw, 10)\n",
556
+ "\n",
557
+ "base_model = MobileNetV2(input_shape=(96, 96, 3), include_top=False, weights='imagenet')\n",
558
+ "base_model.trainable = False\n",
559
+ "\n",
560
+ "# Build pipeline that resizes ON THE FLY (fast on GPU, slow on CPU)\n",
561
+ "inputs = Input(shape=(32, 32, 3))\n",
562
+ "x = Resizing(96, 96)(inputs)\n",
563
+ "x = preprocess_input(x)\n",
564
+ "x = base_model(x)\n",
565
+ "x = GlobalAveragePooling2D()(x)\n",
566
+ "outputs = Dense(10, activation='softmax')(x)\n",
567
+ "\n",
568
+ "gpu_model = Model(inputs, outputs)\n",
569
+ "gpu_model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])\n",
570
+ "\n",
571
+ "# Train on ALL data (50k images) - this will be very fast on a GPU!\n",
572
+ "gpu_history = gpu_model.fit(x_train_raw, y_train_cat, epochs=5, batch_size=128, validation_split=0.2)\n",
573
+ "\"\"\""
574
+ ]
575
+ },
576
+ {
577
+ "cell_type": "markdown",
578
+ "metadata": {},
579
+ "source": [
580
+ "---\n",
581
+ "\n",
582
+ "# **Misclassification Analysis**\n",
583
+ "\n",
584
+ "Even our best model (Custom CNN, 85.3% accuracy) makes confident mistakes. Here are 5 examples where the model was **highly confident but wrong**:\n",
585
+ "\n",
586
+ "| True Label | Predicted | Confidence | Why the model failed |\n",
587
+ "|:-----------|:----------|:-----------|:---------------------|\n",
588
+ "| **Cat** | Dog | 99.5% | Both are furry, four-legged animals with similar color palettes at 32×32 |\n",
589
+ "| **Dog** | Cat | 95.2% | Same confusion pair in reverse — shows this is a systematic weakness |\n",
590
+ "| **Bird** | Frog | 100.0% | Small subject against green/natural background at low resolution |\n",
591
+ "| **Automobile** | Truck | 99.7% | Both are wheeled vehicles with boxy shapes — size (the key discriminator) is lost at 32×32 |\n",
592
+ "| **Airplane** | Ship | 99.8% | Both appear against blue backgrounds (sky vs. water) |\n",
593
+ "\n",
594
+ "**Key insight:** The model relies heavily on **color patterns and rough shapes** rather than fine-grained details, because 32×32 resolution simply does not preserve enough information."
595
+ ]
596
+ },
597
+ {
598
+ "cell_type": "code",
599
+ "execution_count": null,
600
+ "metadata": {},
601
+ "outputs": [],
602
+ "source": [
603
+ "from PIL import Image\n",
604
+ "import matplotlib.pyplot as plt\n",
605
+ "\n",
606
+ "# Display the pre-generated misclassification examples\n",
607
+ "img = Image.open(\"../outputs/misclassified_examples.png\")\n",
608
+ "plt.figure(figsize=(20, 5))\n",
609
+ "plt.imshow(img)\n",
610
+ "plt.axis(\"off\")\n",
611
+ "plt.title(\"Misclassification Examples - Custom CNN\", fontsize=16, fontweight=\"bold\")\n",
612
+ "plt.show()"
613
+ ]
614
+ },
615
+ {
616
+ "cell_type": "markdown",
617
+ "metadata": {},
618
+ "source": [
619
+ "---\n",
620
+ "\n",
621
+ "# **Dataset Bias: What The Model Does Not Know**\n",
622
+ "\n",
623
+ "CIFAR-10 was collected from internet images, predominantly from **Western/English-language sources**. This means our model learned a geographically and culturally *narrow* view of each class:\n",
624
+ "\n",
625
+ "| Class | What the model learned | What it probably has not seen |\n",
626
+ "|:------|:----------------------|:----------------------------|\n",
627
+ "| **Automobile** | American/European car designs | Tuk-tuks, rickshaws, vehicles common in Asia/Africa |\n",
628
+ "| **Truck** | Modern pickup & delivery trucks | Flatbed trucks from rural areas, different regions |\n",
629
+ "| **Ship** | Large vessels in open water | Canoes, kayaks, fishing boats from other cultures |\n",
630
+ "| **Horse** | Western contexts (ranches, paddocks) | Horses in different cultural settings or landscapes |\n",
631
+ "| **Bird** | North American bird species | Tropical birds, exotic species from other continents |\n",
632
+ "\n",
633
+ "> **The model classifies an automobile as seen by English-speaking internet users, not a universal automobile.**"
634
+ ]
635
+ },
636
+ {
637
+ "cell_type": "markdown",
638
+ "metadata": {},
639
+ "source": [
640
+ "---\n",
641
+ "\n",
642
+ "# **Why This Matters**\n",
643
+ "\n",
644
+ "### 4 Types of Bias in Our Model\n",
645
+ "\n",
646
+ "1. **Geographic diversity was never audited**\n",
647
+ " - The training data was never checked for regional representation\n",
648
+ " - The model has a Western-centric worldview in every class\n",
649
+ "\n",
650
+ "2. **Background/context bias**\n",
651
+ " - Objects appear in typical contexts (planes in blue sky, ships in water)\n",
652
+ " - An airplane on a tarmac or a ship in dry dock would likely be harder to classify\n",
653
+ "\n",
654
+ "3. **Color/lighting bias**\n",
655
+ " - Nearly all training images are in daylight\n",
656
+ " - Night images or unusual lighting would degrade performance\n",
657
+ "\n",
658
+ "4. **Resolution bias**\n",
659
+ " - Everything is 32x32 pixels: the model uses color patterns and rough shapes, not details\n",
660
+ " - This is itself a form of **information loss bias**\n",
661
+ "\n",
662
+ "### Other Limitations\n",
663
+ "- **No none of the above**: Upload a banana and the model will still confidently say it is a frog\n",
664
+ "- **Perfect class balance** (6,000/class) is unrealistic, real-world data is imbalanced\n",
665
+ "- **Temporal bias**: Cars, trucks, and ships from the dataset era look different from today"
666
+ ]
667
+ }
668
+ ],
669
+ "metadata": {
670
+ "kernelspec": {
671
+ "display_name": "ironhack.nn",
672
+ "language": "python",
673
+ "name": "python3"
674
+ },
675
+ "language_info": {
676
+ "codemirror_mode": {
677
+ "name": "ipython",
678
+ "version": 3
679
+ },
680
+ "file_extension": ".py",
681
+ "mimetype": "text/x-python",
682
+ "name": "python",
683
+ "nbconvert_exporter": "python",
684
+ "pygments_lexer": "ipython3",
685
+ "version": "3.10.18"
686
+ }
687
+ },
688
+ "nbformat": 4,
689
+ "nbformat_minor": 2
690
+ }
notebooks_knowledge&presentation/terminal_commands.md ADDED
@@ -0,0 +1,76 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Important Terminal Commands
2
+
3
+ This document contains a summary of the most important terminal commands for the CIFAR-10 Image Classification project, organized by workflow.
4
+
5
+ ## 1. Environment Setup
6
+
7
+ ```bash
8
+ # Create and activate conda environment
9
+ conda create -n ironhack.nn python=3.10
10
+ conda activate ironhack.nn
11
+
12
+ # Install dependencies
13
+ pip install -r requirements.txt
14
+ ```
15
+
16
+ ## 2. Training the Model
17
+
18
+ ```bash
19
+ # Run the training notebook (or script)
20
+ jupyter notebook notebooks/
21
+
22
+ # Or run training scripts directly
23
+ python src/train.py # if you have a training script
24
+ ```
25
+
26
+ ## 3. Running the Flask Web App
27
+
28
+ ```bash
29
+ # Activate environment first
30
+ conda activate ironhack.nn
31
+
32
+ # Start the Flask server
33
+ python app/app.py
34
+ # or with a custom port:
35
+ python app/app.py (or flask run --port 5001)
36
+ ```
37
+
38
+ ## 4. Docker (Containerized Deployment)
39
+
40
+ ```bash
41
+ # Build the Docker image
42
+ docker build -t cifar10-classifier .
43
+
44
+ # Run the container
45
+ docker run -p 5001:5001 cifar10-classifier
46
+ ```
47
+
48
+ ## 5. Git (Version Control)
49
+
50
+ ```bash
51
+ # Check status of changes
52
+ git status
53
+
54
+ # Stage and commit changes
55
+ git add .
56
+ git commit -m "your message here"
57
+
58
+ # Push to remote
59
+ git push origin main
60
+ ```
61
+
62
+ ## 6. Useful Utility Commands
63
+
64
+ ```bash
65
+ # Check installed packages
66
+ pip list
67
+
68
+ # Check Python version
69
+ python --version
70
+
71
+ # Check GPU availability (if using TensorFlow)
72
+ python -c "import tensorflow as tf; print(tf.config.list_physical_devices('GPU'))"
73
+
74
+ # Check model file sizes
75
+ ls -lh models/
76
+ ```
outputs/augmentation_examples.png ADDED

Git LFS Details

  • SHA256: 05d0c390839e85bd2ec2cb806f0fc65c8b0214ec5a26ac859404968ee00672d7
  • Pointer size: 131 Bytes
  • Size of remote file: 146 kB
outputs/class_distribution.png ADDED

Git LFS Details

  • SHA256: cdd0128676c0bcec00368a3a3a6d64755f853d6b0e27bab6c5ea308f219f18f0
  • Pointer size: 130 Bytes
  • Size of remote file: 58.8 kB
outputs/custom_cnn_cm.png ADDED

Git LFS Details

  • SHA256: cad04f0718ec7d2f05e440fbaead574a8d36b6e5eeef699b5fbe24964624b361
  • Pointer size: 131 Bytes
  • Size of remote file: 109 kB
outputs/custom_cnn_history.png ADDED

Git LFS Details

  • SHA256: 5d32c3db194c516a878add0571116e94ebe4fec1bb0c997d918fb854d0f47f40
  • Pointer size: 131 Bytes
  • Size of remote file: 143 kB
outputs/custom_cnn_metrics.json ADDED
@@ -0,0 +1,8 @@
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "model": "Custom CNN",
3
+ "loss": 0.44597572088241577,
4
+ "accuracy": 0.8529999852180481,
5
+ "precision": 0.855587007688615,
6
+ "recall": 0.853,
7
+ "f1_score": 0.8512058258364695
8
+ }
outputs/misclassified_examples.png ADDED

Git LFS Details

  • SHA256: cf18026c76146b6839cacfedf5d4e84f51c20ab765cfcf156836646a85d6d392
  • Pointer size: 130 Bytes
  • Size of remote file: 63.3 kB
outputs/model_comparison.png ADDED

Git LFS Details

  • SHA256: 815b29c1dac294a5585affcb9f702408f281091e1fb6b23f6d7b6b620396c8bb
  • Pointer size: 130 Bytes
  • Size of remote file: 55.7 kB
outputs/sample_images.png ADDED

Git LFS Details

  • SHA256: 566b90db4d9691787092f0a7c7725c7e29944c5d075129fabab45237a3c0c7ee
  • Pointer size: 131 Bytes
  • Size of remote file: 160 kB
outputs/transfer_learning_cm.png ADDED

Git LFS Details

  • SHA256: b75fe06af0dca14663de5b4d10c9089761ee13c6b896b3422edc5244e7f0d2b1
  • Pointer size: 131 Bytes
  • Size of remote file: 120 kB
outputs/transfer_learning_history.png ADDED

Git LFS Details

  • SHA256: dc1facdc75cec201242c687df7c9879880e0f4d74ea941c70530a1777107995e
  • Pointer size: 131 Bytes
  • Size of remote file: 108 kB
outputs/transfer_learning_metrics.json ADDED
@@ -0,0 +1,8 @@
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "model": "MobileNetV2 Transfer Learning",
3
+ "loss": 0.5526939630508423,
4
+ "accuracy": 0.8339999914169312,
5
+ "precision": 0.8353133287095834,
6
+ "recall": 0.834,
7
+ "f1_score": 0.8339057892820989
8
+ }
requirements.txt ADDED
@@ -0,0 +1,10 @@
 
 
 
 
 
 
 
 
 
 
 
1
+ tensorflow==2.18.1
2
+ keras==3.6.0
3
+ numpy==2.0.1
4
+ pandas==2.3.1
5
+ matplotlib==3.10.0
6
+ seaborn==0.13.2
7
+ scikit-learn==1.7.1
8
+ Pillow==11.3.0
9
+ flask==3.1.3
10
+ opencv-python-headless==4.13.0.92
src/__init__.py ADDED
@@ -0,0 +1 @@
 
 
1
+ # CNN Image Classification Project - Source Package
src/data_loader.py ADDED
@@ -0,0 +1,113 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ Data Loading and Preprocessing Module for CIFAR-10 Image Classification.
3
+
4
+ This module handles:
5
+ - Loading the CIFAR-10 dataset from Keras
6
+ - Normalizing pixel values to [0, 1]
7
+ - One-hot encoding labels
8
+ - Data augmentation via ImageDataGenerator
9
+ """
10
+
11
+ import numpy as np
12
+ from tensorflow import keras
13
+ from keras.api.datasets import cifar10
14
+ from keras.api.utils import to_categorical
15
+ from keras.src.legacy.preprocessing.image import ImageDataGenerator
16
+
17
+
18
+ # CIFAR-10 class names (in order of label index 0-9)
19
+ CLASS_NAMES = [
20
+ 'airplane', 'automobile', 'bird', 'cat', 'deer',
21
+ 'dog', 'frog', 'horse', 'ship', 'truck'
22
+ ]
23
+
24
+ NUM_CLASSES = 10
25
+ IMG_SHAPE = (32, 32, 3)
26
+
27
+
28
+ def load_cifar10_data():
29
+ """
30
+ Load the CIFAR-10 dataset and return raw train/test splits.
31
+
32
+ Returns:
33
+ tuple: (x_train, y_train), (x_test, y_test)
34
+ - x: uint8 images of shape (N, 32, 32, 3)
35
+ - y: integer labels of shape (N, 1)
36
+ """
37
+ (x_train, y_train), (x_test, y_test) = cifar10.load_data()
38
+ return (x_train, y_train), (x_test, y_test)
39
+
40
+
41
+ def preprocess_data(x_train, y_train, x_test, y_test):
42
+ """
43
+ Normalize images and one-hot encode labels.
44
+
45
+ Args:
46
+ x_train: Training images (uint8).
47
+ y_train: Training labels (int).
48
+ x_test: Test images (uint8).
49
+ y_test: Test labels (int).
50
+
51
+ Returns:
52
+ tuple: (x_train, y_train, x_test, y_test)
53
+ - x: float32 images normalized to [0, 1]
54
+ - y: one-hot encoded labels of shape (N, 10)
55
+ """
56
+ # Normalize pixel values to [0, 1]
57
+ x_train = x_train.astype('float32') / 255.0
58
+ x_test = x_test.astype('float32') / 255.0
59
+
60
+ # One-hot encode labels
61
+ y_train = to_categorical(y_train, NUM_CLASSES)
62
+ y_test = to_categorical(y_test, NUM_CLASSES)
63
+
64
+ return x_train, y_train, x_test, y_test
65
+
66
+
67
+ def create_data_augmentation_generator():
68
+ """
69
+ Create an ImageDataGenerator with augmentation transformations.
70
+
71
+ Augmentations applied:
72
+ - Random rotation up to 15 degrees
73
+ - Random width/height shift up to 10%
74
+ - Random horizontal flip
75
+ - Random zoom up to 10%
76
+
77
+ Returns:
78
+ ImageDataGenerator: configured generator for training data augmentation.
79
+ """
80
+ datagen = ImageDataGenerator(
81
+ rotation_range=15,
82
+ width_shift_range=0.1,
83
+ height_shift_range=0.1,
84
+ horizontal_flip=True,
85
+ zoom_range=0.1,
86
+ fill_mode='nearest'
87
+ )
88
+ return datagen
89
+
90
+
91
+ def get_prepared_data():
92
+ """
93
+ Full pipeline: load CIFAR-10, preprocess, and return ready-to-use data.
94
+
95
+ Returns:
96
+ dict with keys:
97
+ 'x_train', 'y_train', 'x_test', 'y_test': processed arrays
98
+ 'datagen': ImageDataGenerator for augmented training
99
+ """
100
+ (x_train, y_train), (x_test, y_test) = load_cifar10_data()
101
+ x_train, y_train, x_test, y_test = preprocess_data(
102
+ x_train, y_train, x_test, y_test
103
+ )
104
+ datagen = create_data_augmentation_generator()
105
+ datagen.fit(x_train)
106
+
107
+ return {
108
+ 'x_train': x_train,
109
+ 'y_train': y_train,
110
+ 'x_test': x_test,
111
+ 'y_test': y_test,
112
+ 'datagen': datagen
113
+ }
src/evaluate.py ADDED
@@ -0,0 +1,156 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ Evaluation Module for CIFAR-10 Image Classification.
3
+
4
+ Provides functions to:
5
+ - Evaluate model accuracy on test data
6
+ - Generate classification reports (precision, recall, F1)
7
+ - Plot and save confusion matrices
8
+ - Plot training history curves
9
+ """
10
+
11
+ import os
12
+ import numpy as np
13
+ import matplotlib
14
+ matplotlib.use('Agg') # Non-interactive backend for saving plots
15
+ import matplotlib.pyplot as plt
16
+ import seaborn as sns
17
+ from sklearn.metrics import (
18
+ classification_report, confusion_matrix,
19
+ accuracy_score, precision_score, recall_score, f1_score
20
+ )
21
+
22
+ from .data_loader import CLASS_NAMES
23
+
24
+
25
+ def evaluate_model(model, x_test, y_test):
26
+ """
27
+ Evaluate a trained model on the test set.
28
+
29
+ Args:
30
+ model: Trained Keras model.
31
+ x_test: Test images (normalized).
32
+ y_test: Test labels (one-hot encoded).
33
+
34
+ Returns:
35
+ dict: Dictionary with loss, accuracy, precision, recall, f1,
36
+ y_true (int labels), y_pred (int predictions),
37
+ and the full classification_report string.
38
+ """
39
+ # Get test loss and accuracy
40
+ loss, accuracy = model.evaluate(x_test, y_test, verbose=0)
41
+
42
+ # Get predictions
43
+ y_pred_proba = model.predict(x_test, verbose=0)
44
+ y_pred = np.argmax(y_pred_proba, axis=1)
45
+ y_true = np.argmax(y_test, axis=1)
46
+
47
+ # Compute metrics
48
+ precision = precision_score(y_true, y_pred, average='weighted')
49
+ recall = recall_score(y_true, y_pred, average='weighted')
50
+ f1 = f1_score(y_true, y_pred, average='weighted')
51
+
52
+ report = classification_report(y_true, y_pred, target_names=CLASS_NAMES)
53
+
54
+ return {
55
+ 'loss': loss,
56
+ 'accuracy': accuracy,
57
+ 'precision': precision,
58
+ 'recall': recall,
59
+ 'f1_score': f1,
60
+ 'y_true': y_true,
61
+ 'y_pred': y_pred,
62
+ 'classification_report': report
63
+ }
64
+
65
+
66
+ def plot_confusion_matrix(y_true, y_pred, class_names=CLASS_NAMES,
67
+ save_path=None, title='Confusion Matrix'):
68
+ """
69
+ Plot and optionally save a confusion matrix heatmap.
70
+
71
+ Args:
72
+ y_true: True integer labels.
73
+ y_pred: Predicted integer labels.
74
+ class_names: List of class name strings.
75
+ save_path: Optional file path to save the plot.
76
+ title: Plot title.
77
+ """
78
+ cm = confusion_matrix(y_true, y_pred)
79
+
80
+ fig, ax = plt.subplots(figsize=(10, 8))
81
+ sns.heatmap(
82
+ cm, annot=True, fmt='d', cmap='Blues',
83
+ xticklabels=class_names, yticklabels=class_names,
84
+ ax=ax
85
+ )
86
+ ax.set_xlabel('Predicted Label', fontsize=12)
87
+ ax.set_ylabel('True Label', fontsize=12)
88
+ ax.set_title(title, fontsize=14)
89
+ plt.tight_layout()
90
+
91
+ if save_path:
92
+ os.makedirs(os.path.dirname(save_path), exist_ok=True)
93
+ plt.savefig(save_path, dpi=150, bbox_inches='tight')
94
+ print(f"Confusion matrix saved to: {save_path}")
95
+
96
+ plt.close(fig)
97
+
98
+
99
+ def plot_training_history(history, save_path=None, title_prefix=''):
100
+ """
101
+ Plot training and validation accuracy/loss curves.
102
+
103
+ Args:
104
+ history: Keras History object from model.fit().
105
+ save_path: Optional file path to save the plot.
106
+ title_prefix: Optional prefix for plot titles (e.g., 'Custom CNN').
107
+ """
108
+ fig, axes = plt.subplots(1, 2, figsize=(14, 5))
109
+
110
+ # Accuracy plot
111
+ axes[0].plot(history.history['accuracy'], label='Train Accuracy', linewidth=2)
112
+ axes[0].plot(history.history['val_accuracy'], label='Validation Accuracy', linewidth=2)
113
+ axes[0].set_title(f'{title_prefix} Accuracy', fontsize=14)
114
+ axes[0].set_xlabel('Epoch')
115
+ axes[0].set_ylabel('Accuracy')
116
+ axes[0].legend()
117
+ axes[0].grid(True, alpha=0.3)
118
+
119
+ # Loss plot
120
+ axes[1].plot(history.history['loss'], label='Train Loss', linewidth=2)
121
+ axes[1].plot(history.history['val_loss'], label='Validation Loss', linewidth=2)
122
+ axes[1].set_title(f'{title_prefix} Loss', fontsize=14)
123
+ axes[1].set_xlabel('Epoch')
124
+ axes[1].set_ylabel('Loss')
125
+ axes[1].legend()
126
+ axes[1].grid(True, alpha=0.3)
127
+
128
+ plt.tight_layout()
129
+
130
+ if save_path:
131
+ os.makedirs(os.path.dirname(save_path), exist_ok=True)
132
+ plt.savefig(save_path, dpi=150, bbox_inches='tight')
133
+ print(f"Training history saved to: {save_path}")
134
+
135
+ plt.close(fig)
136
+
137
+
138
+ def print_evaluation_summary(metrics, model_name='Model'):
139
+ """
140
+ Print a formatted evaluation summary.
141
+
142
+ Args:
143
+ metrics: Dictionary returned by evaluate_model().
144
+ model_name: Name of the model for display.
145
+ """
146
+ print(f"\n{'='*60}")
147
+ print(f" {model_name} — Evaluation Results")
148
+ print(f"{'='*60}")
149
+ print(f" Test Loss: {metrics['loss']:.4f}")
150
+ print(f" Test Accuracy: {metrics['accuracy']:.4f} ({metrics['accuracy']*100:.2f}%)")
151
+ print(f" Precision: {metrics['precision']:.4f}")
152
+ print(f" Recall: {metrics['recall']:.4f}")
153
+ print(f" F1-Score: {metrics['f1_score']:.4f}")
154
+ print(f"{'='*60}")
155
+ print(f"\nClassification Report:\n")
156
+ print(metrics['classification_report'])
src/model_builder.py ADDED
@@ -0,0 +1,153 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ Model Architecture Builder for CIFAR-10 Image Classification.
3
+
4
+ This module defines:
5
+ - A custom CNN architecture designed for 32x32 CIFAR-10 images
6
+ - A transfer learning model using MobileNetV2 pretrained on ImageNet
7
+ """
8
+
9
+ from keras.api.models import Sequential, Model
10
+ from keras.api.layers import (
11
+ Conv2D, MaxPooling2D, Dense, Dropout, Flatten,
12
+ BatchNormalization, GlobalAveragePooling2D, Input,
13
+ UpSampling2D
14
+ )
15
+ from keras.api.applications import MobileNetV2
16
+
17
+ from .data_loader import NUM_CLASSES, IMG_SHAPE
18
+
19
+
20
+ def build_custom_cnn(input_shape=IMG_SHAPE, num_classes=NUM_CLASSES):
21
+ """
22
+ Build a custom CNN architecture for CIFAR-10 classification.
23
+
24
+ Architecture:
25
+ Block 1: Conv2D(32) -> BatchNorm -> Conv2D(32) -> BatchNorm -> MaxPool -> Dropout(0.25)
26
+ Block 2: Conv2D(64) -> BatchNorm -> Conv2D(64) -> BatchNorm -> MaxPool -> Dropout(0.25)
27
+ Block 3: Conv2D(128) -> BatchNorm -> Conv2D(128) -> BatchNorm -> MaxPool -> Dropout(0.25)
28
+ Dense: Flatten -> Dense(256) -> BatchNorm -> Dropout(0.5) -> Dense(10, softmax)
29
+
30
+ Args:
31
+ input_shape (tuple): Shape of input images (H, W, C). Default: (32, 32, 3)
32
+ num_classes (int): Number of output classes. Default: 10
33
+
34
+ Returns:
35
+ keras.Model: Compiled CNN model
36
+ """
37
+ model = Sequential([
38
+ # --- Block 1 ---
39
+ Conv2D(32, (3, 3), activation='relu', padding='same', input_shape=input_shape),
40
+ BatchNormalization(),
41
+ Conv2D(32, (3, 3), activation='relu', padding='same'),
42
+ BatchNormalization(),
43
+ MaxPooling2D(pool_size=(2, 2)),
44
+ Dropout(0.25),
45
+
46
+ # --- Block 2 ---
47
+ Conv2D(64, (3, 3), activation='relu', padding='same'),
48
+ BatchNormalization(),
49
+ Conv2D(64, (3, 3), activation='relu', padding='same'),
50
+ BatchNormalization(),
51
+ MaxPooling2D(pool_size=(2, 2)),
52
+ Dropout(0.25),
53
+
54
+ # --- Block 3 ---
55
+ Conv2D(128, (3, 3), activation='relu', padding='same'),
56
+ BatchNormalization(),
57
+ Conv2D(128, (3, 3), activation='relu', padding='same'),
58
+ BatchNormalization(),
59
+ MaxPooling2D(pool_size=(2, 2)),
60
+ Dropout(0.25),
61
+
62
+ # --- Classifier ---
63
+ Flatten(),
64
+ Dense(256, activation='relu'),
65
+ BatchNormalization(),
66
+ Dropout(0.5),
67
+ Dense(num_classes, activation='softmax')
68
+ ])
69
+
70
+ return model
71
+
72
+
73
+ def build_transfer_learning_model(input_shape=IMG_SHAPE, num_classes=NUM_CLASSES,
74
+ upscale=True):
75
+ """
76
+ Build a transfer learning model using MobileNetV2.
77
+
78
+ MobileNetV2 is chosen because:
79
+ - Lightweight and efficient (good for training on CPU/limited GPU)
80
+ - Strong feature extraction from ImageNet pretraining
81
+ - Keras docs: https://keras.io/api/applications/mobilenet/#mobilenetv2-function
82
+ - Requires minimum 96×96×3 input for efficient performance
83
+
84
+ Architecture (GPU / upscale=True):
85
+ Input(32,32,3) -> UpSampling2D(3x) -> MobileNetV2(frozen) ->
86
+ GlobalAveragePooling2D -> Dense(256) -> Dropout(0.5) -> Dense(10, softmax)
87
+
88
+ Architecture (CPU / upscale=False):
89
+ Input(96,96,3) [pre-resized with cv2] -> MobileNetV2(frozen) ->
90
+ GlobalAveragePooling2D -> Dense(256) -> Dropout(0.5) -> Dense(10, softmax)
91
+
92
+ Args:
93
+ input_shape (tuple): Shape of input images.
94
+ GPU: (32, 32, 3) — UpSampling2D handles resize.
95
+ CPU: (96, 96, 3) — images pre-resized with cv2.
96
+ num_classes (int): Number of output classes. Default: 10
97
+ upscale (bool): If True, adds UpSampling2D(3,3) inside the model
98
+ (GPU version, 03_transfer_learning_gpu.py).
99
+ If False, skips it — images must already be 96×96
100
+ (CPU version, 03_transfer_learning_cpu.py).
101
+
102
+ Returns:
103
+ tuple: (keras.Model, base_model) — full model and MobileNetV2 base
104
+ """
105
+ # Input layer
106
+ inputs = Input(shape=input_shape)
107
+
108
+ if upscale:
109
+ # GPU version: upscale 32×32 → 96×96 inside the graph (runs each batch)
110
+ x = UpSampling2D(size=(3, 3))(inputs)
111
+ mobilenet_input_shape = (96, 96, 3)
112
+ else:
113
+ # CPU version: images already 96×96 (pre-resized with cv2 before training)
114
+ # This eliminates per-batch upscaling — the main CPU bottleneck
115
+ x = inputs
116
+ mobilenet_input_shape = input_shape
117
+
118
+ # Load MobileNetV2 pretrained on ImageNet, without top classification layers
119
+ # Minimum efficient input: 96×96×3 (see Keras docs link above)
120
+ base_model = MobileNetV2(
121
+ input_shape=mobilenet_input_shape,
122
+ include_top=False,
123
+ weights='imagenet'
124
+ )
125
+ # Freeze all base model layers (we only train the new head)
126
+ base_model.trainable = False
127
+
128
+ x = base_model(x, training=False)
129
+
130
+ # Custom classification head
131
+ x = GlobalAveragePooling2D()(x)
132
+ x = Dense(256, activation='relu')(x)
133
+ x = Dropout(0.5)(x)
134
+ outputs = Dense(num_classes, activation='softmax')(x)
135
+
136
+ model = Model(inputs=inputs, outputs=outputs)
137
+
138
+ return model, base_model
139
+
140
+
141
+ def get_model_summary(model):
142
+ """
143
+ Get a string summary of the model architecture.
144
+
145
+ Args:
146
+ model: Keras model
147
+
148
+ Returns:
149
+ str: Model summary string
150
+ """
151
+ summary_lines = []
152
+ model.summary(print_fn=lambda x: summary_lines.append(x))
153
+ return '\n'.join(summary_lines)
src/train.py ADDED
@@ -0,0 +1,147 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ Training Pipeline for CIFAR-10 Image Classification.
3
+
4
+ This module provides functions to compile and train Keras models
5
+ with appropriate callbacks for preventing overfitting.
6
+ """
7
+
8
+ import os
9
+ from keras.api.optimizers import Adam
10
+ from keras.api.callbacks import EarlyStopping, ModelCheckpoint, ReduceLROnPlateau
11
+
12
+
13
+ # Default training configuration
14
+ DEFAULT_CONFIG = {
15
+ 'learning_rate': 0.001,
16
+ 'batch_size': 64,
17
+ 'epochs': 100, # max epochs (early stopping will cut short)
18
+ 'early_stopping_patience': 10,
19
+ 'reduce_lr_patience': 5,
20
+ 'reduce_lr_factor': 0.5,
21
+ 'min_lr': 1e-6,
22
+ }
23
+
24
+
25
+ def compile_model(model, learning_rate=DEFAULT_CONFIG['learning_rate']):
26
+ """
27
+ Compile a Keras model with Adam optimizer and categorical crossentropy loss.
28
+
29
+ Args:
30
+ model: Keras model to compile.
31
+ learning_rate (float): Initial learning rate for Adam optimizer.
32
+
33
+ Returns:
34
+ Compiled Keras model.
35
+ """
36
+ model.compile(
37
+ optimizer=Adam(learning_rate=learning_rate),
38
+ loss='categorical_crossentropy',
39
+ metrics=['accuracy']
40
+ )
41
+ return model
42
+
43
+
44
+ def get_callbacks(model_save_path, patience=DEFAULT_CONFIG['early_stopping_patience']):
45
+ """
46
+ Create a list of training callbacks for robust training.
47
+
48
+ Callbacks:
49
+ - EarlyStopping: Stops training if val_loss doesn't improve for `patience` epochs.
50
+ Restores the best weights.
51
+ - ModelCheckpoint: Saves the best model based on val_loss.
52
+ - ReduceLROnPlateau: Reduces learning rate by half if val_loss plateaus for 5 epochs.
53
+
54
+ Args:
55
+ model_save_path (str): File path to save the best model.
56
+ patience (int): Number of epochs to wait before early stopping.
57
+
58
+ Returns:
59
+ list: List of Keras callback instances.
60
+ """
61
+ # Ensure the directory exists
62
+ os.makedirs(os.path.dirname(model_save_path), exist_ok=True)
63
+
64
+ callbacks = [
65
+ EarlyStopping(
66
+ monitor='val_loss',
67
+ patience=patience,
68
+ restore_best_weights=True,
69
+ verbose=1
70
+ ),
71
+ ModelCheckpoint(
72
+ filepath=model_save_path,
73
+ monitor='val_loss',
74
+ save_best_only=True,
75
+ verbose=1
76
+ ),
77
+ ReduceLROnPlateau(
78
+ monitor='val_loss',
79
+ factor=DEFAULT_CONFIG['reduce_lr_factor'],
80
+ patience=DEFAULT_CONFIG['reduce_lr_patience'],
81
+ min_lr=DEFAULT_CONFIG['min_lr'],
82
+ verbose=1
83
+ )
84
+ ]
85
+ return callbacks
86
+
87
+
88
+ def train_model(model, x_train, y_train, datagen=None,
89
+ validation_split=0.1, x_val=None, y_val=None,
90
+ batch_size=DEFAULT_CONFIG['batch_size'],
91
+ epochs=DEFAULT_CONFIG['epochs'],
92
+ callbacks=None):
93
+ """
94
+ Train a Keras model with optional data augmentation.
95
+
96
+ If `datagen` is provided, training uses the augmentation generator.
97
+ Otherwise, trains directly on the raw data.
98
+
99
+ Args:
100
+ model: Compiled Keras model.
101
+ x_train: Training images array.
102
+ y_train: Training labels array (one-hot).
103
+ datagen: Optional ImageDataGenerator for data augmentation.
104
+ validation_split: Fraction of training data for validation (used if x_val is None).
105
+ x_val: Optional separate validation images.
106
+ y_val: Optional separate validation labels.
107
+ batch_size (int): Training batch size.
108
+ epochs (int): Maximum number of training epochs.
109
+ callbacks (list): List of Keras callbacks.
110
+
111
+ Returns:
112
+ keras.callbacks.History: Training history object.
113
+ """
114
+ # Determine validation data
115
+ if x_val is not None and y_val is not None:
116
+ validation_data = (x_val, y_val)
117
+ else:
118
+ # Split training data for validation
119
+ split_idx = int(len(x_train) * (1 - validation_split))
120
+ x_val = x_train[split_idx:]
121
+ y_val = y_train[split_idx:]
122
+ x_train = x_train[:split_idx]
123
+ y_train = y_train[:split_idx]
124
+ validation_data = (x_val, y_val)
125
+
126
+ if datagen is not None:
127
+ # Train with data augmentation
128
+ history = model.fit(
129
+ datagen.flow(x_train, y_train, batch_size=batch_size),
130
+ epochs=epochs,
131
+ validation_data=validation_data,
132
+ callbacks=callbacks or [],
133
+ steps_per_epoch=len(x_train) // batch_size,
134
+ verbose=1
135
+ )
136
+ else:
137
+ # Train without augmentation
138
+ history = model.fit(
139
+ x_train, y_train,
140
+ batch_size=batch_size,
141
+ epochs=epochs,
142
+ validation_data=validation_data,
143
+ callbacks=callbacks or [],
144
+ verbose=1
145
+ )
146
+
147
+ return history
test_images/.gitkeep ADDED
File without changes
test_images/test_CNN/airplane.png ADDED

Git LFS Details

  • SHA256: c9ada17647e68dbbaa174fc3bfb49955a37621174afbc300d037086908b88464
  • Pointer size: 129 Bytes
  • Size of remote file: 3.66 kB
test_images/test_CNN/automobile.png ADDED

Git LFS Details

  • SHA256: 2362c09b8f26a398daac5f4edc5d04739c8b95b7adddad4e49dd6012d9e8fa1d
  • Pointer size: 129 Bytes
  • Size of remote file: 2.48 kB
test_images/test_CNN/bird.png ADDED

Git LFS Details

  • SHA256: 198eb271be6338cce82000d36be9cb5678cd456a63d36d0c6e8115ab3d0cd4c4
  • Pointer size: 129 Bytes
  • Size of remote file: 4.01 kB
test_images/test_CNN/cat 444.png ADDED

Git LFS Details

  • SHA256: 1e52ca5701c6704c7a60677148c6f9e2fafa6755634302f5148d144bbbf939b5
  • Pointer size: 129 Bytes
  • Size of remote file: 3.42 kB
test_images/test_CNN/cat.png ADDED

Git LFS Details

  • SHA256: 0c5f0483c1ec28548c9195c1bd0e572f7697da3848cfdadef26399dbd2531e2c
  • Pointer size: 129 Bytes
  • Size of remote file: 4.1 kB
test_images/test_CNN/dog.png ADDED

Git LFS Details

  • SHA256: 1093e0979f16bd9d3bc20dc6061826a0e2807319c53c80e12b461cdbe6130429
  • Pointer size: 129 Bytes
  • Size of remote file: 3.97 kB
test_images/test_CNN/frog.png ADDED

Git LFS Details

  • SHA256: b1d148630b4debf310dfc7387252f1ec0309d41094e96f23fe7da8c2d77b0fda
  • Pointer size: 129 Bytes
  • Size of remote file: 4.25 kB
test_images/test_CNN/horse 21.png ADDED

Git LFS Details

  • SHA256: b64f5263a12cda5eb4f93ee1d2835b958d83c5137f8b31da59cb05aa62d739de
  • Pointer size: 129 Bytes
  • Size of remote file: 3.52 kB
test_images/test_CNN/horse.png ADDED

Git LFS Details

  • SHA256: 59be67042af5b833e1c0e5c648f68990e00c90a09b091d1e7ae5ffba1f24a3f3
  • Pointer size: 129 Bytes
  • Size of remote file: 3.98 kB
test_images/test_CNN/truck.png ADDED

Git LFS Details

  • SHA256: 9d710481b1a337074c9492977bf03b1becec40f7ef26f1d4aec13808dd024449
  • Pointer size: 129 Bytes
  • Size of remote file: 4.06 kB