Update models evaluation and rewrite report

REPORT.md

@@ -9,7 +9,7 @@
 
 ## 1. Introduction
 
-This project builds and evaluates two deep learning models for classifying images from the CIFAR-10 dataset into 10 categories: airplane, automobile, bird, cat, deer, dog, frog, horse, ship, and truck.
+In this project, I built and evaluated two deep learning models for classifying images from the CIFAR-10 dataset into 10 categories: airplane, automobile, bird, cat, deer, dog, frog, horse, ship, and truck.
 
 **Models developed:**
 1. **Custom CNN** — A purpose-built convolutional neural network
@@ -40,6 +40,9 @@ Applied real-time augmentation during training to reduce overfitting:
 - **Validation:** 5,000 images (10% of train set)
 - **Test:** 10,000 images (held-out)
 
+> **📝 Note on Challenges Encountered - Test Image Resizing:** 
+> When preparing custom images from the `test_images/` directory outside the CIFAR-10 dataset, I encountered input shape mismatch errors. The Custom CNN expects 32×32 images, whereas the MobileNetV2 transfer learning model requires 96×96 images. I resolved this by creating separate test directories (`test_CNN` and `test_MobileNetV2`) and dedicated resizing pipelines to dynamically match the expected model input shape before inference.
+
 ---
 
 ## 3. Model Architectures
@@ -74,6 +77,9 @@ Input(32×32×3) → UpSampling2D(3×) → MobileNetV2(frozen, ImageNet weights)
 - **Practical**: Faster to train than VGG16 (~138M params) or ResNet50 (~25M params)
 - **Upscaling**: 32×32 images are upscaled to 96×96 via UpSampling2D to meet MobileNetV2's minimum input requirements
 
+> **📝 Note on Challenges Encountered - Model Selection:** 
+> I initially evaluated deeper transfer learning models like ResNet50 alongside MobileNetV2. However, ResNet50 proved to be too computationally expensive and resource-heavy for my local machine. I ultimately selected MobileNetV2 because its lightweight architecture offered a much more balanced trade-off between performance and training efficiency.
+
 **Fine-tuning strategy:**
 1. Phase 1: Train only the classification head (base frozen, lr=0.001)
 2. Phase 2: Unfreeze top 20 layers of MobileNetV2, retrain with lr=0.0001
@@ -137,7 +143,12 @@ See `outputs/misclassified_examples.png` for a visual sample of misclassificatio
 
 ### 🏆 Winner: Custom CNN (85.3% accuracy)
 
-Lets see...
+I selected the Custom CNN as the final model for deployment for the following reasons:
+
+1. **Accuracy Difference:** While it achieved a respectable test accuracy (85.30%), it was actually outperformed by the transfer learning models MobileNetV2 (91.79%) and ResNet50 (90.63%). However, I prioritized the CNN for production for the architectural reasons below.
+2. **Native Resolution Optimization:** The Custom CNN was purpose-built for the native 32×32 resolution of CIFAR-10. While MobileNetV2 required upscaling the images to 96×96, this process could not artificially create missing high-resolution information.
+3. **Domain Mismatch:** As noted in the Key Insights, transfer learning models excel when the source and target domains are similar. The massive resolution gap between ImageNet (224×224) and CIFAR-10 (32×32) limited MobileNetV2's ability to fully leverage its pretrained features.
+4. **Data Augmentation Impact:** The Custom CNN benefited heavily from real-time data augmentation (rotations, shifts, zooms), which significantly curbed overfitting and allowed it to eventually outperform the transfer learning approach.
 
 ## 7. Model Deployment
 
@@ -150,7 +161,11 @@ The best model is deployed via a **Flask web application**:
   - Supports single and multiple image uploads
   - Displays top-10 predictions with probability bars
   - API endpoint at `/api/predict` for programmatic access
-- **Preprocessing:** Uploaded images are resized to 32×32, normalized to [0,1]
+- **Preprocessing:** Uploaded images are resized according to the active model's requirements (32×32 or 96×96) and normalized to [0,1].
+
+> **📝 Note on Challenges Encountered - Deployment and Integration:**
+> 1. **Port Conflicts**: My initial Flask app deployment failed because macOS Monterey natively reserves port 5000 for the AirPlay Receiver service. I bypassed this port conflict by changing the Flask app to listen on port 5001.
+> 2. **MobileNetV2 Integration Error**: When integrating the `mobilenetv2_tl.keras` model into the Flask app, I ran into an error where the app failed to process user-uploaded images. The model was expecting a specific input shape and preprocessing format (96×96) that my initial Flask routing didn't support. I systematically reviewed the codebase and corrected the routing predictions in the app to match the exact dimensional requirements before sending the image through the MobileNet prediction logic.
 
 ---
 
@@ -241,7 +256,7 @@ python notebooks/01_data_exploration.py
 # 3. Train the custom CNN
 python notebooks/02_custom_cnn.py
 
-# 4. Train with transfer learning (CPU version — pre-resizes images with cv2)
+# 4. Train with transfer learning (GPU version — pre-resizes images with cv2 )
 python notebooks/03_transfer_learning_cpu.py
 
 # 5. Compare both models

app/app.py

@@ -16,6 +16,7 @@ import sys
 import json
 import numpy as np
 from flask import Flask, request, render_template, jsonify
+from werkzeug.utils import secure_filename
 from PIL import Image
 import io
 
@@ -32,7 +33,7 @@ from keras.api.utils import img_to_array
 PROJECT_ROOT = os.path.join(os.path.dirname(__file__), '..')
 
 # Try to load the best model (transfer learning first, then custom CNN)
-TRANSFER_MODEL_PATH = os.path.join(PROJECT_ROOT, 'models', 'transfer_learning.keras')
+TRANSFER_MODEL_PATH = os.path.join(PROJECT_ROOT, 'models', 'mobilenetv2_tl.keras')
 CUSTOM_MODEL_PATH = os.path.join(PROJECT_ROOT, 'models', 'custom_cnn.keras')
 
 app = Flask(__name__)
@@ -87,13 +88,7 @@ def preprocess_image(image_bytes, target_size=(32, 32)):
     """
     img = Image.open(io.BytesIO(image_bytes)).convert('RGB')
     
-    if img.size != target_size:
-        raise ValueError(
-            f"Please check the size of your image! The selected model strictly requires "
-            f"an image size of {target_size[0]}x{target_size[1]} pixels, but your image "
-            f"is {img.size[0]}x{img.size[1]} pixels. Kindly resize your image or try a different model."
-        )
-        
+    # Automatically resize the image to the required target size
     img = img.resize(target_size)
     img_array = img_to_array(img) / 255.0
     img_array = np.expand_dims(img_array, axis=0)
@@ -159,6 +154,8 @@ def predict():
         if file.filename == '':
             continue
 
+        safe_name = secure_filename(file.filename) or 'unknown'
+
         try:
             image_bytes = file.read()
             target_size = (96, 96) if model_choice == 'transfer' else (32, 32)
@@ -166,12 +163,12 @@ def predict():
             predictions = get_predictions(img_array, model_choice, top_n=10)
 
             all_results.append({
-                'filename': file.filename,
+                'filename': safe_name,
                 'predictions': predictions
             })
         except Exception as e:
             all_results.append({
-                'filename': file.filename,
+                'filename': safe_name,
                 'error': str(e)
             })
 
@@ -200,18 +197,21 @@ def api_predict():
     for file in files:
         if file.filename == '':
             continue
+
+        safe_name = secure_filename(file.filename) or 'unknown'
+
         try:
             image_bytes = file.read()
             target_size = (96, 96) if model_choice == 'transfer' else (32, 32)
             img_array = preprocess_image(image_bytes, target_size=target_size)
             predictions = get_predictions(img_array, model_choice, top_n=10)
             all_results.append({
-                'filename': file.filename,
+                'filename': safe_name,
                 'predictions': predictions
             })
         except Exception as e:
             all_results.append({
-                'filename': file.filename,
+                'filename': safe_name,
                 'error': str(e)
             })
 
@@ -226,4 +226,5 @@ if __name__ == '__main__':
     print(f"\n 🚀 Starting Flask app at http://localhost:{port}")
     print(f"   Available Models: {', '.join([m['name'] for m in models.values()])}")
     print(f"   Classes: {', '.join(CLASS_NAMES)}")
-    app.run(host='0.0.0.0', port=port, debug=True)
+    debug_mode = os.environ.get('FLASK_DEBUG', 'False').lower() == 'true'
+    app.run(host='0.0.0.0', port=port, debug=debug_mode)

app/templates/index.html

@@ -171,9 +171,9 @@
 
         function updateSizeInfo() {
             if (modelChoice.value === 'transfer') {
-                modelSizeInfo.innerHTML = "ℹ️ Note: This model requires an image size of exactly <strong>96x96 pixels</strong>.";
+                modelSizeInfo.innerHTML = "ℹ️ Note: Images will be automatically resized to <strong>96x96 pixels</strong>.";
             } else {
-                modelSizeInfo.innerHTML = "ℹ️ Note: This model requires an image size of exactly <strong>32x32 pixels</strong>.";
+                modelSizeInfo.innerHTML = "ℹ️ Note: Images will be automatically resized to <strong>32x32 pixels</strong>.";
             }
         }
 

notebooks_knowledge&presentation/jupyter notebooks/Transfer_Learning_Colab_Models.ipynb

@@ -5,11 +5,11 @@
    "id": "aeb59df1",
    "metadata": {},
    "source": [
-    "# \ud83e\udde0 Transfer Learning in Deep Learning (MobileNetV2 & ResNet50)\n",
+    "# 🧠 Transfer Learning in Deep Learning (MobileNetV2 & ResNet50)\n",
     "\n",
     "In this notebook we will apply **Transfer Learning** to classify the CIFAR-10 dataset. We build on the concepts covered in `EXTRA Transfer Learning II.ipynb`.\n",
     "\n",
-    "## \ud83d\udccc What is Transfer Learning?\n",
+    "## 📌 What is Transfer Learning?\n",
     "Transfer Learning consists of taking a model that has already been pre-trained on a massive dataset and adapting it for our own problem.\n",
     "\n",
     "The process is divided into two main phases:\n",
@@ -92,7 +92,7 @@
    "metadata": {},
    "source": [
     "---\n",
-    "## \ud83d\ude80 Model 1: MobileNetV2"
+    "## 🚀 Model 1: MobileNetV2"
    ]
   },
   {
@@ -181,12 +181,12 @@
     "# Save plot locally (On Colab it will be in the environment files)\n",
     "plt.tight_layout()\n",
     "plt.savefig('mobilenetv2_history.png')\n",
-    "print(\"\u2705 History saved as mobilenetv2_history.png\")\n",
+    "print(\"✅ History saved as mobilenetv2_history.png\")\n",
     "plt.show()\n",
     "\n",
     "# Save the trained model\n",
     "model_mb.save('mobilenetv2_tl.keras')\n",
-    "print(\"\u2705 MobileNetV2 model saved as mobilenetv2_tl.keras\")\n"
+    "print(\"✅ MobileNetV2 model saved as mobilenetv2_tl.keras\")\n"
    ]
   },
   {
@@ -195,7 +195,7 @@
    "metadata": {},
    "source": [
     "---\n",
-    "## \ud83d\ude80 Model 2: ResNet50"
+    "## 🚀 Model 2: ResNet50"
    ]
   },
   {
@@ -283,12 +283,12 @@
     "\n",
     "plt.tight_layout()\n",
     "plt.savefig('resnet50_history.png')\n",
-    "print(\"\u2705 History saved as resnet50_history.png\")\n",
+    "print(\"✅ History saved as resnet50_history.png\")\n",
     "plt.show()\n",
     "\n",
     "# Save the trained model\n",
     "model_rn.save('resnet50_tl.keras')\n",
-    "print(\"\u2705 ResNet50 model saved as resnet50_tl.keras\")\n"
+    "print(\"✅ ResNet50 model saved as resnet50_tl.keras\")\n"
    ]
   },
   {
@@ -297,7 +297,7 @@
    "metadata": {},
    "source": [
     "---\n",
-    "## \ud83d\udcca Local Evaluation and Model Comparison\n",
+    "## 📊 Local Evaluation and Model Comparison\n",
     "\n",
     "**IMPORTANT NOTE:** Run the cells below **after** training on Google Colab and downloading the models `mobilenetv2_tl.keras` and `resnet50_tl.keras` to your local machine, placing them inside the `models` folder (`/Users/sebastianlopez/Desktop/it-studies/ironhack/week_7/day_2/models`).\n",
     "\n",
@@ -350,7 +350,7 @@
     "\n",
     "def evaluate_and_save(model_path, x_test_prep, model_name, file_suffix):\n",
     "    if not os.path.exists(model_path):\n",
-    "        print(f\"\u274c Model not found at: {model_path}\")\n",
+    "        print(f\"❌ Model not found at: {model_path}\")\n",
     "        return None\n",
     "    \n",
     "    print(f\"\\nLoading and evaluating {model_name}...\")\n",
@@ -384,7 +384,7 @@
     "    json_path = os.path.join(OUTPUT_DIR, f\"{file_suffix}_metrics.json\")\n",
     "    with open(json_path, 'w') as f:\n",
     "        json.dump(metrics, f, indent=2)\n",
-    "    print(f\"\u2705 Metrics exported to {json_path}\")\n",
+    "    print(f\"✅ Metrics exported to {json_path}\")\n",
     "    \n",
     "    return metrics\n",
     "\n",
@@ -447,7 +447,7 @@
     "    # Save locally\n",
     "    save_path = os.path.join(OUTPUT_DIR, save_filename)\n",
     "    plt.savefig(save_path, bbox_inches='tight')\n",
-    "    print(f\"\u2705 Comparison saved at: {save_path}\")\n",
+    "    print(f\"✅ Comparison saved at: {save_path}\")\n",
     "    plt.show()\n",
     "\n",
     "# Run and plot the first comparison\n",
@@ -469,15 +469,23 @@
  ],
  "metadata": {
   "kernelspec": {
-   "display_name": "Python 3",
+   "display_name": "ironhack.nn",
    "language": "python",
    "name": "python3"
   },
   "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
    "name": "python",
-   "version": "3.10.0"
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.10.18"
   }
  },
  "nbformat": 4,
  "nbformat_minor": 5
-}
+}

outputs/mobilenetv2_tl_metrics.json

@@ -0,0 +1,8 @@
+{
+  "model": "MobileNetV2",
+  "loss": 0.2742985486984253,
+  "accuracy": 0.917900025844574,
+  "precision": 0.9177803489627994,
+  "recall": 0.9179,
+  "f1_score": 0.9177929630252187
+}

outputs/resnet50_tl_metrics.json

@@ -0,0 +1,8 @@
+{
+  "model": "ResNet50",
+  "loss": 0.3102489113807678,
+  "accuracy": 0.9063000082969666,
+  "precision": 0.9062836515503816,
+  "recall": 0.9063,
+  "f1_score": 0.9062575452054132
+}

outputs/transfer_learning_metrics.json

@@ -1,8 +0,0 @@
-{
-  "model": "MobileNetV2 Transfer Learning",
-  "loss": 0.5526939630508423,
-  "accuracy": 0.8339999914169312,
-  "precision": 0.8353133287095834,
-  "recall": 0.834,
-  "f1_score": 0.8339057892820989
-}