Transfer Learning Update: Synchronized models and added comparative documentation

models/transfer_learning.keras

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:5516031beafea75d838dc61097ee0bd08eef101f2e9c661495f512ab15cac7e5
-size 19458191

notebooks_knowledge&presentation/jupyter notebooks/Transfer_Learning_Colab_Models.ipynb

@@ -0,0 +1,483 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "aeb59df1",
+   "metadata": {},
+   "source": [
+    "# \ud83e\udde0 Transfer Learning in Deep Learning (MobileNetV2 & ResNet50)\n",
+    "\n",
+    "In this notebook we will apply **Transfer Learning** to classify the CIFAR-10 dataset. We build on the concepts covered in `EXTRA Transfer Learning II.ipynb`.\n",
+    "\n",
+    "## \ud83d\udccc What is Transfer Learning?\n",
+    "Transfer Learning consists of taking a model that has already been pre-trained on a massive dataset and adapting it for our own problem.\n",
+    "\n",
+    "The process is divided into two main phases:\n",
+    "\n",
+    "1. **Feature Extraction:**\n",
+    "   - We import the pre-trained base model without the final classification layer (`include_top=False`).\n",
+    "   - We **freeze** the base model weights (`trainable = False`) so they are not modified.\n",
+    "   - We add our own classification \"head\" (Dense layers) at the end.\n",
+    "   - We train **only** our new classification head. The base model acts solely as a generic \"feature extractor\".\n",
+    "\n",
+    "2. **Fine-Tuning:**\n",
+    "   - Once our classification head has learned initial patterns, we **unfreeze** some or all of the top layers of the base model.\n",
+    "   - We retrain using a **very small Learning Rate**.\n",
+    "   - This allows the model to subtly adapt its prior knowledge (weights) to the specific characteristics of our images.\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "88788953",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# 1. Setup and imports for training (Google Colab)\n",
+    "import os\n",
+    "import json\n",
+    "import numpy as np\n",
+    "import matplotlib.pyplot as plt\n",
+    "\n",
+    "import tensorflow as tf\n",
+    "from tensorflow import keras\n",
+    "from keras.datasets import cifar10\n",
+    "from keras.utils import to_categorical\n",
+    "from keras.layers import Dense, GlobalAveragePooling2D, Input, Dropout\n",
+    "from keras.models import Sequential, Model\n",
+    "from keras.callbacks import EarlyStopping\n",
+    "from keras.optimizers import Adam\n",
+    "\n",
+    "# Configuration\n",
+    "BATCH_SIZE = 64\n",
+    "CLASS_NAMES = ['airplane', 'automobile', 'bird', 'cat', 'deer','dog', 'frog', 'horse', 'ship', 'truck']\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "8f1bf95a",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# 2. Load and prepare data\n",
+    "(x_train_raw, y_train_raw), (x_test_raw, y_test_raw) = cifar10.load_data()\n",
+    "\n",
+    "# One-hot encode labels\n",
+    "y_train = to_categorical(y_train_raw, 10)\n",
+    "y_test = to_categorical(y_test_raw, 10)\n",
+    "\n",
+    "# Split training set for validation\n",
+    "from sklearn.model_selection import train_test_split\n",
+    "x_train_split, x_val, y_train_split, y_val = train_test_split(\n",
+    "    x_train_raw, y_train, test_size=0.2, random_state=42\n",
+    ")\n",
+    "\n",
+    "# Resize for Transfer Learning models: 32x32 -> 96x96 (bilinear interpolation)\n",
+    "# Pre-trained models need larger images to extract features correctly.\n",
+    "# We pre-resize once here to avoid doing it on every batch during training.\n",
+    "x_train_split_96 = tf.image.resize(x_train_split, [96, 96]).numpy()\n",
+    "x_val_96 = tf.image.resize(x_val, [96, 96]).numpy()\n",
+    "\n",
+    "print(f\"X_train_split shape (original): {x_train_split.shape}\")\n",
+    "print(f\"X_train_split_96 shape (resized): {x_train_split_96.shape}\")\n",
+    "print(f\"X_val shape: {x_val.shape}\")\n",
+    "print(f\"X_val_96 shape (resized): {x_val_96.shape}\")\n",
+    "print(\"Data ready for training.\")\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "fafb0d92",
+   "metadata": {},
+   "source": [
+    "---\n",
+    "## \ud83d\ude80 Model 1: MobileNetV2"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "15187f8d",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# MobileNetV2 - PHASE 1: Feature Extraction\n",
+    "from keras.applications import MobileNetV2\n",
+    "from keras.applications.mobilenet_v2 import preprocess_input\n",
+    "\n",
+    "# Preprocess the RESIZED images using the model-specific function\n",
+    "x_train_mb = preprocess_input(x_train_split_96.astype('float32'))\n",
+    "x_val_mb = preprocess_input(x_val_96.astype('float32'))\n",
+    "\n",
+    "# Load base model with 96x96 input and FREEZE it\n",
+    "base_model_mb = MobileNetV2(input_shape=(96, 96, 3), include_top=False, weights='imagenet')\n",
+    "base_model_mb.trainable = False\n",
+    "\n",
+    "# Build our architecture\n",
+    "model_mb = Sequential([\n",
+    "    base_model_mb,\n",
+    "    GlobalAveragePooling2D(),\n",
+    "    Dropout(0.2), # To prevent overfitting\n",
+    "    Dense(10, activation='softmax')\n",
+    "])\n",
+    "\n",
+    "model_mb.compile(optimizer=Adam(learning_rate=0.001),\n",
+    "                 loss='categorical_crossentropy',\n",
+    "                 metrics=['accuracy'])\n",
+    "\n",
+    "print(\"Starting Phase 1: Feature Extraction...\")\n",
+    "history_mb_fe = model_mb.fit(x_train_mb, y_train_split, epochs=5, validation_data=(x_val_mb, y_val), batch_size=BATCH_SIZE)\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "17584d82",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# MobileNetV2 - PHASE 2: Fine-Tuning\n",
+    "\n",
+    "# Unfreeze the base model\n",
+    "base_model_mb.trainable = True\n",
+    "\n",
+    "# Recompile with a MUCH SMALLER Learning Rate (1e-5 or 1e-4) to avoid destroying weights\n",
+    "model_mb.compile(optimizer=Adam(learning_rate=1e-5),\n",
+    "                 loss='categorical_crossentropy',\n",
+    "                 metrics=['accuracy'])\n",
+    "\n",
+    "print(\"Starting Phase 2: Fine-Tuning...\")\n",
+    "early_stop = EarlyStopping(monitor='val_loss', patience=3, restore_best_weights=True)\n",
+    "\n",
+    "history_mb_ft = model_mb.fit(x_train_mb, y_train_split, epochs=10, \n",
+    "                             validation_data=(x_val_mb, y_val), \n",
+    "                             batch_size=BATCH_SIZE, callbacks=[early_stop])\n",
+    "\n",
+    "# Combine training histories to plot everything together\n",
+    "acc_mbV2 = history_mb_fe.history['accuracy'] + history_mb_ft.history['accuracy']\n",
+    "val_acc_mbV2 = history_mb_fe.history['val_accuracy'] + history_mb_ft.history['val_accuracy']\n",
+    "loss_mbV2 = history_mb_fe.history['loss'] + history_mb_ft.history['loss']\n",
+    "val_loss_mbV2 = history_mb_fe.history['val_loss'] + history_mb_ft.history['val_loss']\n",
+    "\n",
+    "# Save training plot\n",
+    "fig, axes = plt.subplots(1, 2, figsize=(14, 5))\n",
+    "axes[0].plot(acc_mbV2, label='Train Accuracy')\n",
+    "axes[0].plot(val_acc_mbV2, label='Validation Accuracy')\n",
+    "axes[0].axvline(len(history_mb_fe.history['accuracy'])-1, color='r', linestyle='--', label='Start Fine Tuning')\n",
+    "axes[0].set_title('MobileNetV2 Accuracy')\n",
+    "axes[0].set_xlabel('Epochs')\n",
+    "axes[0].set_ylabel('Accuracy')\n",
+    "axes[0].legend()\n",
+    "\n",
+    "axes[1].plot(loss_mbV2, label='Train Loss')\n",
+    "axes[1].plot(val_loss_mbV2, label='Validation Loss')\n",
+    "axes[1].axvline(len(history_mb_fe.history['loss'])-1, color='r', linestyle='--', label='Start Fine Tuning')\n",
+    "axes[1].set_title('MobileNetV2 Loss')\n",
+    "axes[1].set_xlabel('Epochs')\n",
+    "axes[1].set_ylabel('Loss')\n",
+    "axes[1].legend()\n",
+    "\n",
+    "# Save plot locally (On Colab it will be in the environment files)\n",
+    "plt.tight_layout()\n",
+    "plt.savefig('mobilenetv2_history.png')\n",
+    "print(\"\u2705 History saved as mobilenetv2_history.png\")\n",
+    "plt.show()\n",
+    "\n",
+    "# Save the trained model\n",
+    "model_mb.save('mobilenetv2_tl.keras')\n",
+    "print(\"\u2705 MobileNetV2 model saved as mobilenetv2_tl.keras\")\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "84ad9b75",
+   "metadata": {},
+   "source": [
+    "---\n",
+    "## \ud83d\ude80 Model 2: ResNet50"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "1e8efedc",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# ResNet50 - PHASE 1: Feature Extraction\n",
+    "from keras.applications import ResNet50\n",
+    "from keras.applications.resnet50 import preprocess_input as preprocess_input_rn\n",
+    "\n",
+    "# Preprocess the RESIZED images using the ResNet50-specific function\n",
+    "x_train_rn = preprocess_input_rn(x_train_split_96.astype('float32'))\n",
+    "x_val_rn = preprocess_input_rn(x_val_96.astype('float32'))\n",
+    "\n",
+    "# Load base model with 96x96 input and FREEZE it\n",
+    "base_model_rn = ResNet50(input_shape=(96, 96, 3), include_top=False, weights='imagenet')\n",
+    "base_model_rn.trainable = False\n",
+    "\n",
+    "# Build our architecture\n",
+    "model_rn = Sequential([\n",
+    "    base_model_rn,\n",
+    "    GlobalAveragePooling2D(),\n",
+    "    Dropout(0.2), # Added to prevent overfitting\n",
+    "    Dense(10, activation='softmax')\n",
+    "])\n",
+    "\n",
+    "model_rn.compile(optimizer=Adam(learning_rate=0.001),\n",
+    "                 loss='categorical_crossentropy',\n",
+    "                 metrics=['accuracy'])\n",
+    "\n",
+    "print(\"Starting Phase 1: Feature Extraction for ResNet50...\")\n",
+    "history_rn_fe = model_rn.fit(x_train_rn, y_train_split, epochs=5, validation_data=(x_val_rn, y_val), batch_size=BATCH_SIZE)\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "eca746e6",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# ResNet50 - PHASE 2: Fine-Tuning\n",
+    "\n",
+    "# Unfreeze the base model\n",
+    "base_model_rn.trainable = True\n",
+    "\n",
+    "# Recompile with a MUCH SMALLER Learning Rate\n",
+    "model_rn.compile(optimizer=Adam(learning_rate=1e-5),\n",
+    "                 loss='categorical_crossentropy',\n",
+    "                 metrics=['accuracy'])\n",
+    "\n",
+    "print(\"Starting Phase 2: Fine-Tuning for ResNet50...\")\n",
+    "early_stop_rn = EarlyStopping(monitor='val_loss', patience=3, restore_best_weights=True)\n",
+    "\n",
+    "history_rn_ft = model_rn.fit(x_train_rn, y_train_split, epochs=10, \n",
+    "                             validation_data=(x_val_rn, y_val), \n",
+    "                             batch_size=BATCH_SIZE, callbacks=[early_stop_rn])\n",
+    "\n",
+    "# Combine training histories\n",
+    "acc_rn50 = history_rn_fe.history['accuracy'] + history_rn_ft.history['accuracy']\n",
+    "val_acc_rn50 = history_rn_fe.history['val_accuracy'] + history_rn_ft.history['val_accuracy']\n",
+    "loss_rn50 = history_rn_fe.history['loss'] + history_rn_ft.history['loss']\n",
+    "val_loss_rn50 = history_rn_fe.history['val_loss'] + history_rn_ft.history['val_loss']\n",
+    "\n",
+    "# Save training plot\n",
+    "fig, axes = plt.subplots(1, 2, figsize=(14, 5))\n",
+    "axes[0].plot(acc_rn50, label='Train Accuracy')\n",
+    "axes[0].plot(val_acc_rn50, label='Validation Accuracy')\n",
+    "axes[0].axvline(len(history_rn_fe.history['accuracy'])-1, color='r', linestyle='--', label='Start Fine Tuning')\n",
+    "axes[0].set_title('ResNet50 Accuracy')\n",
+    "axes[0].set_xlabel('Epochs')\n",
+    "axes[0].set_ylabel('Accuracy')\n",
+    "axes[0].legend()\n",
+    "\n",
+    "axes[1].plot(loss_rn50, label='Train Loss')\n",
+    "axes[1].plot(val_loss_rn50, label='Validation Loss')\n",
+    "axes[1].axvline(len(history_rn_fe.history['loss'])-1, color='r', linestyle='--', label='Start Fine Tuning')\n",
+    "axes[1].set_title('ResNet50 Loss')\n",
+    "axes[1].set_xlabel('Epochs')\n",
+    "axes[1].set_ylabel('Loss')\n",
+    "axes[1].legend()\n",
+    "\n",
+    "plt.tight_layout()\n",
+    "plt.savefig('resnet50_history.png')\n",
+    "print(\"\u2705 History saved as resnet50_history.png\")\n",
+    "plt.show()\n",
+    "\n",
+    "# Save the trained model\n",
+    "model_rn.save('resnet50_tl.keras')\n",
+    "print(\"\u2705 ResNet50 model saved as resnet50_tl.keras\")\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "37b2be11",
+   "metadata": {},
+   "source": [
+    "---\n",
+    "## \ud83d\udcca Local Evaluation and Model Comparison\n",
+    "\n",
+    "**IMPORTANT NOTE:** Run the cells below **after** training on Google Colab and downloading the models `mobilenetv2_tl.keras` and `resnet50_tl.keras` to your local machine, placing them inside the `models` folder (`/Users/sebastianlopez/Desktop/it-studies/ironhack/week_7/day_2/models`).\n",
+    "\n",
+    "In this section we will:\n",
+    "1. Load the Custom CNN and compute its metrics on the Test dataset.\n",
+    "2. Load MobileNetV2 and ResNet50, preprocessing Test images according to each model's requirements.\n",
+    "3. Evaluate and compute `accuracy`, `precision`, `recall` and `f1-score`.\n",
+    "4. Export the new models' metrics as `.json`.\n",
+    "5. Plot and save comparisons (`model_comparison_mobilenetv2_cnn.png` and `model_comparison_resnet50_cnn.png`).\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "5b0eb49d",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# 3. Local Evaluation and JSON Export\n",
+    "import os\n",
+    "import json\n",
+    "import numpy as np\n",
+    "import matplotlib.pyplot as plt\n",
+    "import tensorflow as tf\n",
+    "from sklearn.metrics import classification_report, accuracy_score, precision_score, recall_score, f1_score\n",
+    "\n",
+    "from tensorflow import keras\n",
+    "from keras.datasets import cifar10\n",
+    "from keras.utils import to_categorical\n",
+    "from keras.applications.mobilenet_v2 import preprocess_input as mb_prep\n",
+    "from keras.applications.resnet50 import preprocess_input as rn_prep\n",
+    "\n",
+    "# Load test data\n",
+    "(_, _), (x_test_raw, y_test_raw) = cifar10.load_data()\n",
+    "y_test = to_categorical(y_test_raw, 10)\n",
+    "\n",
+    "# Preprocessing for the test set (same as during training)\n",
+    "x_test_custom = x_test_raw.astype('float32') / 255.0\n",
+    "\n",
+    "# Resize test images to 96x96 for Transfer Learning models\n",
+    "# (same resize applied to training data)\n",
+    "x_test_96 = tf.image.resize(x_test_raw, [96, 96]).numpy()\n",
+    "x_test_mb = mb_prep(x_test_96.astype('float32'))\n",
+    "x_test_rn = rn_prep(x_test_96.astype('float32'))\n",
+    "\n",
+    "# Paths - Make sure the files exist locally\n",
+    "MODEL_DIR = '/Users/sebastianlopez/Desktop/it-studies/ironhack/week_7/day_2/models'\n",
+    "OUTPUT_DIR = '/Users/sebastianlopez/Desktop/it-studies/ironhack/week_7/day_2/outputs'\n",
+    "os.makedirs(OUTPUT_DIR, exist_ok=True)\n",
+    "\n",
+    "def evaluate_and_save(model_path, x_test_prep, model_name, file_suffix):\n",
+    "    if not os.path.exists(model_path):\n",
+    "        print(f\"\u274c Model not found at: {model_path}\")\n",
+    "        return None\n",
+    "    \n",
+    "    print(f\"\\nLoading and evaluating {model_name}...\")\n",
+    "    model = keras.models.load_model(model_path)\n",
+    "    loss, accuracy = model.evaluate(x_test_prep, y_test, verbose=0)\n",
+    "    \n",
+    "    y_pred_proba = model.predict(x_test_prep, verbose=0)\n",
+    "    y_pred = np.argmax(y_pred_proba, axis=1)\n",
+    "    y_true = np.argmax(y_test, axis=1)\n",
+    "    \n",
+    "    prec = precision_score(y_true, y_pred, average='weighted', zero_division=0)\n",
+    "    rec = recall_score(y_true, y_pred, average='weighted', zero_division=0)\n",
+    "    f1 = f1_score(y_true, y_pred, average='weighted', zero_division=0)\n",
+    "    \n",
+    "    print(f\"  Test Loss:     {loss:.4f}\")\n",
+    "    print(f\"  Test Accuracy: {accuracy:.4f}\")\n",
+    "    print(f\"  Precision:     {prec:.4f}\")\n",
+    "    print(f\"  Recall:        {rec:.4f}\")\n",
+    "    print(f\"  F1-Score:      {f1:.4f}\")\n",
+    "    \n",
+    "    metrics = {\n",
+    "        \"model\": model_name,\n",
+    "        \"loss\": float(loss),\n",
+    "        \"accuracy\": float(accuracy),\n",
+    "        \"precision\": float(prec),\n",
+    "        \"recall\": float(rec),\n",
+    "        \"f1_score\": float(f1)\n",
+    "    }\n",
+    "    \n",
+    "    # Export JSON\n",
+    "    json_path = os.path.join(OUTPUT_DIR, f\"{file_suffix}_metrics.json\")\n",
+    "    with open(json_path, 'w') as f:\n",
+    "        json.dump(metrics, f, indent=2)\n",
+    "    print(f\"\u2705 Metrics exported to {json_path}\")\n",
+    "    \n",
+    "    return metrics\n",
+    "\n",
+    "# Load metrics for all 3 models\n",
+    "custom_cnn_path = os.path.join(MODEL_DIR, 'custom_cnn.keras')\n",
+    "mb_path = os.path.join(MODEL_DIR, 'mobilenetv2_tl.keras')\n",
+    "rn_path = os.path.join(MODEL_DIR, 'resnet50_tl.keras')\n",
+    "\n",
+    "metrics_custom = evaluate_and_save(custom_cnn_path, x_test_custom, 'Custom CNN', 'custom_cnn')\n",
+    "metrics_mb = evaluate_and_save(mb_path, x_test_mb, 'MobileNetV2', 'mobilenetv2_tl')\n",
+    "metrics_rn = evaluate_and_save(rn_path, x_test_rn, 'ResNet50', 'resnet50_tl')\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "ea5d7c75",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# 4. Comparison Chart: Custom CNN vs MobileNetV2\n",
+    "def plot_comparison(m1, m2, save_filename, title):\n",
+    "    if m1 is None or m2 is None:\n",
+    "        print(f\"Missing data to plot {title}\")\n",
+    "        return\n",
+    "    \n",
+    "    labels = ['Accuracy', 'Precision', 'Recall', 'F1-Score']\n",
+    "    values_m1 = [m1['accuracy'], m1['precision'], m1['recall'], m1['f1_score']]\n",
+    "    values_m2 = [m2['accuracy'], m2['precision'], m2['recall'], m2['f1_score']]\n",
+    "    \n",
+    "    x = np.arange(len(labels))\n",
+    "    width = 0.35\n",
+    "    \n",
+    "    fig, ax = plt.subplots(figsize=(8, 5))\n",
+    "    rects1 = ax.bar(x - width/2, values_m1, width, label=m1['model'], color='skyblue')\n",
+    "    rects2 = ax.bar(x + width/2, values_m2, width, label=m2['model'], color='salmon')\n",
+    "    \n",
+    "    ax.set_ylabel('Scores')\n",
+    "    ax.set_title(title, fontweight='bold', fontsize=14)\n",
+    "    ax.set_xticks(x)\n",
+    "    ax.set_xticklabels(labels)\n",
+    "    ax.legend(loc='lower right')\n",
+    "    ax.set_ylim(0, 1.1)\n",
+    "    \n",
+    "    # Add numeric values on top of bars\n",
+    "    for rects in [rects1, rects2]:\n",
+    "        for rect in rects:\n",
+    "            height = rect.get_height()\n",
+    "            ax.annotate(f'{height:.3f}',\n",
+    "                        xy=(rect.get_x() + rect.get_width() / 2, height),\n",
+    "                        xytext=(0, 3), \n",
+    "                        textcoords=\"offset points\",\n",
+    "                        ha='center', va='bottom', fontsize=9)\n",
+    "            \n",
+    "    # Add box with Loss info\n",
+    "    textstr = f\"Test Loss {m1['model']}: {m1['loss']:.4f}\\nTest Loss {m2['model']}: {m2['loss']:.4f}\"\n",
+    "    props = dict(boxstyle='round', facecolor='wheat', alpha=0.5)\n",
+    "    ax.text(1.05, 0.5, textstr, transform=ax.transAxes, fontsize=10, verticalalignment='center', bbox=props)\n",
+    "            \n",
+    "    # Save locally\n",
+    "    save_path = os.path.join(OUTPUT_DIR, save_filename)\n",
+    "    plt.savefig(save_path, bbox_inches='tight')\n",
+    "    print(f\"\u2705 Comparison saved at: {save_path}\")\n",
+    "    plt.show()\n",
+    "\n",
+    "# Run and plot the first comparison\n",
+    "plot_comparison(metrics_custom, metrics_mb, 'model_comparison_mobilenetv2_cnn.png', 'Comparison: Custom CNN vs MobileNetV2')\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "b28030be",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# 5. Comparison Chart: Custom CNN vs ResNet50\n",
+    "# Run and plot the second comparison\n",
+    "plot_comparison(metrics_custom, metrics_rn, 'model_comparison_resnet50_cnn.png', 'Comparison: Custom CNN vs ResNet50')\n"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "name": "python",
+   "version": "3.10.0"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

notebooks_knowledge&presentation/src folder.md

@@ -0,0 +1,65 @@
+In software development and data science, creating a src (short for source) folder is a widespread best practice for organizing a project.
+
+Looking at your current CIFAR-10 image classification project, here is why you and most developers put core Python files (like 
+
+data_loader.py
+, 
+
+model_builder.py
+, etc.) into a src/ directory:
+
+1. Separation of Concerns
+A real-world project has many different types of files:
+
+Documentation: 
+
+README.md
+, 
+
+REPORT.md
+Environment settings: 
+
+requirements.txt
+, 
+
+.gitignore
+Exploration & Presentation: Jupyter Notebooks (in your notebooks_knowledge&presentation/ folder)
+User Interface/App: Streamlit or Flask code (in your app/ folder)
+The Core Logic: The actual python scripts that do the heavy lifting.
+By placing the core logic inside the src/ folder, you perfectly separate the "engine" of your project from the documentation, the UI, and the configuration.
+
+2. Modularity and Reusability (Easy Importing)
+By having a src/ folder (with an init.py file inside), Python treats it as a module. This means you can easily reuse the exact same code in different places without copying and pasting.
+
+For example, whether you are experimenting in a Jupyter Notebook or running your web interface in 
+
+app/app.py
+, you can effortlessly use your data loader like this:
+
+python
+from src.data_loader import load_data
+from src.model_builder import build_model
+This is much cleaner than defining standard functions repeatedly in different notebooks.
+
+3. A Clean Root Directory
+Without a src/ folder, your root project directory would be flooded with python files: 
+
+train.py
+, 
+
+evaluate.py
+, 
+
+data_loader.py
+, app.py, mixed alongside 
+
+requirements.txt
+ and .git. By storing them in src/, your root directory stays clean and easy to read for any other developer (or yourself in the future) who lands on your project's GitHub page.
+
+4. Preventing Import Conflicts (The src Layout)
+In standard Python packaging, using a src layout forces you to test your code exactly how it will be imported by others. It prevents accidental import errors that can happen when your application code is sitting right next to your top-level scripts.
+
+Summary: 
+
+You create src/ to house the core "engine" of your machine learning workflow (loading data, building models, training, evaluating), allowing you to easily import those functions into both your Jupyter Notebooks and your app.py while keeping your workspace organized!
+