{ "cells": [ { "cell_type": "markdown", "id": "3de9762e", "metadata": {}, "source": [ "# πŸ† Pest Classification Tournament: Finding the Ultimate Prediction Model\n", "\n", "## Project Overview\n", "\n", "Welcome to our comprehensive **tournament-style classification analysis** designed to identify the best performing model for predicting pest occurrence (`NEW_CATCHES`). This notebook represents the culmination of our data science pipeline, where we pit multiple machine learning algorithms against each other in a rigorous competition.\n", "\n", "### Tournament Structure\n", "\n", "Our tournament is organized into three distinct phases:\n", "\n", "1. **πŸ₯Š Standard Classifiers Tournament**: Traditional machine learning algorithms compete\n", " - RandomForestClassifier\n", " - XGBoostClassifier \n", " - LightGBMClassifier\n", "\n", "2. **🧠 Deep Learning Tournament**: Advanced neural networks battle for supremacy\n", " - LSTM (Long Short-Term Memory)\n", " - GRU (Gated Recurrent Unit)\n", " - CNN-LSTM Hybrid\n", "\n", "3. **πŸ… Grand Finale**: The champions from each category face off to determine the ultimate winner\n", "\n", "### Key Principles\n", "\n", "- **No Data Leakage**: All splits maintain chronological order (shuffle=False)\n", "- **Robust Imbalance Handling**: Class weighting and optimal threshold tuning\n", "- **Interactive Visualizations**: All plots use Plotly for enhanced exploration\n", "- **Comprehensive Evaluation**: F1-score, AUC, precision, recall, and confusion matrices\n", "\n", "Let the tournament begin! πŸš€" ] }, { "cell_type": "markdown", "id": "d7c96b62", "metadata": {}, "source": [ "## πŸ“¦ Global Setup and Library Imports\n", "\n", "We begin by importing all necessary libraries for our complete analysis. This includes traditional machine learning libraries, deep learning frameworks, visualization tools, and utility packages." ] }, { "cell_type": "code", "execution_count": 13, "id": "a3c1141c", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Requirement already satisfied: tensorflow==2.15.0 in c:\\users\\parha\\anaconda3\\envs\\pest_pred_specific\\lib\\site-packages (2.15.0)\n", "Requirement already satisfied: tensorflow-intel==2.15.0 in c:\\users\\parha\\anaconda3\\envs\\pest_pred_specific\\lib\\site-packages (from tensorflow==2.15.0) (2.15.0)\n", "Requirement already satisfied: absl-py>=1.0.0 in c:\\users\\parha\\anaconda3\\envs\\pest_pred_specific\\lib\\site-packages (from tensorflow-intel==2.15.0->tensorflow==2.15.0) (2.3.0)\n", "Requirement already satisfied: astunparse>=1.6.0 in c:\\users\\parha\\anaconda3\\envs\\pest_pred_specific\\lib\\site-packages (from tensorflow-intel==2.15.0->tensorflow==2.15.0) (1.6.3)\n", "Requirement already satisfied: flatbuffers>=23.5.26 in c:\\users\\parha\\anaconda3\\envs\\pest_pred_specific\\lib\\site-packages (from tensorflow-intel==2.15.0->tensorflow==2.15.0) (25.2.10)\n", "Requirement already satisfied: gast!=0.5.0,!=0.5.1,!=0.5.2,>=0.2.1 in c:\\users\\parha\\anaconda3\\envs\\pest_pred_specific\\lib\\site-packages (from tensorflow-intel==2.15.0->tensorflow==2.15.0) (0.4.0)\n", "Requirement already satisfied: google-pasta>=0.1.1 in c:\\users\\parha\\anaconda3\\envs\\pest_pred_specific\\lib\\site-packages (from tensorflow-intel==2.15.0->tensorflow==2.15.0) (0.2.0)\n", "Requirement already satisfied: h5py>=2.9.0 in c:\\users\\parha\\anaconda3\\envs\\pest_pred_specific\\lib\\site-packages (from tensorflow-intel==2.15.0->tensorflow==2.15.0) (3.14.0)\n", "Requirement already satisfied: libclang>=13.0.0 in c:\\users\\parha\\anaconda3\\envs\\pest_pred_specific\\lib\\site-packages (from tensorflow-intel==2.15.0->tensorflow==2.15.0) (18.1.1)\n", "Requirement already satisfied: ml-dtypes~=0.2.0 in c:\\users\\parha\\anaconda3\\envs\\pest_pred_specific\\lib\\site-packages (from tensorflow-intel==2.15.0->tensorflow==2.15.0) (0.2.0)\n", "Requirement already satisfied: numpy<2.0.0,>=1.23.5 in c:\\users\\parha\\anaconda3\\envs\\pest_pred_specific\\lib\\site-packages (from tensorflow-intel==2.15.0->tensorflow==2.15.0) (1.26.4)\n", "Requirement already satisfied: opt-einsum>=2.3.2 in c:\\users\\parha\\anaconda3\\envs\\pest_pred_specific\\lib\\site-packages (from tensorflow-intel==2.15.0->tensorflow==2.15.0) (3.4.0)\n", "Requirement already satisfied: packaging in c:\\users\\parha\\anaconda3\\envs\\pest_pred_specific\\lib\\site-packages (from tensorflow-intel==2.15.0->tensorflow==2.15.0) (25.0)\n", "Requirement already satisfied: protobuf!=4.21.0,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<5.0.0dev,>=3.20.3 in c:\\users\\parha\\anaconda3\\envs\\pest_pred_specific\\lib\\site-packages (from tensorflow-intel==2.15.0->tensorflow==2.15.0) (4.25.8)\n", "Requirement already satisfied: setuptools in c:\\users\\parha\\anaconda3\\envs\\pest_pred_specific\\lib\\site-packages (from tensorflow-intel==2.15.0->tensorflow==2.15.0) (78.1.1)\n", "Requirement already satisfied: six>=1.12.0 in c:\\users\\parha\\anaconda3\\envs\\pest_pred_specific\\lib\\site-packages (from tensorflow-intel==2.15.0->tensorflow==2.15.0) (1.17.0)\n", "Requirement already satisfied: termcolor>=1.1.0 in c:\\users\\parha\\anaconda3\\envs\\pest_pred_specific\\lib\\site-packages (from tensorflow-intel==2.15.0->tensorflow==2.15.0) (3.1.0)\n", "Requirement already satisfied: typing-extensions>=3.6.6 in c:\\users\\parha\\anaconda3\\envs\\pest_pred_specific\\lib\\site-packages (from tensorflow-intel==2.15.0->tensorflow==2.15.0) (4.14.0)\n", "Requirement already satisfied: wrapt<1.15,>=1.11.0 in c:\\users\\parha\\anaconda3\\envs\\pest_pred_specific\\lib\\site-packages (from tensorflow-intel==2.15.0->tensorflow==2.15.0) (1.14.1)\n", "Requirement already satisfied: tensorflow-io-gcs-filesystem>=0.23.1 in c:\\users\\parha\\anaconda3\\envs\\pest_pred_specific\\lib\\site-packages (from tensorflow-intel==2.15.0->tensorflow==2.15.0) (0.31.0)\n", "Requirement already satisfied: grpcio<2.0,>=1.24.3 in c:\\users\\parha\\anaconda3\\envs\\pest_pred_specific\\lib\\site-packages (from tensorflow-intel==2.15.0->tensorflow==2.15.0) (1.73.0)\n", "Requirement already satisfied: tensorboard<2.16,>=2.15 in c:\\users\\parha\\anaconda3\\envs\\pest_pred_specific\\lib\\site-packages (from tensorflow-intel==2.15.0->tensorflow==2.15.0) (2.15.2)\n", "Requirement already satisfied: tensorflow-estimator<2.16,>=2.15.0 in c:\\users\\parha\\anaconda3\\envs\\pest_pred_specific\\lib\\site-packages (from tensorflow-intel==2.15.0->tensorflow==2.15.0) (2.15.0)\n", "Requirement already satisfied: keras<2.16,>=2.15.0 in c:\\users\\parha\\anaconda3\\envs\\pest_pred_specific\\lib\\site-packages (from tensorflow-intel==2.15.0->tensorflow==2.15.0) (2.15.0)\n", "Requirement already satisfied: google-auth<3,>=1.6.3 in c:\\users\\parha\\anaconda3\\envs\\pest_pred_specific\\lib\\site-packages (from tensorboard<2.16,>=2.15->tensorflow-intel==2.15.0->tensorflow==2.15.0) (2.40.3)\n", "Requirement already satisfied: google-auth-oauthlib<2,>=0.5 in c:\\users\\parha\\anaconda3\\envs\\pest_pred_specific\\lib\\site-packages (from tensorboard<2.16,>=2.15->tensorflow-intel==2.15.0->tensorflow==2.15.0) (1.0.0)\n", "Requirement already satisfied: markdown>=2.6.8 in c:\\users\\parha\\anaconda3\\envs\\pest_pred_specific\\lib\\site-packages (from tensorboard<2.16,>=2.15->tensorflow-intel==2.15.0->tensorflow==2.15.0) (3.8)\n", "Requirement already satisfied: requests<3,>=2.21.0 in c:\\users\\parha\\anaconda3\\envs\\pest_pred_specific\\lib\\site-packages (from tensorboard<2.16,>=2.15->tensorflow-intel==2.15.0->tensorflow==2.15.0) (2.32.3)\n", "Requirement already satisfied: tensorboard-data-server<0.8.0,>=0.7.0 in c:\\users\\parha\\anaconda3\\envs\\pest_pred_specific\\lib\\site-packages (from tensorboard<2.16,>=2.15->tensorflow-intel==2.15.0->tensorflow==2.15.0) (0.7.2)\n", "Requirement already satisfied: werkzeug>=1.0.1 in c:\\users\\parha\\anaconda3\\envs\\pest_pred_specific\\lib\\site-packages (from tensorboard<2.16,>=2.15->tensorflow-intel==2.15.0->tensorflow==2.15.0) (3.1.3)\n", "Requirement already satisfied: cachetools<6.0,>=2.0.0 in c:\\users\\parha\\anaconda3\\envs\\pest_pred_specific\\lib\\site-packages (from google-auth<3,>=1.6.3->tensorboard<2.16,>=2.15->tensorflow-intel==2.15.0->tensorflow==2.15.0) (5.5.2)\n", "Requirement already satisfied: pyasn1-modules>=0.2.1 in c:\\users\\parha\\anaconda3\\envs\\pest_pred_specific\\lib\\site-packages (from google-auth<3,>=1.6.3->tensorboard<2.16,>=2.15->tensorflow-intel==2.15.0->tensorflow==2.15.0) (0.4.2)\n", "Requirement already satisfied: rsa<5,>=3.1.4 in c:\\users\\parha\\anaconda3\\envs\\pest_pred_specific\\lib\\site-packages (from google-auth<3,>=1.6.3->tensorboard<2.16,>=2.15->tensorflow-intel==2.15.0->tensorflow==2.15.0) (4.9.1)\n", "Requirement already satisfied: requests-oauthlib>=0.7.0 in c:\\users\\parha\\anaconda3\\envs\\pest_pred_specific\\lib\\site-packages (from google-auth-oauthlib<2,>=0.5->tensorboard<2.16,>=2.15->tensorflow-intel==2.15.0->tensorflow==2.15.0) (2.0.0)\n", "Requirement already satisfied: charset-normalizer<4,>=2 in c:\\users\\parha\\anaconda3\\envs\\pest_pred_specific\\lib\\site-packages (from requests<3,>=2.21.0->tensorboard<2.16,>=2.15->tensorflow-intel==2.15.0->tensorflow==2.15.0) (3.1.0)\n", "Requirement already satisfied: idna<4,>=2.5 in c:\\users\\parha\\anaconda3\\envs\\pest_pred_specific\\lib\\site-packages (from requests<3,>=2.21.0->tensorboard<2.16,>=2.15->tensorflow-intel==2.15.0->tensorflow==2.15.0) (3.10)\n", "Requirement already satisfied: urllib3<3,>=1.21.1 in c:\\users\\parha\\anaconda3\\envs\\pest_pred_specific\\lib\\site-packages (from requests<3,>=2.21.0->tensorboard<2.16,>=2.15->tensorflow-intel==2.15.0->tensorflow==2.15.0) (2.4.0)\n", "Requirement already satisfied: certifi>=2017.4.17 in c:\\users\\parha\\anaconda3\\envs\\pest_pred_specific\\lib\\site-packages (from requests<3,>=2.21.0->tensorboard<2.16,>=2.15->tensorflow-intel==2.15.0->tensorflow==2.15.0) (2025.4.26)\n", "Requirement already satisfied: pyasn1>=0.1.3 in c:\\users\\parha\\anaconda3\\envs\\pest_pred_specific\\lib\\site-packages (from rsa<5,>=3.1.4->google-auth<3,>=1.6.3->tensorboard<2.16,>=2.15->tensorflow-intel==2.15.0->tensorflow==2.15.0) (0.6.1)\n", "Requirement already satisfied: wheel<1.0,>=0.23.0 in c:\\users\\parha\\anaconda3\\envs\\pest_pred_specific\\lib\\site-packages (from astunparse>=1.6.0->tensorflow-intel==2.15.0->tensorflow==2.15.0) (0.45.1)\n", "Requirement already satisfied: oauthlib>=3.0.0 in c:\\users\\parha\\anaconda3\\envs\\pest_pred_specific\\lib\\site-packages (from requests-oauthlib>=0.7.0->google-auth-oauthlib<2,>=0.5->tensorboard<2.16,>=2.15->tensorflow-intel==2.15.0->tensorflow==2.15.0) (3.2.2)\n", "Requirement already satisfied: MarkupSafe>=2.1.1 in c:\\users\\parha\\anaconda3\\envs\\pest_pred_specific\\lib\\site-packages (from werkzeug>=1.0.1->tensorboard<2.16,>=2.15->tensorflow-intel==2.15.0->tensorflow==2.15.0) (3.0.2)\n", "Note: you may need to restart the kernel to use updated packages.\n", "Requirement already satisfied: scikit-learn in c:\\users\\parha\\anaconda3\\envs\\pest_pred_specific\\lib\\site-packages (1.7.0)\n", "Requirement already satisfied: pandas in c:\\users\\parha\\anaconda3\\envs\\pest_pred_specific\\lib\\site-packages (2.2.2)\n", "Requirement already satisfied: numpy in c:\\users\\parha\\anaconda3\\envs\\pest_pred_specific\\lib\\site-packages (1.26.4)\n", "Requirement already satisfied: plotly in c:\\users\\parha\\anaconda3\\envs\\pest_pred_specific\\lib\\site-packages (5.22.0)\n", "Requirement already satisfied: scipy>=1.8.0 in c:\\users\\parha\\anaconda3\\envs\\pest_pred_specific\\lib\\site-packages (from scikit-learn) (1.15.3)\n", "Requirement already satisfied: joblib>=1.2.0 in c:\\users\\parha\\anaconda3\\envs\\pest_pred_specific\\lib\\site-packages (from scikit-learn) (1.5.1)\n", "Requirement already satisfied: threadpoolctl>=3.1.0 in c:\\users\\parha\\anaconda3\\envs\\pest_pred_specific\\lib\\site-packages (from scikit-learn) (3.6.0)\n", "Requirement already satisfied: python-dateutil>=2.8.2 in c:\\users\\parha\\anaconda3\\envs\\pest_pred_specific\\lib\\site-packages (from pandas) (2.9.0.post0)\n", "Requirement already satisfied: pytz>=2020.1 in c:\\users\\parha\\anaconda3\\envs\\pest_pred_specific\\lib\\site-packages (from pandas) (2025.2)\n", "Requirement already satisfied: tzdata>=2022.7 in c:\\users\\parha\\anaconda3\\envs\\pest_pred_specific\\lib\\site-packages (from pandas) (2025.2)\n", "Requirement already satisfied: tenacity>=6.2.0 in c:\\users\\parha\\anaconda3\\envs\\pest_pred_specific\\lib\\site-packages (from plotly) (8.5.0)\n", "Requirement already satisfied: packaging in c:\\users\\parha\\anaconda3\\envs\\pest_pred_specific\\lib\\site-packages (from plotly) (25.0)\n", "Requirement already satisfied: six>=1.5 in c:\\users\\parha\\anaconda3\\envs\\pest_pred_specific\\lib\\site-packages (from python-dateutil>=2.8.2->pandas) (1.17.0)\n", "Note: you may need to restart the kernel to use updated packages.\n", "Requirement already satisfied: xgboost in c:\\users\\parha\\anaconda3\\envs\\pest_pred_specific\\lib\\site-packages (2.0.3)\n", "Requirement already satisfied: lightgbm in c:\\users\\parha\\anaconda3\\envs\\pest_pred_specific\\lib\\site-packages (4.3.0)\n", "Requirement already satisfied: numpy in c:\\users\\parha\\anaconda3\\envs\\pest_pred_specific\\lib\\site-packages (from xgboost) (1.26.4)\n", "Requirement already satisfied: scipy in c:\\users\\parha\\anaconda3\\envs\\pest_pred_specific\\lib\\site-packages (from xgboost) (1.15.3)\n", "Note: you may need to restart the kernel to use updated packages.\n" ] } ], "source": [ "%pip install tensorflow==2.15.0\n", "%pip install scikit-learn pandas numpy plotly\n", "%pip install xgboost lightgbm" ] }, { "cell_type": "code", "execution_count": 14, "id": "da6bf0e8", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "βœ… All libraries imported successfully!\n", "πŸ“Š Pandas version: 2.2.2\n", "πŸ€– TensorFlow version: 2.15.0\n", "πŸ“ˆ Plotly available for interactive visualizations\n" ] } ], "source": [ "# Core Data Science Libraries\n", "import pandas as pd\n", "import numpy as np\n", "import warnings\n", "warnings.filterwarnings('ignore')\n", "\n", "# Machine Learning - Core\n", "from sklearn.model_selection import train_test_split, GridSearchCV, TimeSeriesSplit\n", "from sklearn.preprocessing import StandardScaler\n", "from sklearn.metrics import (\n", " classification_report, confusion_matrix, roc_auc_score, \n", " roc_curve, f1_score, precision_recall_curve, accuracy_score\n", ")\n", "from sklearn.utils.class_weight import compute_class_weight\n", "\n", "# Machine Learning - Algorithms\n", "from sklearn.ensemble import RandomForestClassifier\n", "from xgboost import XGBClassifier\n", "from lightgbm import LGBMClassifier\n", "\n", "# Deep Learning\n", "import tensorflow as tf\n", "from tensorflow.keras.models import Sequential\n", "from tensorflow.keras.layers import LSTM, GRU, Dense, Dropout, Conv1D, MaxPooling1D, Flatten\n", "from tensorflow.keras.optimizers import Adam\n", "from tensorflow.keras.callbacks import EarlyStopping, ReduceLROnPlateau\n", "\n", "# Visualization\n", "import plotly.express as px\n", "import plotly.graph_objects as go\n", "from plotly.subplots import make_subplots\n", "import plotly.figure_factory as ff\n", "\n", "# Utilities\n", "import joblib\n", "import json\n", "from datetime import datetime\n", "import os\n", "\n", "# Set random seeds for reproducibility\n", "np.random.seed(42)\n", "tf.random.set_seed(42)\n", "\n", "print(\"βœ… All libraries imported successfully!\")\n", "print(f\"πŸ“Š Pandas version: {pd.__version__}\")\n", "print(f\"πŸ€– TensorFlow version: {tf.__version__}\")\n", "print(f\"πŸ“ˆ Plotly available for interactive visualizations\")" ] }, { "cell_type": "markdown", "id": "823d6575", "metadata": {}, "source": [ "## πŸ“ Data Loading and Initial Exploration\n", "\n", "We load both our engineered dataset (for modeling) and the merged dataset (for additional context). The engineered dataset contains all our carefully crafted features, while the merged dataset provides the complete picture of our data transformation journey." ] }, { "cell_type": "code", "execution_count": 15, "id": "2839d001", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "πŸ“– Loading datasets...\n", "βœ… Engineered dataset loaded: 245 rows Γ— 17 columns\n", "βœ… Merged dataset loaded: 245 rows Γ— 9 columns\n", "\n", "πŸ“‹ Engineered Dataset Overview:\n", " β€’ Date range: 2024-07-06 to 2024-08-23\n", " β€’ Memory usage: 0.1 MB\n", " β€’ Target variable: New catches\n", "\n", "πŸ“Š Available columns: ['Date', 'Location', 'Location_Code', 'Average Temperature', 'Average Humidity', 'Temp_Range', 'Temp_Avg_3d', 'Humidity_Avg_3d', 'Insects_Lag1', 'Insects_Lag3', 'Recent_Activity', 'Days_Since_Cleaning', 'Month', 'Day', 'Season', 'Number of insects', 'New catches']\n" ] }, { "data": { "application/vnd.microsoft.datawrangler.viewer.v0+json": { "columns": [ { "name": "index", "rawType": "int64", "type": "integer" }, { "name": "Date", "rawType": "object", "type": "string" }, { "name": "Location", "rawType": "object", "type": "string" }, { "name": "Location_Code", "rawType": "int64", "type": "integer" }, { "name": "Average Temperature", "rawType": "float64", "type": "float" }, { "name": "Average Humidity", "rawType": "float64", "type": "float" }, { "name": "Temp_Range", "rawType": "float64", "type": "float" }, { "name": "Temp_Avg_3d", "rawType": "float64", "type": "float" }, { "name": "Humidity_Avg_3d", "rawType": "float64", "type": "float" }, { "name": "Insects_Lag1", "rawType": "float64", "type": "float" }, { "name": "Insects_Lag3", "rawType": "float64", "type": "float" }, { "name": "Recent_Activity", "rawType": "int64", "type": "integer" }, { "name": "Days_Since_Cleaning", "rawType": "int64", "type": "integer" }, { "name": "Month", "rawType": "int64", "type": "integer" }, { "name": "Day", "rawType": "int64", "type": "integer" }, { "name": "Season", "rawType": "object", "type": "string" }, { "name": "Number of insects", "rawType": "float64", "type": "float" }, { "name": "New catches", "rawType": "float64", "type": "float" } ], "ref": "c0fcabf5-d2e6-47d8-971d-c22efc579977", "rows": [ [ "0", "2024-07-06", "Cicalino 1", "0", "22.34", "72.25", "1.59", "22.34", "72.25", "0.0", "0.0", "0", "0", "7", "6", "Mid_Summer", "0.0", "0.0" ], [ "1", "2024-07-07", "Cicalino 1", "0", "23.52", "76.73", "1.2199999999999989", "22.93", "74.49000000000001", "0.0", "0.0", "0", "0", "7", "7", "Mid_Summer", "0.0", "0.0" ], [ "2", "2024-07-08", "Cicalino 1", "0", "25.67", "69.14", "1.8699999999999968", "23.843333333333334", "72.70666666666666", "0.0", "0.0", "0", "0", "7", "8", "Mid_Summer", "0.0", "0.0" ], [ "3", "2024-07-09", "Cicalino 1", "0", "25.87", "53.65", "1.9799999999999969", "25.02", "66.50666666666667", "0.0", "0.0", "0", "0", "7", "9", "Mid_Summer", "0.0", "0.0" ], [ "4", "2024-07-10", "Cicalino 1", "0", "26.41", "58.94", "1.9099999999999968", "25.98333333333333", "60.57666666666666", "0.0", "0.0", "0", "0", "7", "10", "Mid_Summer", "0.0", "0.0" ] ], "shape": { "columns": 17, "rows": 5 } }, "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
DateLocationLocation_CodeAverage TemperatureAverage HumidityTemp_RangeTemp_Avg_3dHumidity_Avg_3dInsects_Lag1Insects_Lag3Recent_ActivityDays_Since_CleaningMonthDaySeasonNumber of insectsNew catches
02024-07-06Cicalino 1022.3472.251.5922.34000072.2500000.00.00076Mid_Summer0.00.0
12024-07-07Cicalino 1023.5276.731.2222.93000074.4900000.00.00077Mid_Summer0.00.0
22024-07-08Cicalino 1025.6769.141.8723.84333372.7066670.00.00078Mid_Summer0.00.0
32024-07-09Cicalino 1025.8753.651.9825.02000066.5066670.00.00079Mid_Summer0.00.0
42024-07-10Cicalino 1026.4158.941.9125.98333360.5766670.00.000710Mid_Summer0.00.0
\n", "
" ], "text/plain": [ " Date Location Location_Code Average Temperature \\\n", "0 2024-07-06 Cicalino 1 0 22.34 \n", "1 2024-07-07 Cicalino 1 0 23.52 \n", "2 2024-07-08 Cicalino 1 0 25.67 \n", "3 2024-07-09 Cicalino 1 0 25.87 \n", "4 2024-07-10 Cicalino 1 0 26.41 \n", "\n", " Average Humidity Temp_Range Temp_Avg_3d Humidity_Avg_3d Insects_Lag1 \\\n", "0 72.25 1.59 22.340000 72.250000 0.0 \n", "1 76.73 1.22 22.930000 74.490000 0.0 \n", "2 69.14 1.87 23.843333 72.706667 0.0 \n", "3 53.65 1.98 25.020000 66.506667 0.0 \n", "4 58.94 1.91 25.983333 60.576667 0.0 \n", "\n", " Insects_Lag3 Recent_Activity Days_Since_Cleaning Month Day Season \\\n", "0 0.0 0 0 7 6 Mid_Summer \n", "1 0.0 0 0 7 7 Mid_Summer \n", "2 0.0 0 0 7 8 Mid_Summer \n", "3 0.0 0 0 7 9 Mid_Summer \n", "4 0.0 0 0 7 10 Mid_Summer \n", "\n", " Number of insects New catches \n", "0 0.0 0.0 \n", "1 0.0 0.0 \n", "2 0.0 0.0 \n", "3 0.0 0.0 \n", "4 0.0 0.0 " ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "\n", "πŸ“Š Dataset Info:\n", "\n", "RangeIndex: 245 entries, 0 to 244\n", "Data columns (total 17 columns):\n", " # Column Non-Null Count Dtype \n", "--- ------ -------------- ----- \n", " 0 Date 245 non-null object \n", " 1 Location 245 non-null object \n", " 2 Location_Code 245 non-null int64 \n", " 3 Average Temperature 245 non-null float64\n", " 4 Average Humidity 245 non-null float64\n", " 5 Temp_Range 245 non-null float64\n", " 6 Temp_Avg_3d 245 non-null float64\n", " 7 Humidity_Avg_3d 245 non-null float64\n", " 8 Insects_Lag1 245 non-null float64\n", " 9 Insects_Lag3 245 non-null float64\n", " 10 Recent_Activity 245 non-null int64 \n", " 11 Days_Since_Cleaning 245 non-null int64 \n", " 12 Month 245 non-null int64 \n", " 13 Day 245 non-null int64 \n", " 14 Season 245 non-null object \n", " 15 Number of insects 245 non-null float64\n", " 16 New catches 245 non-null float64\n", "dtypes: float64(9), int64(5), object(3)\n", "memory usage: 32.7+ KB\n" ] } ], "source": [ "# Load the datasets\n", "print(\"πŸ“– Loading datasets...\")\n", "\n", "# Primary dataset for modeling (engineered features)\n", "df_engineered = pd.read_csv('cleaned_engineered_data.csv')\n", "\n", "# Secondary dataset for context\n", "df_merged = pd.read_csv('cleaned_merged_data.csv')\n", "\n", "print(f\"βœ… Engineered dataset loaded: {df_engineered.shape[0]:,} rows Γ— {df_engineered.shape[1]} columns\")\n", "print(f\"βœ… Merged dataset loaded: {df_merged.shape[0]:,} rows Γ— {df_merged.shape[1]} columns\")\n", "\n", "# Display basic information about our primary dataset\n", "print(\"\\nπŸ“‹ Engineered Dataset Overview:\")\n", "print(f\" β€’ Date range: {df_engineered['Date'].min()} to {df_engineered['Date'].max()}\")\n", "print(f\" β€’ Memory usage: {df_engineered.memory_usage(deep=True).sum() / 1024**2:.1f} MB\")\n", "print(f\" β€’ Target variable: New catches\")\n", "\n", "# Check actual column names\n", "print(f\"\\nπŸ“Š Available columns: {list(df_engineered.columns)}\")\n", "\n", "# Quick peek at the data structure\n", "display(df_engineered.head())\n", "print(\"\\nπŸ“Š Dataset Info:\")\n", "df_engineered.info()" ] }, { "cell_type": "markdown", "id": "ab308cc1", "metadata": {}, "source": [ "## βš–οΈ Class Imbalance Diagnosis\n", "\n", "Before diving into modeling, we must understand the distribution of our target variable `NEW_CATCHES`. Class imbalance is a critical factor that can severely impact model performance if not properly addressed. We'll visualize the distribution and calculate key metrics to inform our modeling strategy." ] }, { "cell_type": "code", "execution_count": 16, "id": "8745ddbf", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "πŸ”„ Converting to Binary Classification...\n", "🎯 Target Variable Distribution Analysis (Binary):\n", "==================================================\n", " Class 0 (No Catch): 224 samples (91.43%)\n", " Class 1 (Catch): 21 samples (8.57%)\n", "\n", "πŸ“Š Imbalance Metrics:\n", " β€’ Imbalance Ratio: 10.67:1\n", " β€’ Majority Class: 224 samples\n", " β€’ Minority Class: 21 samples\n", " β€’ Imbalance Severity: πŸ”΄ SEVERE\n" ] }, { "data": { "application/vnd.plotly.v1+json": { "config": { "plotlyServerURL": "https://plot.ly" }, "data": [ { "alignmentgroup": "True", "customdata": [ 91.42857142857143, 8.571428571428571 ], "hovertemplate": "Class Label=%{x}
Number of Samples=%{y}
text=%{text}
color=%{marker.color}", "legendgroup": "", "marker": { "color": [ 224, 21 ], "coloraxis": "coloraxis", "pattern": { "shape": "" } }, "name": "", "offsetgroup": "", "orientation": "v", "showlegend": false, "text": [ 224, 21 ], "textposition": "outside", "texttemplate": "%{text:,}
(%{customdata:.1f}%)", "type": "bar", "x": [ "No Catch (0)", "Catch (1)" ], "xaxis": "x", "y": [ 224, 21 ], "yaxis": "y" } ], "layout": { "annotations": [ { "font": { "color": "red", "size": 12 }, "showarrow": false, "text": "Imbalance Ratio: 10.67:1 (SEVERE)", "x": 0.02, "xref": "paper", "y": 0.98, "yref": "paper" } ], "barmode": "relative", "coloraxis": { "colorbar": { "title": { "text": "color" } }, "colorscale": [ [ 0, "#440154" ], [ 0.1111111111111111, "#482878" ], [ 0.2222222222222222, "#3e4989" ], [ 0.3333333333333333, "#31688e" ], [ 0.4444444444444444, "#26828e" ], [ 0.5555555555555556, "#1f9e89" ], [ 0.6666666666666666, "#35b779" ], [ 0.7777777777777778, "#6ece58" ], [ 0.8888888888888888, "#b5de2b" ], [ 1, "#fde725" ] ] }, "height": 500, "legend": { "tracegroupgap": 0 }, "showlegend": false, "template": { "data": { "bar": [ { "error_x": { "color": "#2a3f5f" }, "error_y": { "color": "#2a3f5f" }, "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "bar" } ], "barpolar": [ { "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "barpolar" } ], "carpet": [ { "aaxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "baxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "type": "carpet" } ], "choropleth": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "choropleth" } ], "contour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "contour" } ], "contourcarpet": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "contourcarpet" } ], "heatmap": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "heatmap" } ], "heatmapgl": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "heatmapgl" } ], "histogram": [ { "marker": { "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "histogram" } ], "histogram2d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "histogram2d" } ], "histogram2dcontour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "histogram2dcontour" } ], "mesh3d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "mesh3d" } ], "parcoords": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "parcoords" } ], "pie": [ { "automargin": true, "type": "pie" } ], "scatter": [ { "fillpattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 }, "type": "scatter" } ], "scatter3d": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatter3d" } ], "scattercarpet": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattercarpet" } ], "scattergeo": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergeo" } ], "scattergl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergl" } ], "scattermapbox": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattermapbox" } ], "scatterpolar": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolar" } ], "scatterpolargl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolargl" } ], "scatterternary": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterternary" } ], "surface": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "surface" } ], "table": [ { "cells": { "fill": { "color": "#EBF0F8" }, "line": { "color": "white" } }, "header": { "fill": { "color": "#C8D4E3" }, "line": { "color": "white" } }, "type": "table" } ] }, "layout": { "annotationdefaults": { "arrowcolor": "#2a3f5f", "arrowhead": 0, "arrowwidth": 1 }, "autotypenumbers": "strict", "coloraxis": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "colorscale": { "diverging": [ [ 0, "#8e0152" ], [ 0.1, "#c51b7d" ], [ 0.2, "#de77ae" ], [ 0.3, "#f1b6da" ], [ 0.4, "#fde0ef" ], [ 0.5, "#f7f7f7" ], [ 0.6, "#e6f5d0" ], [ 0.7, "#b8e186" ], [ 0.8, "#7fbc41" ], [ 0.9, "#4d9221" ], [ 1, "#276419" ] ], "sequential": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "sequentialminus": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ] }, "colorway": [ "#636efa", "#EF553B", "#00cc96", "#ab63fa", "#FFA15A", "#19d3f3", "#FF6692", "#B6E880", "#FF97FF", "#FECB52" ], "font": { "color": "#2a3f5f" }, "geo": { "bgcolor": "white", "lakecolor": "white", "landcolor": "#E5ECF6", "showlakes": true, "showland": true, "subunitcolor": "white" }, "hoverlabel": { "align": "left" }, "hovermode": "closest", "mapbox": { "style": "light" }, "paper_bgcolor": "white", "plot_bgcolor": "#E5ECF6", "polar": { "angularaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "radialaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "scene": { "xaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "yaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "zaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" } }, "shapedefaults": { "line": { "color": "#2a3f5f" } }, "ternary": { "aaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "baxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "caxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "title": { "x": 0.05 }, "xaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 }, "yaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 } } }, "title": { "text": "🎯 Target Variable Distribution: New catches (Binary)", "x": 0.5 }, "xaxis": { "anchor": "y", "domain": [ 0, 1 ], "title": { "text": "Class Label" } }, "yaxis": { "anchor": "x", "domain": [ 0, 1 ], "title": { "text": "Number of Samples" } } } } }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "\n", "⚠️ CRITICAL INSIGHT:\n", " The severe class imbalance requires robust handling through:\n", " 1. 🎯 Class weighting in all models\n", " 2. πŸ”§ Optimal threshold tuning\n", " 3. πŸ“Š F1-score as primary evaluation metric\n", "\n", "βœ… Binary classification setup complete!\n", " β€’ No Catch (0): Any day with 0 insects caught\n", " β€’ Catch (1): Any day with 1+ insects caught\n" ] } ], "source": [ "# First, convert to binary classification (0 = No Catch, 1+ = Catch)\n", "print(\"πŸ”„ Converting to Binary Classification...\")\n", "df_engineered['New catches'] = (df_engineered['New catches'] > 0).astype(int)\n", "\n", "# Calculate class distribution for binary classification\n", "class_counts = df_engineered['New catches'].value_counts().sort_index()\n", "class_percentages = df_engineered['New catches'].value_counts(normalize=True).sort_index() * 100\n", "\n", "print(\"🎯 Target Variable Distribution Analysis (Binary):\")\n", "print(\"=\" * 50)\n", "\n", "for class_label in class_counts.index:\n", " count = class_counts[class_label]\n", " percentage = class_percentages[class_label]\n", " class_name = \"No Catch\" if class_label == 0 else \"Catch\"\n", " print(f\" Class {class_label} ({class_name}): {count:,} samples ({percentage:.2f}%)\")\n", "\n", "# Calculate imbalance ratio\n", "majority_class = class_counts.max()\n", "minority_class = class_counts.min()\n", "imbalance_ratio = majority_class / minority_class\n", "\n", "print(f\"\\nπŸ“Š Imbalance Metrics:\")\n", "print(f\" β€’ Imbalance Ratio: {imbalance_ratio:.2f}:1\")\n", "print(f\" β€’ Majority Class: {majority_class:,} samples\")\n", "print(f\" β€’ Minority Class: {minority_class:,} samples\")\n", "\n", "# Determine imbalance severity\n", "if imbalance_ratio > 10:\n", " severity = \"πŸ”΄ SEVERE\"\n", "elif imbalance_ratio > 3:\n", " severity = \"🟑 MODERATE\"\n", "else:\n", " severity = \"🟒 MILD\"\n", "\n", "print(f\" β€’ Imbalance Severity: {severity}\")\n", "\n", "# Create interactive visualization\n", "fig = px.bar(\n", " x=['No Catch (0)', 'Catch (1)'],\n", " y=class_counts.values,\n", " title=\"🎯 Target Variable Distribution: New catches (Binary)\",\n", " labels={'x': 'Class Label', 'y': 'Number of Samples'},\n", " text=class_counts.values,\n", " color=class_counts.values,\n", " color_continuous_scale='Viridis'\n", ")\n", "\n", "# Enhance the visualization\n", "fig.update_traces(\n", " texttemplate='%{text:,}
(%{customdata:.1f}%)',\n", " customdata=class_percentages.values,\n", " textposition='outside'\n", ")\n", "\n", "fig.update_layout(\n", " height=500,\n", " showlegend=False,\n", " title_x=0.5,\n", " annotations=[\n", " dict(\n", " text=f\"Imbalance Ratio: {imbalance_ratio:.2f}:1 ({severity.split()[-1]})\",\n", " xref=\"paper\", yref=\"paper\",\n", " x=0.02, y=0.98,\n", " showarrow=False,\n", " font=dict(size=12, color=\"red\" if \"SEVERE\" in severity else \"orange\" if \"MODERATE\" in severity else \"green\")\n", " )\n", " ]\n", ")\n", "\n", "fig.show()\n", "\n", "print(\"\\n⚠️ CRITICAL INSIGHT:\")\n", "print(f\" The {severity.split()[-1].lower()} class imbalance requires robust handling through:\")\n", "print(\" 1. 🎯 Class weighting in all models\")\n", "print(\" 2. πŸ”§ Optimal threshold tuning\")\n", "print(\" 3. πŸ“Š F1-score as primary evaluation metric\")\n", "print(f\"\\nβœ… Binary classification setup complete!\")\n", "print(f\" β€’ No Catch (0): Any day with 0 insects caught\")\n", "print(f\" β€’ Catch (1): Any day with 1+ insects caught\")\n" ] }, { "cell_type": "markdown", "id": "1f8e7762", "metadata": {}, "source": [ "## πŸ”ͺ Data Splitting and Weight Calculation\n", "\n", "We perform a **chronological split** to respect the time-series nature of our data. This prevents data leakage and ensures our model evaluation reflects real-world performance. We also calculate class weights to handle the imbalance we identified above." ] }, { "cell_type": "code", "execution_count": 17, "id": "d49cf288", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "πŸ”§ Preparing features and target variable...\n", "🚫 Excluded columns (potential leakage/non-features): ['Date', 'New catches', 'Number of insects']\n", "βœ… Feature columns: ['Location', 'Location_Code', 'Average Temperature', 'Average Humidity', 'Temp_Range', 'Temp_Avg_3d', 'Humidity_Avg_3d', 'Insects_Lag1', 'Insects_Lag3', 'Recent_Activity', 'Days_Since_Cleaning', 'Month', 'Day', 'Season']\n", "βœ… Features prepared: 14 columns\n", "βœ… Target prepared: 245 samples\n", "πŸ“… Data sorted chronologically from 2024-07-06 to 2024-08-23\n", "\n", "βœ‚οΈ Performing chronological train-test split...\n", "πŸ“Š Training data: 196 samples (up to 2024-08-14)\n", "πŸ“Š Test data: 49 samples (from 2024-08-14)\n", "\n", "βš–οΈ Calculating class weights...\n", "βœ… Class weights calculated:\n", " Class 0: 0.533\n", " Class 1: 8.167\n", "\n", "πŸ” Verifying class distribution preservation:\n" ] }, { "data": { "application/vnd.microsoft.datawrangler.viewer.v0+json": { "columns": [ { "name": "New catches", "rawType": "int32", "type": "integer" }, { "name": "Training (%)", "rawType": "float64", "type": "float" }, { "name": "Test (%)", "rawType": "float64", "type": "float" } ], "ref": "4568e6b2-0f15-4394-80dd-e53bc9daa008", "rows": [ [ "0", "93.88", "81.63" ], [ "1", "6.12", "18.37" ] ], "shape": { "columns": 2, "rows": 2 } }, "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
Training (%)Test (%)
New catches
093.8881.63
16.1218.37
\n", "
" ], "text/plain": [ " Training (%) Test (%)\n", "New catches \n", "0 93.88 81.63\n", "1 6.12 18.37" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "\n", "πŸ”„ Encoding categorical variables...\n", "πŸ“Š Categorical columns found: ['Location', 'Season']\n", "βœ… Encoded features: 17 columns\n", "βœ… Training set: (196, 17)\n", "βœ… Test set: (49, 17)\n", "\n", "πŸ” Data leakage check - Feature-target correlations:\n", "Top 5 feature correlations with target:\n", " Recent_Activity: 0.557\n", " Insects_Lag1: 0.347\n", " Insects_Lag3: 0.347\n", " Days_Since_Cleaning: 0.329\n", " Location_Code: 0.302\n", "\n", "🎯 Data splitting and encoding complete! Ready for model training.\n" ] } ], "source": [ "# Prepare features and target\n", "print(\"πŸ”§ Preparing features and target variable...\")\n", "\n", "# Convert Date to datetime for proper sorting\n", "df_engineered['Date'] = pd.to_datetime(df_engineered['Date'])\n", "\n", "# Sort data chronologically to maintain time series integrity\n", "df_sorted = df_engineered.sort_values('Date').reset_index(drop=True)\n", "\n", "# Identify feature columns (exclude non-feature columns and potential leakage)\n", "exclude_cols = ['Date', 'New catches', 'Number of insects'] # Exclude target and potential leakage\n", "feature_cols = [col for col in df_sorted.columns if col not in exclude_cols]\n", "\n", "print(f\"🚫 Excluded columns (potential leakage/non-features): {exclude_cols}\")\n", "print(f\"βœ… Feature columns: {feature_cols}\")\n", "\n", "X = df_sorted[feature_cols]\n", "y = df_sorted['New catches']\n", "dates = df_sorted['Date']\n", "\n", "print(f\"βœ… Features prepared: {len(feature_cols)} columns\")\n", "print(f\"βœ… Target prepared: {len(y)} samples\")\n", "print(f\"πŸ“… Data sorted chronologically from {dates.min().date()} to {dates.max().date()}\")\n", "\n", "# Perform chronological train-test split (80-20)\n", "print(\"\\nβœ‚οΈ Performing chronological train-test split...\")\n", "\n", "split_idx = int(len(X) * 0.8)\n", "\n", "X_train = X.iloc[:split_idx]\n", "X_test = X.iloc[split_idx:]\n", "y_train = y.iloc[:split_idx]\n", "y_test = y.iloc[split_idx:]\n", "train_dates = dates.iloc[:split_idx]\n", "test_dates = dates.iloc[split_idx:]\n", "\n", "train_end_date = train_dates.iloc[-1].date()\n", "test_start_date = test_dates.iloc[0].date()\n", "\n", "print(f\"πŸ“Š Training data: {len(X_train):,} samples (up to {train_end_date})\")\n", "print(f\"πŸ“Š Test data: {len(X_test):,} samples (from {test_start_date})\")\n", "\n", "# Calculate class weights from training data\n", "print(\"\\nβš–οΈ Calculating class weights...\")\n", "\n", "classes = np.unique(y_train)\n", "class_weights_array = compute_class_weight(\n", " class_weight='balanced',\n", " classes=classes,\n", " y=y_train\n", ")\n", "\n", "# Create class weight dictionary\n", "class_weights = dict(zip(classes, class_weights_array))\n", "\n", "print(\"βœ… Class weights calculated:\")\n", "for class_label, weight in class_weights.items():\n", " print(f\" Class {class_label}: {weight:.3f}\")\n", "\n", "# Verify split preserves class distribution\n", "print(\"\\nπŸ” Verifying class distribution preservation:\")\n", "train_dist = y_train.value_counts(normalize=True).sort_index() * 100\n", "test_dist = y_test.value_counts(normalize=True).sort_index() * 100\n", "\n", "comparison_df = pd.DataFrame({\n", " 'Training (%)': train_dist,\n", " 'Test (%)': test_dist\n", "})\n", "\n", "display(comparison_df.round(2))\n", "\n", "# Handle categorical variables - encode them for machine learning models\n", "print(\"\\nπŸ”„ Encoding categorical variables...\")\n", "\n", "# Identify categorical columns\n", "categorical_cols = X_train.select_dtypes(include=['object']).columns.tolist()\n", "print(f\"πŸ“Š Categorical columns found: {categorical_cols}\")\n", "\n", "if categorical_cols:\n", " # Use pandas get_dummies for one-hot encoding\n", " X_train_encoded = pd.get_dummies(X_train, columns=categorical_cols, drop_first=True)\n", " X_test_encoded = pd.get_dummies(X_test, columns=categorical_cols, drop_first=True)\n", " \n", " # Ensure both train and test have the same columns\n", " # Get all columns from training set\n", " train_cols = set(X_train_encoded.columns)\n", " test_cols = set(X_test_encoded.columns)\n", " \n", " # Add missing columns to test set (fill with 0)\n", " missing_in_test = train_cols - test_cols\n", " for col in missing_in_test:\n", " X_test_encoded[col] = 0\n", " \n", " # Add missing columns to train set (fill with 0)\n", " missing_in_train = test_cols - train_cols\n", " for col in missing_in_train:\n", " X_train_encoded[col] = 0\n", " \n", " # Reorder columns to match\n", " X_test_encoded = X_test_encoded[X_train_encoded.columns]\n", " \n", " print(f\"βœ… Encoded features: {X_train_encoded.shape[1]} columns\")\n", " print(f\"βœ… Training set: {X_train_encoded.shape}\")\n", " print(f\"βœ… Test set: {X_test_encoded.shape}\")\n", " \n", " # Update feature sets\n", " X_train = X_train_encoded\n", " X_test = X_test_encoded\n", "else:\n", " print(\"βœ… No categorical variables found\")\n", "\n", "# Check for data leakage by examining feature correlations with target\n", "print(\"\\nπŸ” Data leakage check - Feature-target correlations:\")\n", "# Only check numeric columns for correlation\n", "numeric_cols = X_train.select_dtypes(include=[np.number]).columns\n", "correlations = X_train[numeric_cols].corrwith(y_train).abs().sort_values(ascending=False)\n", "print(\"Top 5 feature correlations with target:\")\n", "for feature, corr in correlations.head().items():\n", " print(f\" {feature}: {corr:.3f}\")\n", "\n", "print(\"\\n🎯 Data splitting and encoding complete! Ready for model training.\")" ] }, { "cell_type": "markdown", "id": "1ce4d1ac", "metadata": {}, "source": [ "# πŸ₯Š Standard Classifiers Tournament\n", "\n", "Now begins our first tournament phase! We'll pit three powerful machine learning algorithms against each other:\n", "\n", "1. **🌳 RandomForestClassifier**: Ensemble of decision trees with voting\n", "2. **πŸš€ XGBoostClassifier**: Gradient boosting with advanced optimization\n", "3. **⚑ LightGBMClassifier**: Microsoft's efficient gradient boosting\n", "\n", "Each contestant will undergo:\n", "- **Hyperparameter tuning** via GridSearchCV\n", "- **Class-weighted training** to handle imbalance\n", "- **Optimal threshold discovery** for maximum F1-score\n", "- **Comprehensive evaluation** on test data\n", "\n", "Let the battle commence! βš”οΈ" ] }, { "cell_type": "markdown", "id": "3a6c0f9a", "metadata": {}, "source": [ "## 🌳 Contestant 1: RandomForestClassifier\n", "\n", "Our first competitor leverages the wisdom of crowds through ensemble learning. Random Forest builds multiple decision trees and combines their predictions, making it robust against overfitting and excellent for handling mixed data types." ] }, { "cell_type": "code", "execution_count": 18, "id": "8c5315ce", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "🌳 Training RandomForestClassifier...\n", "==================================================\n", "πŸ“Š Using 17 features for RandomForest modeling\n", "βœ… Training set: (196, 17)\n", "βœ… Test set: (49, 17)\n", "πŸ”§ Hyperparameter combinations to test: 216\n", "πŸ” Performing hyperparameter tuning with TimeSeriesSplit...\n", "Fitting 3 folds for each of 216 candidates, totalling 648 fits\n", "βœ… Best parameters found: {'max_depth': 10, 'max_features': 'sqrt', 'min_samples_leaf': 2, 'min_samples_split': 5, 'n_estimators': 100}\n", "🎯 Best CV F1-score: 0.1212\n", "\n", "πŸ”§ Finding optimal prediction threshold using temporal validation...\n", "πŸ“Š CV Training set: (156, 17), CV Validation set: (40, 17)\n", "🎯 Optimal threshold: 0.100\n", "πŸ† Validation F1-score at optimal threshold: 0.0000\n", "\n", "πŸ”„ Retraining on full training set...\n", "\n", "πŸ“Š Evaluating on test set...\n", "🎯 Test F1-score: 0.6667\n", "πŸ“ˆ Test AUC: 0.9194\n", "🎯 Test Accuracy: 0.8163\n", "\n", "πŸ“‹ Detailed Classification Report:\n", " precision recall f1-score support\n", "\n", " No Catch 1.00 0.78 0.87 40\n", " Catch 0.50 1.00 0.67 9\n", "\n", " accuracy 0.82 49\n", " macro avg 0.75 0.89 0.77 49\n", "weighted avg 0.91 0.82 0.84 49\n", "\n", "\n", "βœ… RandomForestClassifier training complete!\n" ] } ], "source": [ "from sklearn.model_selection import TimeSeriesSplit\n", "from sklearn.metrics import classification_report\n", "\n", "print(\"🌳 Training RandomForestClassifier...\")\n", "print(\"=\" * 50)\n", "\n", "# Use all available features (already cleaned of leakage)\n", "X_train_rf = X_train.copy()\n", "X_test_rf = X_test.copy()\n", "\n", "print(f\"πŸ“Š Using {len(X_train_rf.columns)} features for RandomForest modeling\")\n", "print(f\"βœ… Training set: {X_train_rf.shape}\")\n", "print(f\"βœ… Test set: {X_test_rf.shape}\")\n", "\n", "# Use focused hyperparameter grid for RandomForest\n", "rf_param_grid = {\n", " 'n_estimators': [100, 200, 300],\n", " 'max_depth': [10, 15, 20, None],\n", " 'min_samples_split': [5, 10, 15], # Higher values to prevent overfitting on imbalanced data\n", " 'min_samples_leaf': [2, 4, 6], # Higher values to prevent overfitting\n", " 'max_features': ['sqrt', 'log2']\n", "}\n", "\n", "total_combinations = (len(rf_param_grid['n_estimators']) * len(rf_param_grid['max_depth']) * \n", " len(rf_param_grid['min_samples_split']) * len(rf_param_grid['min_samples_leaf']) * \n", " len(rf_param_grid['max_features']))\n", "print(f\"πŸ”§ Hyperparameter combinations to test: {total_combinations}\")\n", "\n", "# Initialize Random Forest with class weights\n", "rf_base = RandomForestClassifier(\n", " class_weight='balanced',\n", " random_state=42,\n", " n_jobs=-1\n", ")\n", "\n", "# Use TimeSeriesSplit for cross-validation to respect temporal order\n", "tscv = TimeSeriesSplit(n_splits=3)\n", "\n", "print(\"πŸ” Performing hyperparameter tuning with TimeSeriesSplit...\")\n", "rf_grid = GridSearchCV(\n", " rf_base,\n", " rf_param_grid,\n", " cv=tscv, # Use TimeSeriesSplit instead of regular CV\n", " scoring='f1',\n", " n_jobs=-1,\n", " verbose=1\n", ")\n", "\n", "rf_grid.fit(X_train_rf, y_train)\n", "\n", "print(f\"βœ… Best parameters found: {rf_grid.best_params_}\")\n", "print(f\"🎯 Best CV F1-score: {rf_grid.best_score_:.4f}\")\n", "\n", "# Get the best model\n", "rf_best = rf_grid.best_estimator_\n", "\n", "# Use a separate time-based validation set for threshold tuning\n", "# Split training data chronologically (use 80% for training, 20% for validation)\n", "split_point = int(len(X_train_rf) * 0.8)\n", "X_train_cv_rf = X_train_rf.iloc[:split_point]\n", "X_val_cv_rf = X_train_rf.iloc[split_point:]\n", "y_train_cv_rf = y_train.iloc[:split_point]\n", "y_val_cv_rf = y_train.iloc[split_point:]\n", "\n", "# Train on the CV training set for threshold optimization\n", "rf_threshold_model = RandomForestClassifier(**rf_grid.best_params_, class_weight='balanced', random_state=42, n_jobs=-1)\n", "rf_threshold_model.fit(X_train_cv_rf, y_train_cv_rf)\n", "\n", "print(f\"\\nπŸ”§ Finding optimal prediction threshold using temporal validation...\")\n", "print(f\"πŸ“Š CV Training set: {X_train_cv_rf.shape}, CV Validation set: {X_val_cv_rf.shape}\")\n", "\n", "# Get prediction probabilities for threshold tuning\n", "y_val_proba_rf = rf_threshold_model.predict_proba(X_val_cv_rf)[:, 1]\n", "\n", "# Find optimal threshold using F1-score\n", "thresholds = np.arange(0.1, 0.9, 0.01)\n", "f1_scores_rf = []\n", "\n", "for threshold in thresholds:\n", " y_pred_thresh = (y_val_proba_rf >= threshold).astype(int)\n", " if len(np.unique(y_pred_thresh)) > 1: # Avoid division by zero\n", " f1 = f1_score(y_val_cv_rf, y_pred_thresh)\n", " else:\n", " f1 = 0.0\n", " f1_scores_rf.append(f1)\n", "\n", "optimal_threshold_rf = thresholds[np.argmax(f1_scores_rf)]\n", "max_f1_rf = max(f1_scores_rf)\n", "\n", "print(f\"🎯 Optimal threshold: {optimal_threshold_rf:.3f}\")\n", "print(f\"πŸ† Validation F1-score at optimal threshold: {max_f1_rf:.4f}\")\n", "\n", "# Retrain on full training set with best parameters\n", "print(\"\\nπŸ”„ Retraining on full training set...\")\n", "rf_final = RandomForestClassifier(**rf_grid.best_params_, class_weight='balanced', random_state=42, n_jobs=-1)\n", "rf_final.fit(X_train_rf, y_train)\n", "\n", "# Make predictions on test set with optimal threshold\n", "print(\"\\nπŸ“Š Evaluating on test set...\")\n", "y_test_proba_rf = rf_final.predict_proba(X_test_rf)[:, 1]\n", "y_test_pred_rf = (y_test_proba_rf >= optimal_threshold_rf).astype(int)\n", "\n", "# Calculate metrics\n", "rf_test_f1 = f1_score(y_test, y_test_pred_rf)\n", "rf_test_auc = roc_auc_score(y_test, y_test_proba_rf)\n", "rf_test_accuracy = accuracy_score(y_test, y_test_pred_rf)\n", "\n", "print(f\"🎯 Test F1-score: {rf_test_f1:.4f}\")\n", "print(f\"πŸ“ˆ Test AUC: {rf_test_auc:.4f}\")\n", "print(f\"🎯 Test Accuracy: {rf_test_accuracy:.4f}\")\n", "\n", "# Detailed classification report\n", "print(f\"\\nπŸ“‹ Detailed Classification Report:\")\n", "print(classification_report(y_test, y_test_pred_rf, target_names=['No Catch', 'Catch']))\n", "\n", "# Store results for later comparison\n", "rf_results = {\n", " 'model': rf_final,\n", " 'threshold': optimal_threshold_rf,\n", " 'test_f1': rf_test_f1,\n", " 'test_auc': rf_test_auc,\n", " 'test_accuracy': rf_test_accuracy,\n", " 'y_pred': y_test_pred_rf,\n", " 'y_proba': y_test_proba_rf,\n", " 'best_params': rf_grid.best_params_\n", "}\n", "\n", "print(\"\\nβœ… RandomForestClassifier training complete!\")" ] }, { "cell_type": "markdown", "id": "cef4e403", "metadata": {}, "source": [ "### πŸ“Š RandomForest Performance Analysis\n", "\n", "Let's dive deep into how our Random Forest model performed with interactive visualizations." ] }, { "cell_type": "code", "execution_count": 19, "id": "87b90cde", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "πŸ“‹ RandomForest Classification Report:\n", "============================================================\n", " precision recall f1-score support\n", "\n", " No Catch 1.00 0.78 0.87 40\n", " Catch 0.50 1.00 0.67 9\n", "\n", " accuracy 0.82 49\n", " macro avg 0.75 0.89 0.77 49\n", "weighted avg 0.91 0.82 0.84 49\n", "\n" ] }, { "data": { "application/vnd.plotly.v1+json": { "config": { "plotlyServerURL": "https://plot.ly" }, "data": [ { "colorscale": [ [ 0, "rgb(247,251,255)" ], [ 0.125, "rgb(222,235,247)" ], [ 0.25, "rgb(198,219,239)" ], [ 0.375, "rgb(158,202,225)" ], [ 0.5, "rgb(107,174,214)" ], [ 0.625, "rgb(66,146,198)" ], [ 0.75, "rgb(33,113,181)" ], [ 0.875, "rgb(8,81,156)" ], [ 1, "rgb(8,48,107)" ] ], "reversescale": false, "showscale": true, "type": "heatmap", "x": [ "Predicted: No Catch", "Predicted: Catch" ], "y": [ "Actual: No Catch", "Actual: Catch" ], "z": [ [ 31, 9 ], [ 0, 9 ] ] } ], "layout": { "annotations": [ { "font": { "color": "#FFFFFF" }, "showarrow": false, "text": "31", "x": "Predicted: No Catch", "xref": "x", "y": "Actual: No Catch", "yref": "y" }, { "font": { "color": "#000000" }, "showarrow": false, "text": "9", "x": "Predicted: Catch", "xref": "x", "y": "Actual: No Catch", "yref": "y" }, { "font": { "color": "#000000" }, "showarrow": false, "text": "0", "x": "Predicted: No Catch", "xref": "x", "y": "Actual: Catch", "yref": "y" }, { "font": { "color": "#000000" }, "showarrow": false, "text": "9", "x": "Predicted: Catch", "xref": "x", "y": "Actual: Catch", "yref": "y" } ], "height": 400, "template": { "data": { "bar": [ { "error_x": { "color": "#2a3f5f" }, "error_y": { "color": "#2a3f5f" }, "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "bar" } ], "barpolar": [ { "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "barpolar" } ], "carpet": [ { "aaxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "baxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "type": "carpet" } ], "choropleth": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "choropleth" } ], "contour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "contour" } ], "contourcarpet": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "contourcarpet" } ], "heatmap": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "heatmap" } ], "heatmapgl": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "heatmapgl" } ], "histogram": [ { "marker": { "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "histogram" } ], "histogram2d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "histogram2d" } ], "histogram2dcontour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "histogram2dcontour" } ], "mesh3d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "mesh3d" } ], "parcoords": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "parcoords" } ], "pie": [ { "automargin": true, "type": "pie" } ], "scatter": [ { "fillpattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 }, "type": "scatter" } ], "scatter3d": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatter3d" } ], "scattercarpet": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattercarpet" } ], "scattergeo": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergeo" } ], "scattergl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergl" } ], "scattermapbox": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattermapbox" } ], "scatterpolar": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolar" } ], "scatterpolargl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolargl" } ], "scatterternary": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterternary" } ], "surface": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "surface" } ], "table": [ { "cells": { "fill": { "color": "#EBF0F8" }, "line": { "color": "white" } }, "header": { "fill": { "color": "#C8D4E3" }, "line": { "color": "white" } }, "type": "table" } ] }, "layout": { "annotationdefaults": { "arrowcolor": "#2a3f5f", "arrowhead": 0, "arrowwidth": 1 }, "autotypenumbers": "strict", "coloraxis": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "colorscale": { "diverging": [ [ 0, "#8e0152" ], [ 0.1, "#c51b7d" ], [ 0.2, "#de77ae" ], [ 0.3, "#f1b6da" ], [ 0.4, "#fde0ef" ], [ 0.5, "#f7f7f7" ], [ 0.6, "#e6f5d0" ], [ 0.7, "#b8e186" ], [ 0.8, "#7fbc41" ], [ 0.9, "#4d9221" ], [ 1, "#276419" ] ], "sequential": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "sequentialminus": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ] }, "colorway": [ "#636efa", "#EF553B", "#00cc96", "#ab63fa", "#FFA15A", "#19d3f3", "#FF6692", "#B6E880", "#FF97FF", "#FECB52" ], "font": { "color": "#2a3f5f" }, "geo": { "bgcolor": "white", "lakecolor": "white", "landcolor": "#E5ECF6", "showlakes": true, "showland": true, "subunitcolor": "white" }, "hoverlabel": { "align": "left" }, "hovermode": "closest", "mapbox": { "style": "light" }, "paper_bgcolor": "white", "plot_bgcolor": "#E5ECF6", "polar": { "angularaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "radialaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "scene": { "xaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "yaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "zaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" } }, "shapedefaults": { "line": { "color": "#2a3f5f" } }, "ternary": { "aaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "baxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "caxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "title": { "x": 0.05 }, "xaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 }, "yaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 } } }, "title": { "text": "🌳 RandomForest Confusion Matrix", "x": 0.5 }, "width": 500, "xaxis": { "dtick": 1, "gridcolor": "rgb(0, 0, 0)", "side": "top", "ticks": "" }, "yaxis": { "dtick": 1, "ticks": "", "ticksuffix": " " } } } }, "metadata": {}, "output_type": "display_data" }, { "data": { "application/vnd.plotly.v1+json": { "config": { "plotlyServerURL": "https://plot.ly" }, "data": [ { "line": { "color": "green", "width": 3 }, "mode": "lines", "name": "RandomForest (AUC = 0.919)", "type": "scatter", "x": [ 0, 0, 0, 0.025, 0.075, 0.1, 0.1, 0.125, 0.125, 0.375, 0.425, 0.825, 0.875, 1 ], "y": [ 0, 0.1111111111111111, 0.2222222222222222, 0.2222222222222222, 0.2222222222222222, 0.2222222222222222, 0.8888888888888888, 0.8888888888888888, 1, 1, 1, 1, 1, 1 ] }, { "line": { "color": "red", "dash": "dash", "width": 2 }, "mode": "lines", "name": "Random Classifier", "type": "scatter", "x": [ 0, 1 ], "y": [ 0, 1 ] } ], "layout": { "height": 500, "template": { "data": { "bar": [ { "error_x": { "color": "#2a3f5f" }, "error_y": { "color": "#2a3f5f" }, "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "bar" } ], "barpolar": [ { "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "barpolar" } ], "carpet": [ { "aaxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "baxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "type": "carpet" } ], "choropleth": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "choropleth" } ], "contour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "contour" } ], "contourcarpet": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "contourcarpet" } ], "heatmap": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "heatmap" } ], "heatmapgl": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "heatmapgl" } ], "histogram": [ { "marker": { "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "histogram" } ], "histogram2d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "histogram2d" } ], "histogram2dcontour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "histogram2dcontour" } ], "mesh3d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "mesh3d" } ], "parcoords": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "parcoords" } ], "pie": [ { "automargin": true, "type": "pie" } ], "scatter": [ { "fillpattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 }, "type": "scatter" } ], "scatter3d": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatter3d" } ], "scattercarpet": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattercarpet" } ], "scattergeo": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergeo" } ], "scattergl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergl" } ], "scattermapbox": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattermapbox" } ], "scatterpolar": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolar" } ], "scatterpolargl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolargl" } ], "scatterternary": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterternary" } ], "surface": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "surface" } ], "table": [ { "cells": { "fill": { "color": "#EBF0F8" }, "line": { "color": "white" } }, "header": { "fill": { "color": "#C8D4E3" }, "line": { "color": "white" } }, "type": "table" } ] }, "layout": { "annotationdefaults": { "arrowcolor": "#2a3f5f", "arrowhead": 0, "arrowwidth": 1 }, "autotypenumbers": "strict", "coloraxis": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "colorscale": { "diverging": [ [ 0, "#8e0152" ], [ 0.1, "#c51b7d" ], [ 0.2, "#de77ae" ], [ 0.3, "#f1b6da" ], [ 0.4, "#fde0ef" ], [ 0.5, "#f7f7f7" ], [ 0.6, "#e6f5d0" ], [ 0.7, "#b8e186" ], [ 0.8, "#7fbc41" ], [ 0.9, "#4d9221" ], [ 1, "#276419" ] ], "sequential": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "sequentialminus": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ] }, "colorway": [ "#636efa", "#EF553B", "#00cc96", "#ab63fa", "#FFA15A", "#19d3f3", "#FF6692", "#B6E880", "#FF97FF", "#FECB52" ], "font": { "color": "#2a3f5f" }, "geo": { "bgcolor": "white", "lakecolor": "white", "landcolor": "#E5ECF6", "showlakes": true, "showland": true, "subunitcolor": "white" }, "hoverlabel": { "align": "left" }, "hovermode": "closest", "mapbox": { "style": "light" }, "paper_bgcolor": "white", "plot_bgcolor": "#E5ECF6", "polar": { "angularaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "radialaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "scene": { "xaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "yaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "zaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" } }, "shapedefaults": { "line": { "color": "#2a3f5f" } }, "ternary": { "aaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "baxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "caxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "title": { "x": 0.05 }, "xaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 }, "yaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 } } }, "title": { "text": "🌳 RandomForest ROC Curve", "x": 0.5 }, "width": 600, "xaxis": { "title": { "text": "False Positive Rate" } }, "yaxis": { "title": { "text": "True Positive Rate" } } } } }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "\n", "🎯 Top 10 Most Important Features:\n", " 1. Recent_Activity: 0.3403\n", " 2. Days_Since_Cleaning: 0.1148\n", " 3. Location_Code: 0.0998\n", " 4. Temp_Avg_3d: 0.0703\n", " 5. Insects_Lag1: 0.0647\n", " 6. Average Humidity: 0.0631\n", " 7. Day: 0.0624\n", " 8. Average Temperature: 0.0490\n", " 9. Temp_Range: 0.0434\n", " 10. Humidity_Avg_3d: 0.0406\n" ] }, { "data": { "application/vnd.plotly.v1+json": { "config": { "plotlyServerURL": "https://plot.ly" }, "data": [ { "alignmentgroup": "True", "hovertemplate": "Feature Importance=%{x}
Features=%{y}", "legendgroup": "", "marker": { "color": "#636efa", "pattern": { "shape": "" } }, "name": "", "offsetgroup": "", "orientation": "h", "showlegend": false, "textposition": "auto", "type": "bar", "x": [ 0.3402541452633382, 0.11480168877216199, 0.09975014369710003, 0.07033907595270707, 0.06465227471827, 0.06309828112342333, 0.06235613233443926, 0.04902998110544893, 0.04343416155151357, 0.040573170301179726 ], "xaxis": "x", "y": [ "Recent_Activity", "Days_Since_Cleaning", "Location_Code", "Temp_Avg_3d", "Insects_Lag1", "Average Humidity", "Day", "Average Temperature", "Temp_Range", "Humidity_Avg_3d" ], "yaxis": "y" } ], "layout": { "barmode": "relative", "height": 500, "legend": { "tracegroupgap": 0 }, "template": { "data": { "bar": [ { "error_x": { "color": "#2a3f5f" }, "error_y": { "color": "#2a3f5f" }, "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "bar" } ], "barpolar": [ { "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "barpolar" } ], "carpet": [ { "aaxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "baxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "type": "carpet" } ], "choropleth": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "choropleth" } ], "contour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "contour" } ], "contourcarpet": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "contourcarpet" } ], "heatmap": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "heatmap" } ], "heatmapgl": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "heatmapgl" } ], "histogram": [ { "marker": { "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "histogram" } ], "histogram2d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "histogram2d" } ], "histogram2dcontour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "histogram2dcontour" } ], "mesh3d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "mesh3d" } ], "parcoords": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "parcoords" } ], "pie": [ { "automargin": true, "type": "pie" } ], "scatter": [ { "fillpattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 }, "type": "scatter" } ], "scatter3d": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatter3d" } ], "scattercarpet": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattercarpet" } ], "scattergeo": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergeo" } ], "scattergl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergl" } ], "scattermapbox": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattermapbox" } ], "scatterpolar": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolar" } ], "scatterpolargl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolargl" } ], "scatterternary": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterternary" } ], "surface": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "surface" } ], "table": [ { "cells": { "fill": { "color": "#EBF0F8" }, "line": { "color": "white" } }, "header": { "fill": { "color": "#C8D4E3" }, "line": { "color": "white" } }, "type": "table" } ] }, "layout": { "annotationdefaults": { "arrowcolor": "#2a3f5f", "arrowhead": 0, "arrowwidth": 1 }, "autotypenumbers": "strict", "coloraxis": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "colorscale": { "diverging": [ [ 0, "#8e0152" ], [ 0.1, "#c51b7d" ], [ 0.2, "#de77ae" ], [ 0.3, "#f1b6da" ], [ 0.4, "#fde0ef" ], [ 0.5, "#f7f7f7" ], [ 0.6, "#e6f5d0" ], [ 0.7, "#b8e186" ], [ 0.8, "#7fbc41" ], [ 0.9, "#4d9221" ], [ 1, "#276419" ] ], "sequential": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "sequentialminus": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ] }, "colorway": [ "#636efa", "#EF553B", "#00cc96", "#ab63fa", "#FFA15A", "#19d3f3", "#FF6692", "#B6E880", "#FF97FF", "#FECB52" ], "font": { "color": "#2a3f5f" }, "geo": { "bgcolor": "white", "lakecolor": "white", "landcolor": "#E5ECF6", "showlakes": true, "showland": true, "subunitcolor": "white" }, "hoverlabel": { "align": "left" }, "hovermode": "closest", "mapbox": { "style": "light" }, "paper_bgcolor": "white", "plot_bgcolor": "#E5ECF6", "polar": { "angularaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "radialaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "scene": { "xaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "yaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "zaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" } }, "shapedefaults": { "line": { "color": "#2a3f5f" } }, "ternary": { "aaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "baxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "caxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "title": { "x": 0.05 }, "xaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 }, "yaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 } } }, "title": { "text": "🌳 RandomForest - Top 10 Feature Importances" }, "xaxis": { "anchor": "y", "domain": [ 0, 1 ], "title": { "text": "Feature Importance" } }, "yaxis": { "anchor": "x", "categoryorder": "total ascending", "domain": [ 0, 1 ], "title": { "text": "Features" } } } } }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "\n", "πŸ† RandomForest Tournament Summary:\n", " β€’ F1-Score: 0.6667\n", " β€’ AUC: 0.9194\n", " β€’ Optimal Threshold: 0.100\n", " β€’ Test Accuracy: 0.8163\n", " β€’ Best Parameters: {'max_depth': 10, 'max_features': 'sqrt', 'min_samples_leaf': 2, 'min_samples_split': 5, 'n_estimators': 100}\n" ] } ], "source": [ "# Generate comprehensive classification report\n", "print(\"πŸ“‹ RandomForest Classification Report:\")\n", "print(\"=\" * 60)\n", "print(classification_report(y_test, y_test_pred_rf, target_names=['No Catch', 'Catch']))\n", "\n", "# Create interactive confusion matrix\n", "cm_rf = confusion_matrix(y_test, y_test_pred_rf)\n", "\n", "fig_cm_rf = ff.create_annotated_heatmap(\n", " z=cm_rf,\n", " x=['Predicted: No Catch', 'Predicted: Catch'],\n", " y=['Actual: No Catch', 'Actual: Catch'],\n", " annotation_text=cm_rf,\n", " colorscale='Blues',\n", " showscale=True\n", ")\n", "\n", "fig_cm_rf.update_layout(\n", " title='🌳 RandomForest Confusion Matrix',\n", " title_x=0.5,\n", " width=500,\n", " height=400\n", ")\n", "\n", "fig_cm_rf.show()\n", "\n", "# Create interactive ROC curve\n", "fpr_rf, tpr_rf, _ = roc_curve(y_test, y_test_proba_rf)\n", "\n", "fig_roc_rf = go.Figure()\n", "\n", "fig_roc_rf.add_trace(go.Scatter(\n", " x=fpr_rf,\n", " y=tpr_rf,\n", " mode='lines',\n", " name=f'RandomForest (AUC = {rf_test_auc:.3f})',\n", " line=dict(color='green', width=3)\n", "))\n", "\n", "# Add diagonal reference line\n", "fig_roc_rf.add_trace(go.Scatter(\n", " x=[0, 1],\n", " y=[0, 1],\n", " mode='lines',\n", " name='Random Classifier',\n", " line=dict(color='red', width=2, dash='dash')\n", "))\n", "\n", "fig_roc_rf.update_layout(\n", " title='🌳 RandomForest ROC Curve',\n", " title_x=0.5,\n", " xaxis_title='False Positive Rate',\n", " yaxis_title='True Positive Rate',\n", " width=600,\n", " height=500\n", ")\n", "\n", "fig_roc_rf.show()\n", "\n", "# Feature importance analysis\n", "feature_importance = pd.DataFrame({\n", " 'feature': X_train_rf.columns,\n", " 'importance': rf_final.feature_importances_\n", "}).sort_values('importance', ascending=False)\n", "\n", "print(f\"\\n🎯 Top 10 Most Important Features:\")\n", "for i, (idx, row) in enumerate(feature_importance.head(10).iterrows()):\n", " print(f\" {i+1}. {row['feature']}: {row['importance']:.4f}\")\n", "\n", "# Create feature importance plot\n", "fig_importance = px.bar(\n", " feature_importance.head(10),\n", " x='importance',\n", " y='feature',\n", " orientation='h',\n", " title='🌳 RandomForest - Top 10 Feature Importances',\n", " labels={'importance': 'Feature Importance', 'feature': 'Features'}\n", ")\n", "fig_importance.update_layout(height=500, yaxis={'categoryorder':'total ascending'})\n", "fig_importance.show()\n", "\n", "print(f\"\\nπŸ† RandomForest Tournament Summary:\")\n", "print(f\" β€’ F1-Score: {rf_test_f1:.4f}\")\n", "print(f\" β€’ AUC: {rf_test_auc:.4f}\")\n", "print(f\" β€’ Optimal Threshold: {optimal_threshold_rf:.3f}\")\n", "print(f\" β€’ Test Accuracy: {rf_test_accuracy:.4f}\")\n", "print(f\" β€’ Best Parameters: {rf_results['best_params']}\")" ] }, { "cell_type": "markdown", "id": "5c49a3d1", "metadata": {}, "source": [ "## πŸš€ Contestant 2: XGBoostClassifier\n", "\n", "Enter our second competitor: XGBoost, the gradient boosting champion! Known for its exceptional performance in machine learning competitions, XGBoost uses advanced optimization techniques and handles missing values naturally." ] }, { "cell_type": "markdown", "id": "6a8a2402", "metadata": {}, "source": [ "### πŸ“Š XGBoost Performance Analysis\n", "\n", "Time to analyze our gradient boosting champion's performance!" ] }, { "cell_type": "code", "execution_count": 20, "id": "f57589de", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "πŸš€ Training XGBoostClassifier...\n", "==================================================\n", "πŸ“Š Using 17 features for XGBoost modeling\n", "βœ… Training set: (196, 17)\n", "βœ… Test set: (49, 17)\n", "πŸ”§ Hyperparameter combinations to test: 108\n", "πŸ” Performing hyperparameter tuning with TimeSeriesSplit...\n", "Fitting 3 folds for each of 108 candidates, totalling 324 fits\n", "βœ… Best parameters found: {'colsample_bytree': 0.8, 'learning_rate': 0.01, 'max_depth': 3, 'n_estimators': 100, 'subsample': 1.0}\n", "🎯 Best CV F1-score: 0.4563\n", "\n", "πŸ”§ Finding optimal prediction threshold using temporal validation...\n", "🎯 Optimal threshold: 0.100\n", "πŸ† Validation F1-score at optimal threshold: 0.0000\n", "\n", "πŸ”„ Retraining on full training set...\n", "\n", "πŸ“Š Evaluating on test set...\n", "🎯 Test F1-score: 0.3103\n", "πŸ“ˆ Test AUC: 0.9319\n", "🎯 Test Accuracy: 0.1837\n", "\n", "βœ… XGBoostClassifier training complete!\n", "πŸ“‹ XGBoost Classification Report:\n", "============================================================\n", " precision recall f1-score support\n", "\n", " No Catch 0.00 0.00 0.00 40\n", " Catch 0.18 1.00 0.31 9\n", "\n", " accuracy 0.18 49\n", " macro avg 0.09 0.50 0.16 49\n", "weighted avg 0.03 0.18 0.06 49\n", "\n" ] }, { "data": { "application/vnd.plotly.v1+json": { "config": { "plotlyServerURL": "https://plot.ly" }, "data": [ { "colorscale": [ [ 0, "rgb(255,245,235)" ], [ 0.125, "rgb(254,230,206)" ], [ 0.25, "rgb(253,208,162)" ], [ 0.375, "rgb(253,174,107)" ], [ 0.5, "rgb(253,141,60)" ], [ 0.625, "rgb(241,105,19)" ], [ 0.75, "rgb(217,72,1)" ], [ 0.875, "rgb(166,54,3)" ], [ 1, "rgb(127,39,4)" ] ], "reversescale": false, "showscale": true, "type": "heatmap", "x": [ "Predicted: No Catch", "Predicted: Catch" ], "y": [ "Actual: No Catch", "Actual: Catch" ], "z": [ [ 0, 40 ], [ 0, 9 ] ] } ], "layout": { "annotations": [ { "font": { "color": "#000000" }, "showarrow": false, "text": "0", "x": "Predicted: No Catch", "xref": "x", "y": "Actual: No Catch", "yref": "y" }, { "font": { "color": "#FFFFFF" }, "showarrow": false, "text": "40", "x": "Predicted: Catch", "xref": "x", "y": "Actual: No Catch", "yref": "y" }, { "font": { "color": "#000000" }, "showarrow": false, "text": "0", "x": "Predicted: No Catch", "xref": "x", "y": "Actual: Catch", "yref": "y" }, { "font": { "color": "#000000" }, "showarrow": false, "text": "9", "x": "Predicted: Catch", "xref": "x", "y": "Actual: Catch", "yref": "y" } ], "height": 400, "template": { "data": { "bar": [ { "error_x": { "color": "#2a3f5f" }, "error_y": { "color": "#2a3f5f" }, "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "bar" } ], "barpolar": [ { "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "barpolar" } ], "carpet": [ { "aaxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "baxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "type": "carpet" } ], "choropleth": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "choropleth" } ], "contour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "contour" } ], "contourcarpet": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "contourcarpet" } ], "heatmap": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "heatmap" } ], "heatmapgl": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "heatmapgl" } ], "histogram": [ { "marker": { "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "histogram" } ], "histogram2d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "histogram2d" } ], "histogram2dcontour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "histogram2dcontour" } ], "mesh3d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "mesh3d" } ], "parcoords": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "parcoords" } ], "pie": [ { "automargin": true, "type": "pie" } ], "scatter": [ { "fillpattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 }, "type": "scatter" } ], "scatter3d": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatter3d" } ], "scattercarpet": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattercarpet" } ], "scattergeo": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergeo" } ], "scattergl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergl" } ], "scattermapbox": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattermapbox" } ], "scatterpolar": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolar" } ], "scatterpolargl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolargl" } ], "scatterternary": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterternary" } ], "surface": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "surface" } ], "table": [ { "cells": { "fill": { "color": "#EBF0F8" }, "line": { "color": "white" } }, "header": { "fill": { "color": "#C8D4E3" }, "line": { "color": "white" } }, "type": "table" } ] }, "layout": { "annotationdefaults": { "arrowcolor": "#2a3f5f", "arrowhead": 0, "arrowwidth": 1 }, "autotypenumbers": "strict", "coloraxis": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "colorscale": { "diverging": [ [ 0, "#8e0152" ], [ 0.1, "#c51b7d" ], [ 0.2, "#de77ae" ], [ 0.3, "#f1b6da" ], [ 0.4, "#fde0ef" ], [ 0.5, "#f7f7f7" ], [ 0.6, "#e6f5d0" ], [ 0.7, "#b8e186" ], [ 0.8, "#7fbc41" ], [ 0.9, "#4d9221" ], [ 1, "#276419" ] ], "sequential": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "sequentialminus": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ] }, "colorway": [ "#636efa", "#EF553B", "#00cc96", "#ab63fa", "#FFA15A", "#19d3f3", "#FF6692", "#B6E880", "#FF97FF", "#FECB52" ], "font": { "color": "#2a3f5f" }, "geo": { "bgcolor": "white", "lakecolor": "white", "landcolor": "#E5ECF6", "showlakes": true, "showland": true, "subunitcolor": "white" }, "hoverlabel": { "align": "left" }, "hovermode": "closest", "mapbox": { "style": "light" }, "paper_bgcolor": "white", "plot_bgcolor": "#E5ECF6", "polar": { "angularaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "radialaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "scene": { "xaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "yaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "zaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" } }, "shapedefaults": { "line": { "color": "#2a3f5f" } }, "ternary": { "aaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "baxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "caxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "title": { "x": 0.05 }, "xaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 }, "yaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 } } }, "title": { "text": "πŸš€ XGBoost Confusion Matrix", "x": 0.5 }, "width": 500, "xaxis": { "dtick": 1, "gridcolor": "rgb(0, 0, 0)", "side": "top", "ticks": "" }, "yaxis": { "dtick": 1, "ticks": "", "ticksuffix": " " } } } }, "metadata": {}, "output_type": "display_data" }, { "data": { "application/vnd.plotly.v1+json": { "config": { "plotlyServerURL": "https://plot.ly" }, "data": [ { "line": { "color": "orange", "width": 3 }, "mode": "lines", "name": "XGBoost (AUC = 0.932)", "type": "scatter", "x": [ 0, 0, 0, 0.025, 0.125, 0.125, 0.15, 0.425, 0.5, 0.625, 0.65, 0.775, 0.85, 1 ], "y": [ 0, 0.1111111111111111, 0.2222222222222222, 0.3333333333333333, 0.6666666666666666, 1, 1, 1, 1, 1, 1, 1, 1, 1 ] }, { "line": { "color": "red", "dash": "dash", "width": 2 }, "mode": "lines", "name": "Random Classifier", "type": "scatter", "x": [ 0, 1 ], "y": [ 0, 1 ] } ], "layout": { "height": 500, "template": { "data": { "bar": [ { "error_x": { "color": "#2a3f5f" }, "error_y": { "color": "#2a3f5f" }, "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "bar" } ], "barpolar": [ { "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "barpolar" } ], "carpet": [ { "aaxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "baxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "type": "carpet" } ], "choropleth": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "choropleth" } ], "contour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "contour" } ], "contourcarpet": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "contourcarpet" } ], "heatmap": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "heatmap" } ], "heatmapgl": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "heatmapgl" } ], "histogram": [ { "marker": { "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "histogram" } ], "histogram2d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "histogram2d" } ], "histogram2dcontour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "histogram2dcontour" } ], "mesh3d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "mesh3d" } ], "parcoords": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "parcoords" } ], "pie": [ { "automargin": true, "type": "pie" } ], "scatter": [ { "fillpattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 }, "type": "scatter" } ], "scatter3d": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatter3d" } ], "scattercarpet": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattercarpet" } ], "scattergeo": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergeo" } ], "scattergl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergl" } ], "scattermapbox": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattermapbox" } ], "scatterpolar": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolar" } ], "scatterpolargl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolargl" } ], "scatterternary": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterternary" } ], "surface": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "surface" } ], "table": [ { "cells": { "fill": { "color": "#EBF0F8" }, "line": { "color": "white" } }, "header": { "fill": { "color": "#C8D4E3" }, "line": { "color": "white" } }, "type": "table" } ] }, "layout": { "annotationdefaults": { "arrowcolor": "#2a3f5f", "arrowhead": 0, "arrowwidth": 1 }, "autotypenumbers": "strict", "coloraxis": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "colorscale": { "diverging": [ [ 0, "#8e0152" ], [ 0.1, "#c51b7d" ], [ 0.2, "#de77ae" ], [ 0.3, "#f1b6da" ], [ 0.4, "#fde0ef" ], [ 0.5, "#f7f7f7" ], [ 0.6, "#e6f5d0" ], [ 0.7, "#b8e186" ], [ 0.8, "#7fbc41" ], [ 0.9, "#4d9221" ], [ 1, "#276419" ] ], "sequential": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "sequentialminus": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ] }, "colorway": [ "#636efa", "#EF553B", "#00cc96", "#ab63fa", "#FFA15A", "#19d3f3", "#FF6692", "#B6E880", "#FF97FF", "#FECB52" ], "font": { "color": "#2a3f5f" }, "geo": { "bgcolor": "white", "lakecolor": "white", "landcolor": "#E5ECF6", "showlakes": true, "showland": true, "subunitcolor": "white" }, "hoverlabel": { "align": "left" }, "hovermode": "closest", "mapbox": { "style": "light" }, "paper_bgcolor": "white", "plot_bgcolor": "#E5ECF6", "polar": { "angularaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "radialaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "scene": { "xaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "yaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "zaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" } }, "shapedefaults": { "line": { "color": "#2a3f5f" } }, "ternary": { "aaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "baxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "caxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "title": { "x": 0.05 }, "xaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 }, "yaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 } } }, "title": { "text": "πŸš€ XGBoost ROC Curve", "x": 0.5 }, "width": 600, "xaxis": { "title": { "text": "False Positive Rate" } }, "yaxis": { "title": { "text": "True Positive Rate" } } } } }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "\n", "🎯 XGBoost - Top 10 Most Important Features:\n", " 1. Recent_Activity: 0.4746\n", " 2. Location_Code: 0.2509\n", " 3. Days_Since_Cleaning: 0.1385\n", " 4. Day: 0.0445\n", " 5. Average Humidity: 0.0334\n", " 6. Temp_Avg_3d: 0.0274\n", " 7. Humidity_Avg_3d: 0.0110\n", " 8. Temp_Range: 0.0103\n", " 9. Average Temperature: 0.0051\n", " 10. Month: 0.0042\n" ] }, { "data": { "application/vnd.plotly.v1+json": { "config": { "plotlyServerURL": "https://plot.ly" }, "data": [ { "alignmentgroup": "True", "hovertemplate": "Feature Importance=%{x}
Features=%{y}", "legendgroup": "", "marker": { "color": "orange", "pattern": { "shape": "" } }, "name": "", "offsetgroup": "", "orientation": "h", "showlegend": false, "textposition": "auto", "type": "bar", "x": [ 0.47456276416778564, 0.2509356439113617, 0.1385187953710556, 0.04454922303557396, 0.033442772924900055, 0.027408014982938766, 0.010977267287671566, 0.010266072116792202, 0.005119608249515295, 0.00421995110809803 ], "xaxis": "x", "y": [ "Recent_Activity", "Location_Code", "Days_Since_Cleaning", "Day", "Average Humidity", "Temp_Avg_3d", "Humidity_Avg_3d", "Temp_Range", "Average Temperature", "Month" ], "yaxis": "y" } ], "layout": { "barmode": "relative", "height": 500, "legend": { "tracegroupgap": 0 }, "template": { "data": { "bar": [ { "error_x": { "color": "#2a3f5f" }, "error_y": { "color": "#2a3f5f" }, "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "bar" } ], "barpolar": [ { "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "barpolar" } ], "carpet": [ { "aaxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "baxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "type": "carpet" } ], "choropleth": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "choropleth" } ], "contour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "contour" } ], "contourcarpet": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "contourcarpet" } ], "heatmap": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "heatmap" } ], "heatmapgl": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "heatmapgl" } ], "histogram": [ { "marker": { "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "histogram" } ], "histogram2d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "histogram2d" } ], "histogram2dcontour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "histogram2dcontour" } ], "mesh3d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "mesh3d" } ], "parcoords": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "parcoords" } ], "pie": [ { "automargin": true, "type": "pie" } ], "scatter": [ { "fillpattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 }, "type": "scatter" } ], "scatter3d": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatter3d" } ], "scattercarpet": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattercarpet" } ], "scattergeo": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergeo" } ], "scattergl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergl" } ], "scattermapbox": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattermapbox" } ], "scatterpolar": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolar" } ], "scatterpolargl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolargl" } ], "scatterternary": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterternary" } ], "surface": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "surface" } ], "table": [ { "cells": { "fill": { "color": "#EBF0F8" }, "line": { "color": "white" } }, "header": { "fill": { "color": "#C8D4E3" }, "line": { "color": "white" } }, "type": "table" } ] }, "layout": { "annotationdefaults": { "arrowcolor": "#2a3f5f", "arrowhead": 0, "arrowwidth": 1 }, "autotypenumbers": "strict", "coloraxis": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "colorscale": { "diverging": [ [ 0, "#8e0152" ], [ 0.1, "#c51b7d" ], [ 0.2, "#de77ae" ], [ 0.3, "#f1b6da" ], [ 0.4, "#fde0ef" ], [ 0.5, "#f7f7f7" ], [ 0.6, "#e6f5d0" ], [ 0.7, "#b8e186" ], [ 0.8, "#7fbc41" ], [ 0.9, "#4d9221" ], [ 1, "#276419" ] ], "sequential": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "sequentialminus": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ] }, "colorway": [ "#636efa", "#EF553B", "#00cc96", "#ab63fa", "#FFA15A", "#19d3f3", "#FF6692", "#B6E880", "#FF97FF", "#FECB52" ], "font": { "color": "#2a3f5f" }, "geo": { "bgcolor": "white", "lakecolor": "white", "landcolor": "#E5ECF6", "showlakes": true, "showland": true, "subunitcolor": "white" }, "hoverlabel": { "align": "left" }, "hovermode": "closest", "mapbox": { "style": "light" }, "paper_bgcolor": "white", "plot_bgcolor": "#E5ECF6", "polar": { "angularaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "radialaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "scene": { "xaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "yaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "zaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" } }, "shapedefaults": { "line": { "color": "#2a3f5f" } }, "ternary": { "aaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "baxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "caxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "title": { "x": 0.05 }, "xaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 }, "yaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 } } }, "title": { "text": "πŸš€ XGBoost - Top 10 Feature Importances" }, "xaxis": { "anchor": "y", "domain": [ 0, 1 ], "title": { "text": "Feature Importance" } }, "yaxis": { "anchor": "x", "categoryorder": "total ascending", "domain": [ 0, 1 ], "title": { "text": "Features" } } } } }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "\n", "πŸ† XGBoost Tournament Summary:\n", " β€’ F1-Score: 0.3103\n", " β€’ AUC: 0.9319\n", " β€’ Optimal Threshold: 0.100\n", " β€’ Test Accuracy: 0.1837\n", " β€’ Best Parameters: {'colsample_bytree': 0.8, 'learning_rate': 0.01, 'max_depth': 3, 'n_estimators': 100, 'subsample': 1.0}\n" ] } ], "source": [ "print(\"πŸš€ Training XGBoostClassifier...\")\n", "print(\"=\" * 50)\n", "\n", "# Use all available features (already cleaned of leakage)\n", "X_train_xgb = X_train.copy()\n", "X_test_xgb = X_test.copy()\n", "\n", "print(f\"πŸ“Š Using {len(X_train_xgb.columns)} features for XGBoost modeling\")\n", "print(f\"βœ… Training set: {X_train_xgb.shape}\")\n", "print(f\"βœ… Test set: {X_test_xgb.shape}\")\n", "\n", "# Use focused hyperparameter grid for XGBoost\n", "xgb_param_grid = {\n", " 'n_estimators': [100, 200, 300],\n", " 'max_depth': [3, 5, 7],\n", " 'learning_rate': [0.01, 0.1, 0.2],\n", " 'subsample': [0.8, 1.0],\n", " 'colsample_bytree': [0.8, 1.0]\n", "}\n", "\n", "total_combinations = (len(xgb_param_grid['n_estimators']) * len(xgb_param_grid['max_depth']) * \n", " len(xgb_param_grid['learning_rate']) * len(xgb_param_grid['subsample']) * \n", " len(xgb_param_grid['colsample_bytree']))\n", "print(f\"πŸ”§ Hyperparameter combinations to test: {total_combinations}\")\n", "\n", "# Calculate scale_pos_weight for XGBoost (equivalent to class_weight='balanced')\n", "scale_pos_weight = len(y_train[y_train == 0]) / len(y_train[y_train == 1])\n", "\n", "# Initialize XGBoost with class weights\n", "xgb_base = XGBClassifier(\n", " scale_pos_weight=scale_pos_weight,\n", " random_state=42,\n", " n_jobs=-1,\n", " eval_metric='logloss'\n", ")\n", "\n", "print(\"πŸ” Performing hyperparameter tuning with TimeSeriesSplit...\")\n", "xgb_grid = GridSearchCV(\n", " xgb_base,\n", " xgb_param_grid,\n", " cv=tscv, # Use TimeSeriesSplit\n", " scoring='f1',\n", " n_jobs=-1,\n", " verbose=1\n", ")\n", "\n", "xgb_grid.fit(X_train_xgb, y_train)\n", "\n", "print(f\"βœ… Best parameters found: {xgb_grid.best_params_}\")\n", "print(f\"🎯 Best CV F1-score: {xgb_grid.best_score_:.4f}\")\n", "\n", "# Get the best model\n", "xgb_best = xgb_grid.best_estimator_\n", "\n", "# Train on the CV training set for threshold optimization\n", "xgb_threshold_model = XGBClassifier(**xgb_grid.best_params_, scale_pos_weight=scale_pos_weight, random_state=42, n_jobs=-1, eval_metric='logloss')\n", "xgb_threshold_model.fit(X_train_cv_rf, y_train_cv_rf)\n", "\n", "print(f\"\\nπŸ”§ Finding optimal prediction threshold using temporal validation...\")\n", "\n", "# Get prediction probabilities for threshold tuning\n", "y_val_proba_xgb = xgb_threshold_model.predict_proba(X_val_cv_rf)[:, 1]\n", "\n", "# Find optimal threshold using F1-score\n", "f1_scores_xgb = []\n", "\n", "for threshold in thresholds:\n", " y_pred_thresh = (y_val_proba_xgb >= threshold).astype(int)\n", " if len(np.unique(y_pred_thresh)) > 1: # Avoid division by zero\n", " f1 = f1_score(y_val_cv_rf, y_pred_thresh)\n", " else:\n", " f1 = 0.0\n", " f1_scores_xgb.append(f1)\n", "\n", "optimal_threshold_xgb = thresholds[np.argmax(f1_scores_xgb)]\n", "max_f1_xgb = max(f1_scores_xgb)\n", "\n", "print(f\"🎯 Optimal threshold: {optimal_threshold_xgb:.3f}\")\n", "print(f\"πŸ† Validation F1-score at optimal threshold: {max_f1_xgb:.4f}\")\n", "\n", "# Retrain on full training set with best parameters\n", "print(\"\\nπŸ”„ Retraining on full training set...\")\n", "xgb_final = XGBClassifier(**xgb_grid.best_params_, scale_pos_weight=scale_pos_weight, random_state=42, n_jobs=-1, eval_metric='logloss')\n", "xgb_final.fit(X_train_xgb, y_train)\n", "\n", "# Make predictions on test set with optimal threshold\n", "print(\"\\nπŸ“Š Evaluating on test set...\")\n", "y_test_proba_xgb = xgb_final.predict_proba(X_test_xgb)[:, 1]\n", "y_test_pred_xgb = (y_test_proba_xgb >= optimal_threshold_xgb).astype(int)\n", "\n", "# Calculate metrics\n", "xgb_test_f1 = f1_score(y_test, y_test_pred_xgb)\n", "xgb_test_auc = roc_auc_score(y_test, y_test_proba_xgb)\n", "xgb_test_accuracy = accuracy_score(y_test, y_test_pred_xgb)\n", "\n", "print(f\"🎯 Test F1-score: {xgb_test_f1:.4f}\")\n", "print(f\"πŸ“ˆ Test AUC: {xgb_test_auc:.4f}\")\n", "print(f\"🎯 Test Accuracy: {xgb_test_accuracy:.4f}\")\n", "\n", "# Store results for later comparison\n", "xgb_results = {\n", " 'model': xgb_final,\n", " 'threshold': optimal_threshold_xgb,\n", " 'test_f1': xgb_test_f1,\n", " 'test_auc': xgb_test_auc,\n", " 'test_accuracy': xgb_test_accuracy,\n", " 'y_pred': y_test_pred_xgb,\n", " 'y_proba': y_test_proba_xgb,\n", " 'best_params': xgb_grid.best_params_\n", "}\n", "\n", "print(\"\\nβœ… XGBoostClassifier training complete!\")\n", "\n", "# Generate comprehensive classification report\n", "print(\"πŸ“‹ XGBoost Classification Report:\")\n", "print(\"=\" * 60)\n", "print(classification_report(y_test, y_test_pred_xgb, target_names=['No Catch', 'Catch']))\n", "\n", "# Create interactive confusion matrix\n", "cm_xgb = confusion_matrix(y_test, y_test_pred_xgb)\n", "\n", "fig_cm_xgb = ff.create_annotated_heatmap(\n", " z=cm_xgb,\n", " x=['Predicted: No Catch', 'Predicted: Catch'],\n", " y=['Actual: No Catch', 'Actual: Catch'],\n", " annotation_text=cm_xgb,\n", " colorscale='Oranges',\n", " showscale=True\n", ")\n", "\n", "fig_cm_xgb.update_layout(\n", " title='πŸš€ XGBoost Confusion Matrix',\n", " title_x=0.5,\n", " width=500,\n", " height=400\n", ")\n", "\n", "fig_cm_xgb.show()\n", "\n", "# Create interactive ROC curve\n", "fpr_xgb, tpr_xgb, _ = roc_curve(y_test, y_test_proba_xgb)\n", "\n", "fig_roc_xgb = go.Figure()\n", "\n", "fig_roc_xgb.add_trace(go.Scatter(\n", " x=fpr_xgb,\n", " y=tpr_xgb,\n", " mode='lines',\n", " name=f'XGBoost (AUC = {xgb_test_auc:.3f})',\n", " line=dict(color='orange', width=3)\n", "))\n", "\n", "# Add diagonal reference line\n", "fig_roc_xgb.add_trace(go.Scatter(\n", " x=[0, 1],\n", " y=[0, 1],\n", " mode='lines',\n", " name='Random Classifier',\n", " line=dict(color='red', width=2, dash='dash')\n", "))\n", "\n", "fig_roc_xgb.update_layout(\n", " title='πŸš€ XGBoost ROC Curve',\n", " title_x=0.5,\n", " xaxis_title='False Positive Rate',\n", " yaxis_title='True Positive Rate',\n", " width=600,\n", " height=500\n", ")\n", "\n", "fig_roc_xgb.show()\n", "\n", "# Feature importance analysis for XGBoost\n", "feature_importance_xgb = pd.DataFrame({\n", " 'feature': X_train_xgb.columns,\n", " 'importance': xgb_final.feature_importances_\n", "}).sort_values('importance', ascending=False)\n", "\n", "print(f\"\\n🎯 XGBoost - Top 10 Most Important Features:\")\n", "for i, (idx, row) in enumerate(feature_importance_xgb.head(10).iterrows()):\n", " print(f\" {i+1}. {row['feature']}: {row['importance']:.4f}\")\n", "\n", "# Create feature importance plot\n", "fig_importance_xgb = px.bar(\n", " feature_importance_xgb.head(10),\n", " x='importance',\n", " y='feature',\n", " orientation='h',\n", " title='πŸš€ XGBoost - Top 10 Feature Importances',\n", " labels={'importance': 'Feature Importance', 'feature': 'Features'},\n", " color_discrete_sequence=['orange']\n", ")\n", "fig_importance_xgb.update_layout(height=500, yaxis={'categoryorder':'total ascending'})\n", "fig_importance_xgb.show()\n", "\n", "print(f\"\\nπŸ† XGBoost Tournament Summary:\")\n", "print(f\" β€’ F1-Score: {xgb_test_f1:.4f}\")\n", "print(f\" β€’ AUC: {xgb_test_auc:.4f}\")\n", "print(f\" β€’ Optimal Threshold: {optimal_threshold_xgb:.3f}\")\n", "print(f\" β€’ Test Accuracy: {xgb_test_accuracy:.4f}\")\n", "print(f\" β€’ Best Parameters: {xgb_results['best_params']}\")" ] }, { "cell_type": "code", "execution_count": 21, "id": "6b063ccb", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "πŸš€ Training XGBoostClassifier...\n", "==================================================\n", "πŸ“Š Using 17 features for XGBoost modeling\n", "βœ… Training set: (196, 17)\n", "βœ… Test set: (49, 17)\n", "βš–οΈ Scale positive weight: 15.333\n", "πŸ”§ Hyperparameter combinations to test: 972\n", "πŸ” Performing hyperparameter tuning with TimeSeriesSplit...\n", "Fitting 3 folds for each of 972 candidates, totalling 2916 fits\n", "βœ… Best parameters found: {'colsample_bytree': 0.9, 'learning_rate': 0.01, 'max_depth': 3, 'n_estimators': 200, 'reg_alpha': 0, 'reg_lambda': 1.1, 'subsample': 1.0}\n", "🎯 Best CV F1-score: 0.4722\n", "\n", "πŸ”§ Finding optimal prediction threshold for XGBoost...\n", "πŸ“Š CV Training set: (156, 17), CV Validation set: (40, 17)\n", "🎯 Optimal threshold: 0.100\n", "πŸ† Validation F1-score at optimal threshold: 0.0000\n", "\n", "πŸ”„ Retraining XGBoost on full training set...\n", "\n", "πŸ“Š Evaluating XGBoost on test set...\n", "🎯 Test F1-score: 0.4737\n", "πŸ“ˆ Test AUC: 0.9319\n", "🎯 Test Accuracy: 0.5918\n", "\n", "πŸ“‹ Detailed Classification Report:\n", " precision recall f1-score support\n", "\n", " No Catch 1.00 0.50 0.67 40\n", " Catch 0.31 1.00 0.47 9\n", "\n", " accuracy 0.59 49\n", " macro avg 0.66 0.75 0.57 49\n", "weighted avg 0.87 0.59 0.63 49\n", "\n", "\n", "βœ… XGBoostClassifier training complete!\n" ] } ], "source": [ "print(\"πŸš€ Training XGBoostClassifier...\")\n", "print(\"=\" * 50)\n", "\n", "# Use all available features for XGBoost\n", "X_train_xgb = X_train.copy()\n", "X_test_xgb = X_test.copy()\n", "\n", "print(f\"πŸ“Š Using {len(X_train_xgb.columns)} features for XGBoost modeling\")\n", "print(f\"βœ… Training set: {X_train_xgb.shape}\")\n", "print(f\"βœ… Test set: {X_test_xgb.shape}\")\n", "\n", "# Calculate scale_pos_weight for XGBoost (equivalent to class_weight='balanced')\n", "neg_count = (y_train == 0).sum()\n", "pos_count = (y_train == 1).sum()\n", "scale_pos_weight = neg_count / pos_count\n", "\n", "print(f\"βš–οΈ Scale positive weight: {scale_pos_weight:.3f}\")\n", "\n", "# Define XGBoost-specific hyperparameter grid\n", "xgb_param_grid = {\n", " 'n_estimators': [100, 200, 300],\n", " 'max_depth': [3, 4, 6], # Different from RandomForest\n", " 'learning_rate': [0.01, 0.1, 0.2], # XGBoost-specific\n", " 'subsample': [0.8, 0.9, 1.0], # XGBoost-specific\n", " 'colsample_bytree': [0.8, 0.9, 1.0], # XGBoost-specific\n", " 'reg_alpha': [0, 0.1], # L1 regularization\n", " 'reg_lambda': [1, 1.1] # L2 regularization\n", "}\n", "\n", "total_combinations_xgb = (len(xgb_param_grid['n_estimators']) * len(xgb_param_grid['max_depth']) * \n", " len(xgb_param_grid['learning_rate']) * len(xgb_param_grid['subsample']) * \n", " len(xgb_param_grid['colsample_bytree']) * len(xgb_param_grid['reg_alpha']) * \n", " len(xgb_param_grid['reg_lambda']))\n", "print(f\"πŸ”§ Hyperparameter combinations to test: {total_combinations_xgb}\")\n", "\n", "# Initialize XGBoost with class balancing\n", "xgb_base = XGBClassifier(\n", " scale_pos_weight=scale_pos_weight,\n", " random_state=42,\n", " eval_metric='logloss',\n", " verbosity=0\n", ")\n", "\n", "# Use TimeSeriesSplit for cross-validation\n", "print(\"πŸ” Performing hyperparameter tuning with TimeSeriesSplit...\")\n", "xgb_grid = GridSearchCV(\n", " xgb_base,\n", " xgb_param_grid,\n", " cv=tscv, # Use TimeSeriesSplit\n", " scoring='f1',\n", " n_jobs=-1,\n", " verbose=1\n", ")\n", "\n", "xgb_grid.fit(X_train_xgb, y_train)\n", "\n", "print(f\"βœ… Best parameters found: {xgb_grid.best_params_}\")\n", "print(f\"🎯 Best CV F1-score: {xgb_grid.best_score_:.4f}\")\n", "\n", "# Get the best model\n", "xgb_best = xgb_grid.best_estimator_\n", "\n", "# Use separate validation set for threshold tuning\n", "split_point_xgb = int(len(X_train_xgb) * 0.8)\n", "X_train_cv_xgb = X_train_xgb.iloc[:split_point_xgb]\n", "X_val_cv_xgb = X_train_xgb.iloc[split_point_xgb:]\n", "y_train_cv_xgb = y_train.iloc[:split_point_xgb]\n", "y_val_cv_xgb = y_train.iloc[split_point_xgb:]\n", "\n", "# Train threshold optimization model\n", "xgb_threshold_model = XGBClassifier(**xgb_grid.best_params_, scale_pos_weight=scale_pos_weight, \n", " random_state=42, eval_metric='logloss', verbosity=0)\n", "xgb_threshold_model.fit(X_train_cv_xgb, y_train_cv_xgb)\n", "\n", "print(f\"\\nπŸ”§ Finding optimal prediction threshold for XGBoost...\")\n", "print(f\"πŸ“Š CV Training set: {X_train_cv_xgb.shape}, CV Validation set: {X_val_cv_xgb.shape}\")\n", "\n", "# Get prediction probabilities\n", "y_val_proba_xgb = xgb_threshold_model.predict_proba(X_val_cv_xgb)[:, 1]\n", "\n", "# Find optimal threshold\n", "thresholds = np.arange(0.1, 0.9, 0.01)\n", "f1_scores_xgb = []\n", "\n", "for threshold in thresholds:\n", " y_pred_thresh = (y_val_proba_xgb >= threshold).astype(int)\n", " if len(np.unique(y_pred_thresh)) > 1:\n", " f1 = f1_score(y_val_cv_xgb, y_pred_thresh)\n", " else:\n", " f1 = 0.0\n", " f1_scores_xgb.append(f1)\n", "\n", "optimal_threshold_xgb = thresholds[np.argmax(f1_scores_xgb)]\n", "max_f1_xgb = max(f1_scores_xgb)\n", "\n", "print(f\"🎯 Optimal threshold: {optimal_threshold_xgb:.3f}\")\n", "print(f\"πŸ† Validation F1-score at optimal threshold: {max_f1_xgb:.4f}\")\n", "\n", "# Retrain on full training set\n", "print(\"\\nπŸ”„ Retraining XGBoost on full training set...\")\n", "xgb_final = XGBClassifier(**xgb_grid.best_params_, scale_pos_weight=scale_pos_weight, \n", " random_state=42, eval_metric='logloss', verbosity=0)\n", "xgb_final.fit(X_train_xgb, y_train)\n", "\n", "# Make predictions on test set\n", "print(\"\\nπŸ“Š Evaluating XGBoost on test set...\")\n", "y_test_proba_xgb = xgb_final.predict_proba(X_test_xgb)[:, 1]\n", "y_test_pred_xgb = (y_test_proba_xgb >= optimal_threshold_xgb).astype(int)\n", "\n", "# Calculate metrics\n", "xgb_test_f1 = f1_score(y_test, y_test_pred_xgb)\n", "xgb_test_auc = roc_auc_score(y_test, y_test_proba_xgb)\n", "xgb_test_accuracy = accuracy_score(y_test, y_test_pred_xgb)\n", "\n", "print(f\"🎯 Test F1-score: {xgb_test_f1:.4f}\")\n", "print(f\"πŸ“ˆ Test AUC: {xgb_test_auc:.4f}\")\n", "print(f\"🎯 Test Accuracy: {xgb_test_accuracy:.4f}\")\n", "\n", "# Detailed classification report\n", "print(f\"\\nπŸ“‹ Detailed Classification Report:\")\n", "print(classification_report(y_test, y_test_pred_xgb, target_names=['No Catch', 'Catch']))\n", "\n", "# Store results for comparison\n", "xgb_results = {\n", " 'model': xgb_final,\n", " 'threshold': optimal_threshold_xgb,\n", " 'test_f1': xgb_test_f1,\n", " 'test_auc': xgb_test_auc,\n", " 'test_accuracy': xgb_test_accuracy,\n", " 'y_pred': y_test_pred_xgb,\n", " 'y_proba': y_test_proba_xgb,\n", " 'best_params': xgb_grid.best_params_\n", "}\n", "\n", "print(\"\\nβœ… XGBoostClassifier training complete!\")" ] }, { "cell_type": "markdown", "id": "f949ea69", "metadata": {}, "source": [ "### πŸ“Š XGBoost Performance Analysis\n", "\n", "Time to analyze our gradient boosting champion's performance!" ] }, { "cell_type": "code", "execution_count": 22, "id": "eb4461d3", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "πŸ“‹ XGBoost Classification Report:\n", "============================================================\n", " precision recall f1-score support\n", "\n", " No Pest 1.00 0.50 0.67 40\n", "Pest Detected 0.31 1.00 0.47 9\n", "\n", " accuracy 0.59 49\n", " macro avg 0.66 0.75 0.57 49\n", " weighted avg 0.87 0.59 0.63 49\n", "\n" ] }, { "data": { "application/vnd.plotly.v1+json": { "config": { "plotlyServerURL": "https://plot.ly" }, "data": [ { "colorscale": [ [ 0, "rgb(255,245,235)" ], [ 0.125, "rgb(254,230,206)" ], [ 0.25, "rgb(253,208,162)" ], [ 0.375, "rgb(253,174,107)" ], [ 0.5, "rgb(253,141,60)" ], [ 0.625, "rgb(241,105,19)" ], [ 0.75, "rgb(217,72,1)" ], [ 0.875, "rgb(166,54,3)" ], [ 1, "rgb(127,39,4)" ] ], "reversescale": false, "showscale": true, "type": "heatmap", "x": [ "Predicted: No Pest", "Predicted: Pest" ], "y": [ "Actual: No Pest", "Actual: Pest" ], "z": [ [ 20, 20 ], [ 0, 9 ] ] } ], "layout": { "annotations": [ { "font": { "color": "#FFFFFF" }, "showarrow": false, "text": "20", "x": "Predicted: No Pest", "xref": "x", "y": "Actual: No Pest", "yref": "y" }, { "font": { "color": "#FFFFFF" }, "showarrow": false, "text": "20", "x": "Predicted: Pest", "xref": "x", "y": "Actual: No Pest", "yref": "y" }, { "font": { "color": "#000000" }, "showarrow": false, "text": "0", "x": "Predicted: No Pest", "xref": "x", "y": "Actual: Pest", "yref": "y" }, { "font": { "color": "#000000" }, "showarrow": false, "text": "9", "x": "Predicted: Pest", "xref": "x", "y": "Actual: Pest", "yref": "y" } ], "height": 400, "template": { "data": { "bar": [ { "error_x": { "color": "#2a3f5f" }, "error_y": { "color": "#2a3f5f" }, "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "bar" } ], "barpolar": [ { "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "barpolar" } ], "carpet": [ { "aaxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "baxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "type": "carpet" } ], "choropleth": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "choropleth" } ], "contour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "contour" } ], "contourcarpet": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "contourcarpet" } ], "heatmap": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "heatmap" } ], "heatmapgl": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "heatmapgl" } ], "histogram": [ { "marker": { "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "histogram" } ], "histogram2d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "histogram2d" } ], "histogram2dcontour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "histogram2dcontour" } ], "mesh3d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "mesh3d" } ], "parcoords": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "parcoords" } ], "pie": [ { "automargin": true, "type": "pie" } ], "scatter": [ { "fillpattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 }, "type": "scatter" } ], "scatter3d": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatter3d" } ], "scattercarpet": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattercarpet" } ], "scattergeo": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergeo" } ], "scattergl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergl" } ], "scattermapbox": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattermapbox" } ], "scatterpolar": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolar" } ], "scatterpolargl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolargl" } ], "scatterternary": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterternary" } ], "surface": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "surface" } ], "table": [ { "cells": { "fill": { "color": "#EBF0F8" }, "line": { "color": "white" } }, "header": { "fill": { "color": "#C8D4E3" }, "line": { "color": "white" } }, "type": "table" } ] }, "layout": { "annotationdefaults": { "arrowcolor": "#2a3f5f", "arrowhead": 0, "arrowwidth": 1 }, "autotypenumbers": "strict", "coloraxis": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "colorscale": { "diverging": [ [ 0, "#8e0152" ], [ 0.1, "#c51b7d" ], [ 0.2, "#de77ae" ], [ 0.3, "#f1b6da" ], [ 0.4, "#fde0ef" ], [ 0.5, "#f7f7f7" ], [ 0.6, "#e6f5d0" ], [ 0.7, "#b8e186" ], [ 0.8, "#7fbc41" ], [ 0.9, "#4d9221" ], [ 1, "#276419" ] ], "sequential": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "sequentialminus": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ] }, "colorway": [ "#636efa", "#EF553B", "#00cc96", "#ab63fa", "#FFA15A", "#19d3f3", "#FF6692", "#B6E880", "#FF97FF", "#FECB52" ], "font": { "color": "#2a3f5f" }, "geo": { "bgcolor": "white", "lakecolor": "white", "landcolor": "#E5ECF6", "showlakes": true, "showland": true, "subunitcolor": "white" }, "hoverlabel": { "align": "left" }, "hovermode": "closest", "mapbox": { "style": "light" }, "paper_bgcolor": "white", "plot_bgcolor": "#E5ECF6", "polar": { "angularaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "radialaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "scene": { "xaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "yaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "zaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" } }, "shapedefaults": { "line": { "color": "#2a3f5f" } }, "ternary": { "aaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "baxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "caxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "title": { "x": 0.05 }, "xaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 }, "yaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 } } }, "title": { "text": "πŸš€ XGBoost Confusion Matrix", "x": 0.5 }, "width": 500, "xaxis": { "dtick": 1, "gridcolor": "rgb(0, 0, 0)", "side": "top", "ticks": "" }, "yaxis": { "dtick": 1, "ticks": "", "ticksuffix": " " } } } }, "metadata": {}, "output_type": "display_data" }, { "data": { "application/vnd.plotly.v1+json": { "config": { "plotlyServerURL": "https://plot.ly" }, "data": [ { "line": { "color": "orange", "width": 3 }, "mode": "lines", "name": "XGBoost (AUC = 0.932)", "type": "scatter", "x": [ 0, 0, 0.025, 0.075, 0.075, 0.125, 0.125, 0.15, 0.425, 0.5, 0.625, 0.675, 0.825, 0.925, 1 ], "y": [ 0, 0.1111111111111111, 0.2222222222222222, 0.3333333333333333, 0.6666666666666666, 0.8888888888888888, 1, 1, 1, 1, 1, 1, 1, 1, 1 ] }, { "line": { "color": "red", "dash": "dash", "width": 2 }, "mode": "lines", "name": "Random Classifier", "type": "scatter", "x": [ 0, 1 ], "y": [ 0, 1 ] } ], "layout": { "height": 500, "template": { "data": { "bar": [ { "error_x": { "color": "#2a3f5f" }, "error_y": { "color": "#2a3f5f" }, "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "bar" } ], "barpolar": [ { "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "barpolar" } ], "carpet": [ { "aaxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "baxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "type": "carpet" } ], "choropleth": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "choropleth" } ], "contour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "contour" } ], "contourcarpet": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "contourcarpet" } ], "heatmap": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "heatmap" } ], "heatmapgl": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "heatmapgl" } ], "histogram": [ { "marker": { "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "histogram" } ], "histogram2d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "histogram2d" } ], "histogram2dcontour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "histogram2dcontour" } ], "mesh3d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "mesh3d" } ], "parcoords": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "parcoords" } ], "pie": [ { "automargin": true, "type": "pie" } ], "scatter": [ { "fillpattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 }, "type": "scatter" } ], "scatter3d": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatter3d" } ], "scattercarpet": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattercarpet" } ], "scattergeo": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergeo" } ], "scattergl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergl" } ], "scattermapbox": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattermapbox" } ], "scatterpolar": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolar" } ], "scatterpolargl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolargl" } ], "scatterternary": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterternary" } ], "surface": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "surface" } ], "table": [ { "cells": { "fill": { "color": "#EBF0F8" }, "line": { "color": "white" } }, "header": { "fill": { "color": "#C8D4E3" }, "line": { "color": "white" } }, "type": "table" } ] }, "layout": { "annotationdefaults": { "arrowcolor": "#2a3f5f", "arrowhead": 0, "arrowwidth": 1 }, "autotypenumbers": "strict", "coloraxis": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "colorscale": { "diverging": [ [ 0, "#8e0152" ], [ 0.1, "#c51b7d" ], [ 0.2, "#de77ae" ], [ 0.3, "#f1b6da" ], [ 0.4, "#fde0ef" ], [ 0.5, "#f7f7f7" ], [ 0.6, "#e6f5d0" ], [ 0.7, "#b8e186" ], [ 0.8, "#7fbc41" ], [ 0.9, "#4d9221" ], [ 1, "#276419" ] ], "sequential": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "sequentialminus": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ] }, "colorway": [ "#636efa", "#EF553B", "#00cc96", "#ab63fa", "#FFA15A", "#19d3f3", "#FF6692", "#B6E880", "#FF97FF", "#FECB52" ], "font": { "color": "#2a3f5f" }, "geo": { "bgcolor": "white", "lakecolor": "white", "landcolor": "#E5ECF6", "showlakes": true, "showland": true, "subunitcolor": "white" }, "hoverlabel": { "align": "left" }, "hovermode": "closest", "mapbox": { "style": "light" }, "paper_bgcolor": "white", "plot_bgcolor": "#E5ECF6", "polar": { "angularaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "radialaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "scene": { "xaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "yaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "zaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" } }, "shapedefaults": { "line": { "color": "#2a3f5f" } }, "ternary": { "aaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "baxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "caxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "title": { "x": 0.05 }, "xaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 }, "yaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 } } }, "title": { "text": "πŸš€ XGBoost ROC Curve", "x": 0.5 }, "width": 600, "xaxis": { "title": { "text": "False Positive Rate" } }, "yaxis": { "title": { "text": "True Positive Rate" } } } } }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "\n", "πŸ† XGBoost Tournament Summary:\n", " β€’ F1-Score: 0.4737\n", " β€’ AUC: 0.9319\n", " β€’ Optimal Threshold: 0.100\n", " β€’ Test Accuracy: 0.5918\n" ] } ], "source": [ "# Generate comprehensive classification report\n", "print(\"πŸ“‹ XGBoost Classification Report:\")\n", "print(\"=\" * 60)\n", "print(classification_report(y_test, y_test_pred_xgb, target_names=['No Pest', 'Pest Detected']))\n", "\n", "# Create interactive confusion matrix\n", "cm_xgb = confusion_matrix(y_test, y_test_pred_xgb)\n", "\n", "fig_cm_xgb = ff.create_annotated_heatmap(\n", " z=cm_xgb,\n", " x=['Predicted: No Pest', 'Predicted: Pest'],\n", " y=['Actual: No Pest', 'Actual: Pest'],\n", " annotation_text=cm_xgb,\n", " colorscale='Oranges',\n", " showscale=True\n", ")\n", "\n", "fig_cm_xgb.update_layout(\n", " title='πŸš€ XGBoost Confusion Matrix',\n", " title_x=0.5,\n", " width=500,\n", " height=400\n", ")\n", "\n", "fig_cm_xgb.show()\n", "\n", "# Create interactive ROC curve\n", "fpr_xgb, tpr_xgb, _ = roc_curve(y_test, y_test_proba_xgb)\n", "\n", "fig_roc_xgb = go.Figure()\n", "\n", "fig_roc_xgb.add_trace(go.Scatter(\n", " x=fpr_xgb,\n", " y=tpr_xgb,\n", " mode='lines',\n", " name=f'XGBoost (AUC = {xgb_test_auc:.3f})',\n", " line=dict(color='orange', width=3)\n", "))\n", "\n", "# Add diagonal reference line\n", "fig_roc_xgb.add_trace(go.Scatter(\n", " x=[0, 1],\n", " y=[0, 1],\n", " mode='lines',\n", " name='Random Classifier',\n", " line=dict(color='red', width=2, dash='dash')\n", "))\n", "\n", "fig_roc_xgb.update_layout(\n", " title='πŸš€ XGBoost ROC Curve',\n", " title_x=0.5,\n", " xaxis_title='False Positive Rate',\n", " yaxis_title='True Positive Rate',\n", " width=600,\n", " height=500\n", ")\n", "\n", "fig_roc_xgb.show()\n", "\n", "print(f\"\\nπŸ† XGBoost Tournament Summary:\")\n", "print(f\" β€’ F1-Score: {xgb_test_f1:.4f}\")\n", "print(f\" β€’ AUC: {xgb_test_auc:.4f}\")\n", "print(f\" β€’ Optimal Threshold: {optimal_threshold_xgb:.3f}\")\n", "print(f\" β€’ Test Accuracy: {xgb_test_accuracy:.4f}\")" ] }, { "cell_type": "markdown", "id": "55ee986e", "metadata": {}, "source": [ "## ⚑ Contestant 3: LightGBMClassifier\n", "\n", "Our final traditional ML competitor enters the arena! LightGBM, Microsoft's lightning-fast gradient boosting framework, promises efficiency without sacrificing performance. Known for its speed and memory efficiency." ] }, { "cell_type": "code", "execution_count": 23, "id": "aa816543", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "⚑ Training LightGBMClassifier...\n", "==================================================\n", "πŸ“Š Training set shape: (196, 13)\n", "πŸ“Š Test set shape: (49, 13)\n", "πŸ“Š CV Training set: (156, 13), CV Validation set: (40, 13)\n", "πŸ”§ Hyperparameter combinations to test: 32\n", "πŸ” Performing hyperparameter tuning with TimeSeriesSplit...\n", "Fitting 3 folds for each of 32 candidates, totalling 96 fits\n", "βœ… Best parameters found: {'learning_rate': 0.01, 'max_depth': 5, 'n_estimators': 200, 'num_leaves': 20, 'subsample': 0.8}\n", "🎯 Best CV F1-score: 0.2165\n", "\n", "πŸ”§ Finding optimal prediction threshold using temporal validation...\n", "πŸ“Š CV Training set: (156, 13), CV Validation set: (40, 13)\n", "🎯 Optimal threshold: 0.100\n", "πŸ† Validation F1-score at optimal threshold: 0.0000\n", "\n", "πŸ”„ Retraining on full training set...\n", "\n", "πŸ“Š Evaluating on test set...\n", "🎯 Test F1-score: 0.7500\n", "πŸ“ˆ Test AUC: 0.9375\n", "🎯 Test Accuracy: 0.8776\n", "\n", "πŸ“‹ Detailed Classification Report:\n", " precision recall f1-score support\n", "\n", " No Catch 1.00 0.85 0.92 40\n", " Catch 0.60 1.00 0.75 9\n", "\n", " accuracy 0.88 49\n", " macro avg 0.80 0.93 0.83 49\n", "weighted avg 0.93 0.88 0.89 49\n", "\n", "\n", "βœ… LightGBMClassifier training complete!\n" ] } ], "source": [ "print(\"⚑ Training LightGBMClassifier...\")\n", "print(\"=\" * 50)\n", "\n", "# Create numeric datasets for LightGBM (ensure same features as other models)\n", "# Use the same approach as RandomForest and XGBoost to ensure consistency\n", "X_train_numeric = X_train_encoded.select_dtypes(include=[np.number])\n", "X_test_numeric = X_test_encoded.select_dtypes(include=[np.number])\n", "\n", "# Ensure both datasets have the same columns\n", "train_cols = set(X_train_numeric.columns)\n", "test_cols = set(X_test_numeric.columns)\n", "\n", "# Add missing columns to test set with zeros\n", "missing_in_test = train_cols - test_cols\n", "for col in missing_in_test:\n", " X_test_numeric[col] = 0\n", "\n", "# Add missing columns to train set with zeros (if any)\n", "missing_in_train = test_cols - train_cols\n", "for col in missing_in_train:\n", " X_train_numeric[col] = 0\n", "\n", "# Ensure columns are in the same order\n", "X_test_numeric = X_test_numeric[X_train_numeric.columns]\n", "\n", "X_train_cv = X_train_numeric.iloc[:split_point]\n", "X_val_cv = X_train_numeric.iloc[split_point:]\n", "y_train_cv = y_train.iloc[:split_point]\n", "y_val_cv = y_train.iloc[split_point:]\n", "\n", "print(f\"πŸ“Š Training set shape: {X_train_numeric.shape}\")\n", "print(f\"πŸ“Š Test set shape: {X_test_numeric.shape}\")\n", "print(f\"πŸ“Š CV Training set: {X_train_cv.shape}, CV Validation set: {X_val_cv.shape}\")\n", "\n", "# Use smaller hyperparameter grid due to severe imbalance and computational efficiency\n", "lgb_param_grid = {\n", " 'n_estimators': [100, 200],\n", " 'max_depth': [5, 10],\n", " 'learning_rate': [0.01, 0.1],\n", " 'num_leaves': [20, 31],\n", " 'subsample': [0.8, 1.0]\n", "}\n", "\n", "print(f\"πŸ”§ Hyperparameter combinations to test: {len(lgb_param_grid['n_estimators']) * len(lgb_param_grid['max_depth']) * len(lgb_param_grid['learning_rate']) * len(lgb_param_grid['num_leaves']) * len(lgb_param_grid['subsample'])}\")\n", "\n", "# Initialize LightGBM with class balancing\n", "lgb_base = LGBMClassifier(\n", " class_weight='balanced',\n", " random_state=42,\n", " verbosity=-1,\n", " force_col_wise=True\n", ")\n", "\n", "# Use TimeSeriesSplit for cross-validation to respect temporal order\n", "print(\"πŸ” Performing hyperparameter tuning with TimeSeriesSplit...\")\n", "lgb_grid = GridSearchCV(\n", " lgb_base,\n", " lgb_param_grid,\n", " cv=tscv, # Use TimeSeriesSplit instead of regular CV\n", " scoring='f1',\n", " n_jobs=-1,\n", " verbose=1\n", ")\n", "\n", "lgb_grid.fit(X_train_numeric, y_train)\n", "\n", "print(f\"βœ… Best parameters found: {lgb_grid.best_params_}\")\n", "print(f\"🎯 Best CV F1-score: {lgb_grid.best_score_:.4f}\")\n", "\n", "# Get the best model\n", "lgb_best = lgb_grid.best_estimator_\n", "\n", "# Use the same temporal validation approach as RandomForest and XGBoost\n", "print(f\"\\nπŸ”§ Finding optimal prediction threshold using temporal validation...\")\n", "print(f\"πŸ“Š CV Training set: {X_train_cv.shape}, CV Validation set: {X_val_cv.shape}\")\n", "\n", "# Train on the CV training set\n", "lgb_best.fit(X_train_cv, y_train_cv)\n", "\n", "# Get prediction probabilities for threshold tuning on validation set\n", "y_val_proba_lgb = lgb_best.predict_proba(X_val_cv)[:, 1]\n", "\n", "# Find optimal threshold using validation set\n", "thresholds = np.arange(0.1, 0.9, 0.01)\n", "f1_scores_lgb = []\n", "\n", "for threshold in thresholds:\n", " y_pred_thresh = (y_val_proba_lgb >= threshold).astype(int)\n", " if len(np.unique(y_pred_thresh)) > 1: # Avoid division by zero\n", " f1 = f1_score(y_val_cv, y_pred_thresh)\n", " else:\n", " f1 = 0.0\n", " f1_scores_lgb.append(f1)\n", "\n", "optimal_threshold_lgb = thresholds[np.argmax(f1_scores_lgb)]\n", "max_f1_lgb = max(f1_scores_lgb)\n", "\n", "print(f\"🎯 Optimal threshold: {optimal_threshold_lgb:.3f}\")\n", "print(f\"πŸ† Validation F1-score at optimal threshold: {max_f1_lgb:.4f}\")\n", "\n", "# Retrain on full training set with best parameters\n", "print(\"\\nπŸ”„ Retraining on full training set...\")\n", "lgb_final = LGBMClassifier(**lgb_grid.best_params_, class_weight='balanced', random_state=42, verbosity=-1, force_col_wise=True)\n", "lgb_final.fit(X_train_numeric, y_train)\n", "\n", "# Make predictions on test set with optimal threshold\n", "print(\"\\nπŸ“Š Evaluating on test set...\")\n", "y_test_proba_lgb = lgb_final.predict_proba(X_test_numeric)[:, 1]\n", "y_test_pred_lgb = (y_test_proba_lgb >= optimal_threshold_lgb).astype(int)\n", "\n", "# Calculate metrics\n", "lgb_test_f1 = f1_score(y_test, y_test_pred_lgb)\n", "lgb_test_auc = roc_auc_score(y_test, y_test_proba_lgb)\n", "lgb_test_accuracy = accuracy_score(y_test, y_test_pred_lgb)\n", "\n", "print(f\"🎯 Test F1-score: {lgb_test_f1:.4f}\")\n", "print(f\"πŸ“ˆ Test AUC: {lgb_test_auc:.4f}\")\n", "print(f\"🎯 Test Accuracy: {lgb_test_accuracy:.4f}\")\n", "\n", "# Detailed classification report\n", "print(f\"\\nπŸ“‹ Detailed Classification Report:\")\n", "print(classification_report(y_test, y_test_pred_lgb, target_names=['No Catch', 'Catch']))\n", "\n", "# Store results for later comparison\n", "lgb_results = {\n", " 'model': lgb_final,\n", " 'threshold': optimal_threshold_lgb,\n", " 'test_f1': lgb_test_f1,\n", " 'test_auc': lgb_test_auc,\n", " 'test_accuracy': lgb_test_accuracy,\n", " 'y_pred': y_test_pred_lgb,\n", " 'y_proba': y_test_proba_lgb\n", "}\n", "\n", "print(\"\\nβœ… LightGBMClassifier training complete!\")" ] }, { "cell_type": "markdown", "id": "b6f75a0b", "metadata": {}, "source": [ "### πŸ“Š LightGBM Performance Analysis\n", "\n", "Let's examine our speed demon's performance metrics!" ] }, { "cell_type": "code", "execution_count": 24, "id": "50ef29e4", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "⚑ Training LightGBMClassifier...\n", "==================================================\n", "πŸ“Š Using 17 features for LightGBM modeling\n", "βœ… Training set: (196, 17)\n", "βœ… Test set: (49, 17)\n", "πŸ”§ Hyperparameter combinations to test: 2187\n", "πŸ” Performing hyperparameter tuning with TimeSeriesSplit...\n", "Fitting 3 folds for each of 2187 candidates, totalling 6561 fits\n", "βœ… Best parameters found: {'colsample_bytree': 0.8, 'learning_rate': 0.01, 'max_depth': 5, 'min_child_samples': 20, 'n_estimators': 300, 'num_leaves': 15, 'subsample': 0.7}\n", "🎯 Best CV F1-score: 0.2343\n", "\n", "πŸ”§ Finding optimal prediction threshold for LightGBM...\n", "πŸ“Š CV Training set: (156, 17), CV Validation set: (40, 17)\n", "🎯 Optimal threshold: 0.100\n", "πŸ† Validation F1-score at optimal threshold: 0.0000\n", "\n", "πŸ”„ Retraining LightGBM on full training set...\n", "\n", "πŸ“Š Evaluating LightGBM on test set...\n", "🎯 Test F1-score: 0.5625\n", "πŸ“ˆ Test AUC: 0.9250\n", "🎯 Test Accuracy: 0.7143\n", "\n", "πŸ“‹ Detailed Classification Report:\n", " precision recall f1-score support\n", "\n", " No Catch 1.00 0.65 0.79 40\n", " Catch 0.39 1.00 0.56 9\n", "\n", " accuracy 0.71 49\n", " macro avg 0.70 0.82 0.68 49\n", "weighted avg 0.89 0.71 0.75 49\n", "\n", "\n", "βœ… LightGBMClassifier training complete!\n", "\n", "πŸ“‹ LightGBM Classification Report:\n", "============================================================\n", " precision recall f1-score support\n", "\n", " No Catch 1.00 0.65 0.79 40\n", " Catch 0.39 1.00 0.56 9\n", "\n", " accuracy 0.71 49\n", " macro avg 0.70 0.82 0.68 49\n", "weighted avg 0.89 0.71 0.75 49\n", "\n" ] }, { "data": { "application/vnd.plotly.v1+json": { "config": { "plotlyServerURL": "https://plot.ly" }, "data": [ { "colorscale": [ [ 0, "rgb(252,251,253)" ], [ 0.125, "rgb(239,237,245)" ], [ 0.25, "rgb(218,218,235)" ], [ 0.375, "rgb(188,189,220)" ], [ 0.5, "rgb(158,154,200)" ], [ 0.625, "rgb(128,125,186)" ], [ 0.75, "rgb(106,81,163)" ], [ 0.875, "rgb(84,39,143)" ], [ 1, "rgb(63,0,125)" ] ], "reversescale": false, "showscale": true, "type": "heatmap", "x": [ "Predicted: No Catch", "Predicted: Catch" ], "y": [ "Actual: No Catch", "Actual: Catch" ], "z": [ [ 26, 14 ], [ 0, 9 ] ] } ], "layout": { "annotations": [ { "font": { "color": "#FFFFFF" }, "showarrow": false, "text": "26", "x": "Predicted: No Catch", "xref": "x", "y": "Actual: No Catch", "yref": "y" }, { "font": { "color": "#FFFFFF" }, "showarrow": false, "text": "14", "x": "Predicted: Catch", "xref": "x", "y": "Actual: No Catch", "yref": "y" }, { "font": { "color": "#000000" }, "showarrow": false, "text": "0", "x": "Predicted: No Catch", "xref": "x", "y": "Actual: Catch", "yref": "y" }, { "font": { "color": "#000000" }, "showarrow": false, "text": "9", "x": "Predicted: Catch", "xref": "x", "y": "Actual: Catch", "yref": "y" } ], "height": 400, "template": { "data": { "bar": [ { "error_x": { "color": "#2a3f5f" }, "error_y": { "color": "#2a3f5f" }, "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "bar" } ], "barpolar": [ { "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "barpolar" } ], "carpet": [ { "aaxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "baxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "type": "carpet" } ], "choropleth": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "choropleth" } ], "contour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "contour" } ], "contourcarpet": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "contourcarpet" } ], "heatmap": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "heatmap" } ], "heatmapgl": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "heatmapgl" } ], "histogram": [ { "marker": { "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "histogram" } ], "histogram2d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "histogram2d" } ], "histogram2dcontour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "histogram2dcontour" } ], "mesh3d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "mesh3d" } ], "parcoords": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "parcoords" } ], "pie": [ { "automargin": true, "type": "pie" } ], "scatter": [ { "fillpattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 }, "type": "scatter" } ], "scatter3d": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatter3d" } ], "scattercarpet": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattercarpet" } ], "scattergeo": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergeo" } ], "scattergl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergl" } ], "scattermapbox": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattermapbox" } ], "scatterpolar": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolar" } ], "scatterpolargl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolargl" } ], "scatterternary": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterternary" } ], "surface": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "surface" } ], "table": [ { "cells": { "fill": { "color": "#EBF0F8" }, "line": { "color": "white" } }, "header": { "fill": { "color": "#C8D4E3" }, "line": { "color": "white" } }, "type": "table" } ] }, "layout": { "annotationdefaults": { "arrowcolor": "#2a3f5f", "arrowhead": 0, "arrowwidth": 1 }, "autotypenumbers": "strict", "coloraxis": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "colorscale": { "diverging": [ [ 0, "#8e0152" ], [ 0.1, "#c51b7d" ], [ 0.2, "#de77ae" ], [ 0.3, "#f1b6da" ], [ 0.4, "#fde0ef" ], [ 0.5, "#f7f7f7" ], [ 0.6, "#e6f5d0" ], [ 0.7, "#b8e186" ], [ 0.8, "#7fbc41" ], [ 0.9, "#4d9221" ], [ 1, "#276419" ] ], "sequential": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "sequentialminus": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ] }, "colorway": [ "#636efa", "#EF553B", "#00cc96", "#ab63fa", "#FFA15A", "#19d3f3", "#FF6692", "#B6E880", "#FF97FF", "#FECB52" ], "font": { "color": "#2a3f5f" }, "geo": { "bgcolor": "white", "lakecolor": "white", "landcolor": "#E5ECF6", "showlakes": true, "showland": true, "subunitcolor": "white" }, "hoverlabel": { "align": "left" }, "hovermode": "closest", "mapbox": { "style": "light" }, "paper_bgcolor": "white", "plot_bgcolor": "#E5ECF6", "polar": { "angularaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "radialaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "scene": { "xaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "yaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "zaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" } }, "shapedefaults": { "line": { "color": "#2a3f5f" } }, "ternary": { "aaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "baxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "caxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "title": { "x": 0.05 }, "xaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 }, "yaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 } } }, "title": { "text": "⚑ LightGBM Confusion Matrix", "x": 0.5 }, "width": 500, "xaxis": { "dtick": 1, "gridcolor": "rgb(0, 0, 0)", "side": "top", "ticks": "" }, "yaxis": { "dtick": 1, "ticks": "", "ticksuffix": " " } } } }, "metadata": {}, "output_type": "display_data" }, { "data": { "application/vnd.plotly.v1+json": { "config": { "plotlyServerURL": "https://plot.ly" }, "data": [ { "line": { "color": "purple", "width": 3 }, "mode": "lines", "name": "LightGBM (AUC = 0.925)", "type": "scatter", "x": [ 0, 0, 0.05, 0.05, 0.075, 0.075, 0.1, 0.1, 0.75, 0.825, 1 ], "y": [ 0, 0.1111111111111111, 0.1111111111111111, 0.3333333333333333, 0.3333333333333333, 0.4444444444444444, 0.4444444444444444, 1, 1, 1, 1 ] }, { "line": { "color": "red", "dash": "dash", "width": 2 }, "mode": "lines", "name": "Random Classifier", "type": "scatter", "x": [ 0, 1 ], "y": [ 0, 1 ] } ], "layout": { "height": 500, "template": { "data": { "bar": [ { "error_x": { "color": "#2a3f5f" }, "error_y": { "color": "#2a3f5f" }, "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "bar" } ], "barpolar": [ { "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "barpolar" } ], "carpet": [ { "aaxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "baxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "type": "carpet" } ], "choropleth": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "choropleth" } ], "contour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "contour" } ], "contourcarpet": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "contourcarpet" } ], "heatmap": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "heatmap" } ], "heatmapgl": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "heatmapgl" } ], "histogram": [ { "marker": { "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "histogram" } ], "histogram2d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "histogram2d" } ], "histogram2dcontour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "histogram2dcontour" } ], "mesh3d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "mesh3d" } ], "parcoords": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "parcoords" } ], "pie": [ { "automargin": true, "type": "pie" } ], "scatter": [ { "fillpattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 }, "type": "scatter" } ], "scatter3d": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatter3d" } ], "scattercarpet": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattercarpet" } ], "scattergeo": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergeo" } ], "scattergl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergl" } ], "scattermapbox": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattermapbox" } ], "scatterpolar": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolar" } ], "scatterpolargl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolargl" } ], "scatterternary": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterternary" } ], "surface": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "surface" } ], "table": [ { "cells": { "fill": { "color": "#EBF0F8" }, "line": { "color": "white" } }, "header": { "fill": { "color": "#C8D4E3" }, "line": { "color": "white" } }, "type": "table" } ] }, "layout": { "annotationdefaults": { "arrowcolor": "#2a3f5f", "arrowhead": 0, "arrowwidth": 1 }, "autotypenumbers": "strict", "coloraxis": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "colorscale": { "diverging": [ [ 0, "#8e0152" ], [ 0.1, "#c51b7d" ], [ 0.2, "#de77ae" ], [ 0.3, "#f1b6da" ], [ 0.4, "#fde0ef" ], [ 0.5, "#f7f7f7" ], [ 0.6, "#e6f5d0" ], [ 0.7, "#b8e186" ], [ 0.8, "#7fbc41" ], [ 0.9, "#4d9221" ], [ 1, "#276419" ] ], "sequential": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "sequentialminus": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ] }, "colorway": [ "#636efa", "#EF553B", "#00cc96", "#ab63fa", "#FFA15A", "#19d3f3", "#FF6692", "#B6E880", "#FF97FF", "#FECB52" ], "font": { "color": "#2a3f5f" }, "geo": { "bgcolor": "white", "lakecolor": "white", "landcolor": "#E5ECF6", "showlakes": true, "showland": true, "subunitcolor": "white" }, "hoverlabel": { "align": "left" }, "hovermode": "closest", "mapbox": { "style": "light" }, "paper_bgcolor": "white", "plot_bgcolor": "#E5ECF6", "polar": { "angularaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "radialaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "scene": { "xaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "yaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "zaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" } }, "shapedefaults": { "line": { "color": "#2a3f5f" } }, "ternary": { "aaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "baxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "caxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "title": { "x": 0.05 }, "xaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 }, "yaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 } } }, "title": { "text": "⚑ LightGBM ROC Curve", "x": 0.5 }, "width": 600, "xaxis": { "title": { "text": "False Positive Rate" } }, "yaxis": { "title": { "text": "True Positive Rate" } } } } }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "\n", "🎯 LightGBM - Top 10 Most Important Features:\n", " 1. Day: 466.0000\n", " 2. Recent_Activity: 237.0000\n", " 3. Location_Code: 208.0000\n", " 4. Temp_Range: 192.0000\n", " 5. Average Temperature: 172.0000\n", " 6. Temp_Avg_3d: 165.0000\n", " 7. Average Humidity: 159.0000\n", " 8. Days_Since_Cleaning: 133.0000\n", " 9. Month: 77.0000\n", " 10. Season_Mid_Summer: 65.0000\n" ] }, { "data": { "application/vnd.plotly.v1+json": { "config": { "plotlyServerURL": "https://plot.ly" }, "data": [ { "alignmentgroup": "True", "hovertemplate": "Feature Importance=%{x}
Features=%{y}", "legendgroup": "", "marker": { "color": "purple", "pattern": { "shape": "" } }, "name": "", "offsetgroup": "", "orientation": "h", "showlegend": false, "textposition": "auto", "type": "bar", "x": [ 466, 237, 208, 192, 172, 165, 159, 133, 77, 65 ], "xaxis": "x", "y": [ "Day", "Recent_Activity", "Location_Code", "Temp_Range", "Average Temperature", "Temp_Avg_3d", "Average Humidity", "Days_Since_Cleaning", "Month", "Season_Mid_Summer" ], "yaxis": "y" } ], "layout": { "barmode": "relative", "height": 500, "legend": { "tracegroupgap": 0 }, "template": { "data": { "bar": [ { "error_x": { "color": "#2a3f5f" }, "error_y": { "color": "#2a3f5f" }, "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "bar" } ], "barpolar": [ { "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "barpolar" } ], "carpet": [ { "aaxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "baxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "type": "carpet" } ], "choropleth": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "choropleth" } ], "contour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "contour" } ], "contourcarpet": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "contourcarpet" } ], "heatmap": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "heatmap" } ], "heatmapgl": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "heatmapgl" } ], "histogram": [ { "marker": { "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "histogram" } ], "histogram2d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "histogram2d" } ], "histogram2dcontour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "histogram2dcontour" } ], "mesh3d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "mesh3d" } ], "parcoords": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "parcoords" } ], "pie": [ { "automargin": true, "type": "pie" } ], "scatter": [ { "fillpattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 }, "type": "scatter" } ], "scatter3d": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatter3d" } ], "scattercarpet": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattercarpet" } ], "scattergeo": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergeo" } ], "scattergl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergl" } ], "scattermapbox": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattermapbox" } ], "scatterpolar": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolar" } ], "scatterpolargl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolargl" } ], "scatterternary": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterternary" } ], "surface": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "surface" } ], "table": [ { "cells": { "fill": { "color": "#EBF0F8" }, "line": { "color": "white" } }, "header": { "fill": { "color": "#C8D4E3" }, "line": { "color": "white" } }, "type": "table" } ] }, "layout": { "annotationdefaults": { "arrowcolor": "#2a3f5f", "arrowhead": 0, "arrowwidth": 1 }, "autotypenumbers": "strict", "coloraxis": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "colorscale": { "diverging": [ [ 0, "#8e0152" ], [ 0.1, "#c51b7d" ], [ 0.2, "#de77ae" ], [ 0.3, "#f1b6da" ], [ 0.4, "#fde0ef" ], [ 0.5, "#f7f7f7" ], [ 0.6, "#e6f5d0" ], [ 0.7, "#b8e186" ], [ 0.8, "#7fbc41" ], [ 0.9, "#4d9221" ], [ 1, "#276419" ] ], "sequential": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "sequentialminus": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ] }, "colorway": [ "#636efa", "#EF553B", "#00cc96", "#ab63fa", "#FFA15A", "#19d3f3", "#FF6692", "#B6E880", "#FF97FF", "#FECB52" ], "font": { "color": "#2a3f5f" }, "geo": { "bgcolor": "white", "lakecolor": "white", "landcolor": "#E5ECF6", "showlakes": true, "showland": true, "subunitcolor": "white" }, "hoverlabel": { "align": "left" }, "hovermode": "closest", "mapbox": { "style": "light" }, "paper_bgcolor": "white", "plot_bgcolor": "#E5ECF6", "polar": { "angularaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "radialaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "scene": { "xaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "yaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "zaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" } }, "shapedefaults": { "line": { "color": "#2a3f5f" } }, "ternary": { "aaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "baxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "caxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "title": { "x": 0.05 }, "xaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 }, "yaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 } } }, "title": { "text": "⚑ LightGBM - Top 10 Feature Importances" }, "xaxis": { "anchor": "y", "domain": [ 0, 1 ], "title": { "text": "Feature Importance" } }, "yaxis": { "anchor": "x", "categoryorder": "total ascending", "domain": [ 0, 1 ], "title": { "text": "Features" } } } } }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "\n", "πŸ† LightGBM Tournament Summary:\n", " β€’ F1-Score: 0.5625\n", " β€’ AUC: 0.9250\n", " β€’ Optimal Threshold: 0.100\n", " β€’ Test Accuracy: 0.7143\n", " β€’ Best Parameters: {'colsample_bytree': 0.8, 'learning_rate': 0.01, 'max_depth': 5, 'min_child_samples': 20, 'n_estimators': 300, 'num_leaves': 15, 'subsample': 0.7}\n" ] } ], "source": [ "print(\"⚑ Training LightGBMClassifier...\")\n", "print(\"=\" * 50)\n", "\n", "# Use all available features for LightGBM\n", "X_train_lgb = X_train.copy()\n", "X_test_lgb = X_test.copy()\n", "\n", "print(f\"πŸ“Š Using {len(X_train_lgb.columns)} features for LightGBM modeling\")\n", "print(f\"βœ… Training set: {X_train_lgb.shape}\")\n", "print(f\"βœ… Test set: {X_test_lgb.shape}\")\n", "\n", "# Define LightGBM-specific hyperparameter grid\n", "lgb_param_grid = {\n", " 'n_estimators': [100, 200, 300],\n", " 'max_depth': [5, 10, 15], # Different range from others\n", " 'learning_rate': [0.01, 0.05, 0.1], # Different from XGBoost\n", " 'num_leaves': [15, 31, 50], # LightGBM-specific parameter\n", " 'subsample': [0.7, 0.8, 1.0], # Different range\n", " 'colsample_bytree': [0.7, 0.8, 1.0], # Different range\n", " 'min_child_samples': [10, 20, 30] # LightGBM-specific\n", "}\n", "\n", "total_combinations_lgb = (len(lgb_param_grid['n_estimators']) * len(lgb_param_grid['max_depth']) * \n", " len(lgb_param_grid['learning_rate']) * len(lgb_param_grid['num_leaves']) * \n", " len(lgb_param_grid['subsample']) * len(lgb_param_grid['colsample_bytree']) * \n", " len(lgb_param_grid['min_child_samples']))\n", "print(f\"πŸ”§ Hyperparameter combinations to test: {total_combinations_lgb}\")\n", "\n", "# Initialize LightGBM with class balancing\n", "lgb_base = LGBMClassifier(\n", " class_weight='balanced',\n", " random_state=42,\n", " verbosity=-1,\n", " force_col_wise=True\n", ")\n", "\n", "# Use TimeSeriesSplit for cross-validation\n", "print(\"πŸ” Performing hyperparameter tuning with TimeSeriesSplit...\")\n", "lgb_grid = GridSearchCV(\n", " lgb_base,\n", " lgb_param_grid,\n", " cv=tscv, # Use TimeSeriesSplit\n", " scoring='f1',\n", " n_jobs=-1,\n", " verbose=1\n", ")\n", "\n", "lgb_grid.fit(X_train_lgb, y_train)\n", "\n", "print(f\"βœ… Best parameters found: {lgb_grid.best_params_}\")\n", "print(f\"🎯 Best CV F1-score: {lgb_grid.best_score_:.4f}\")\n", "\n", "# Get the best model\n", "lgb_best = lgb_grid.best_estimator_\n", "\n", "# Use separate validation set for threshold tuning\n", "split_point_lgb = int(len(X_train_lgb) * 0.8)\n", "X_train_cv_lgb = X_train_lgb.iloc[:split_point_lgb]\n", "X_val_cv_lgb = X_train_lgb.iloc[split_point_lgb:]\n", "y_train_cv_lgb = y_train.iloc[:split_point_lgb]\n", "y_val_cv_lgb = y_train.iloc[split_point_lgb:]\n", "\n", "# Train threshold optimization model\n", "lgb_threshold_model = LGBMClassifier(**lgb_grid.best_params_, class_weight='balanced', \n", " random_state=42, verbosity=-1, force_col_wise=True)\n", "lgb_threshold_model.fit(X_train_cv_lgb, y_train_cv_lgb)\n", "\n", "print(f\"\\nπŸ”§ Finding optimal prediction threshold for LightGBM...\")\n", "print(f\"πŸ“Š CV Training set: {X_train_cv_lgb.shape}, CV Validation set: {X_val_cv_lgb.shape}\")\n", "\n", "# Get prediction probabilities\n", "y_val_proba_lgb = lgb_threshold_model.predict_proba(X_val_cv_lgb)[:, 1]\n", "\n", "# Find optimal threshold\n", "thresholds = np.arange(0.1, 0.9, 0.01)\n", "f1_scores_lgb = []\n", "\n", "for threshold in thresholds:\n", " y_pred_thresh = (y_val_proba_lgb >= threshold).astype(int)\n", " if len(np.unique(y_pred_thresh)) > 1:\n", " f1 = f1_score(y_val_cv_lgb, y_pred_thresh)\n", " else:\n", " f1 = 0.0\n", " f1_scores_lgb.append(f1)\n", "\n", "optimal_threshold_lgb = thresholds[np.argmax(f1_scores_lgb)]\n", "max_f1_lgb = max(f1_scores_lgb)\n", "\n", "print(f\"🎯 Optimal threshold: {optimal_threshold_lgb:.3f}\")\n", "print(f\"πŸ† Validation F1-score at optimal threshold: {max_f1_lgb:.4f}\")\n", "\n", "# Retrain on full training set\n", "print(\"\\nπŸ”„ Retraining LightGBM on full training set...\")\n", "lgb_final = LGBMClassifier(**lgb_grid.best_params_, class_weight='balanced', \n", " random_state=42, verbosity=-1, force_col_wise=True)\n", "lgb_final.fit(X_train_lgb, y_train)\n", "\n", "# Make predictions on test set\n", "print(\"\\nπŸ“Š Evaluating LightGBM on test set...\")\n", "y_test_proba_lgb = lgb_final.predict_proba(X_test_lgb)[:, 1]\n", "y_test_pred_lgb = (y_test_proba_lgb >= optimal_threshold_lgb).astype(int)\n", "\n", "# Calculate metrics\n", "lgb_test_f1 = f1_score(y_test, y_test_pred_lgb)\n", "lgb_test_auc = roc_auc_score(y_test, y_test_proba_lgb)\n", "lgb_test_accuracy = accuracy_score(y_test, y_test_pred_lgb)\n", "\n", "print(f\"🎯 Test F1-score: {lgb_test_f1:.4f}\")\n", "print(f\"πŸ“ˆ Test AUC: {lgb_test_auc:.4f}\")\n", "print(f\"🎯 Test Accuracy: {lgb_test_accuracy:.4f}\")\n", "\n", "# Detailed classification report\n", "print(f\"\\nπŸ“‹ Detailed Classification Report:\")\n", "print(classification_report(y_test, y_test_pred_lgb, target_names=['No Catch', 'Catch']))\n", "\n", "# Store results for comparison\n", "lgb_results = {\n", " 'model': lgb_final,\n", " 'threshold': optimal_threshold_lgb,\n", " 'test_f1': lgb_test_f1,\n", " 'test_auc': lgb_test_auc,\n", " 'test_accuracy': lgb_test_accuracy,\n", " 'y_pred': y_test_pred_lgb,\n", " 'y_proba': y_test_proba_lgb,\n", " 'best_params': lgb_grid.best_params_\n", "}\n", "\n", "print(\"\\nβœ… LightGBMClassifier training complete!\")\n", "\n", "# Generate comprehensive classification report\n", "print(\"\\nπŸ“‹ LightGBM Classification Report:\")\n", "print(\"=\" * 60)\n", "print(classification_report(y_test, y_test_pred_lgb, target_names=['No Catch', 'Catch']))\n", "\n", "# Create interactive confusion matrix\n", "cm_lgb = confusion_matrix(y_test, y_test_pred_lgb)\n", "\n", "fig_cm_lgb = ff.create_annotated_heatmap(\n", " z=cm_lgb,\n", " x=['Predicted: No Catch', 'Predicted: Catch'],\n", " y=['Actual: No Catch', 'Actual: Catch'],\n", " annotation_text=cm_lgb,\n", " colorscale='Purples',\n", " showscale=True\n", ")\n", "\n", "fig_cm_lgb.update_layout(\n", " title='⚑ LightGBM Confusion Matrix',\n", " title_x=0.5,\n", " width=500,\n", " height=400\n", ")\n", "\n", "fig_cm_lgb.show()\n", "\n", "# Create interactive ROC curve\n", "fpr_lgb, tpr_lgb, _ = roc_curve(y_test, y_test_proba_lgb)\n", "\n", "fig_roc_lgb = go.Figure()\n", "\n", "fig_roc_lgb.add_trace(go.Scatter(\n", " x=fpr_lgb,\n", " y=tpr_lgb,\n", " mode='lines',\n", " name=f'LightGBM (AUC = {lgb_test_auc:.3f})',\n", " line=dict(color='purple', width=3)\n", "))\n", "\n", "# Add diagonal reference line\n", "fig_roc_lgb.add_trace(go.Scatter(\n", " x=[0, 1],\n", " y=[0, 1],\n", " mode='lines',\n", " name='Random Classifier',\n", " line=dict(color='red', width=2, dash='dash')\n", "))\n", "\n", "fig_roc_lgb.update_layout(\n", " title='⚑ LightGBM ROC Curve',\n", " title_x=0.5,\n", " xaxis_title='False Positive Rate',\n", " yaxis_title='True Positive Rate',\n", " width=600,\n", " height=500\n", ")\n", "\n", "fig_roc_lgb.show()\n", "\n", "# Feature importance analysis for LightGBM\n", "feature_importance_lgb = pd.DataFrame({\n", " 'feature': X_train_lgb.columns,\n", " 'importance': lgb_final.feature_importances_\n", "}).sort_values('importance', ascending=False)\n", "\n", "print(f\"\\n🎯 LightGBM - Top 10 Most Important Features:\")\n", "for i, (idx, row) in enumerate(feature_importance_lgb.head(10).iterrows()):\n", " print(f\" {i+1}. {row['feature']}: {row['importance']:.4f}\")\n", "\n", "# Create feature importance plot\n", "fig_importance_lgb = px.bar(\n", " feature_importance_lgb.head(10),\n", " x='importance',\n", " y='feature',\n", " orientation='h',\n", " title='⚑ LightGBM - Top 10 Feature Importances',\n", " labels={'importance': 'Feature Importance', 'feature': 'Features'},\n", " color_discrete_sequence=['purple']\n", ")\n", "fig_importance_lgb.update_layout(height=500, yaxis={'categoryorder':'total ascending'})\n", "fig_importance_lgb.show()\n", "\n", "print(f\"\\nπŸ† LightGBM Tournament Summary:\")\n", "print(f\" β€’ F1-Score: {lgb_test_f1:.4f}\")\n", "print(f\" β€’ AUC: {lgb_test_auc:.4f}\")\n", "print(f\" β€’ Optimal Threshold: {optimal_threshold_lgb:.3f}\")\n", "print(f\" β€’ Test Accuracy: {lgb_test_accuracy:.4f}\")\n", "print(f\" β€’ Best Parameters: {lgb_results['best_params']}\")" ] }, { "cell_type": "markdown", "id": "13411e02", "metadata": {}, "source": [ "# πŸ† Standard Classifiers Champion Selection\n", "\n", "The moment of truth has arrived! Let's compare all three contestants and crown our **Standard Classifier Champion**. We'll use F1-score as our primary metric (perfect for imbalanced datasets) and AUC as our secondary metric." ] }, { "cell_type": "code", "execution_count": 25, "id": "3188fd88", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "πŸ† STANDARD CLASSIFIERS TOURNAMENT RESULTS\n", "======================================================================\n", "πŸ“Š Final Standings:\n" ] }, { "data": { "application/vnd.microsoft.datawrangler.viewer.v0+json": { "columns": [ { "name": "index", "rawType": "int64", "type": "integer" }, { "name": "Model", "rawType": "object", "type": "string" }, { "name": "F1-Score", "rawType": "float64", "type": "float" }, { "name": "AUC", "rawType": "float64", "type": "float" }, { "name": "Accuracy", "rawType": "float64", "type": "float" }, { "name": "Optimal Threshold", "rawType": "float64", "type": "float" } ], "ref": "be2196b3-193a-4c6a-a69e-182ed486f2f4", "rows": [ [ "0", "🌳 RandomForest", "0.6667", "0.9194", "0.8163", "0.1" ], [ "2", "⚑ LightGBM", "0.5625", "0.925", "0.7143", "0.1" ], [ "1", "πŸš€ XGBoost", "0.4737", "0.9319", "0.5918", "0.1" ] ], "shape": { "columns": 5, "rows": 3 } }, "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
ModelF1-ScoreAUCAccuracyOptimal Threshold
0🌳 RandomForest0.66670.91940.81630.1
2⚑ LightGBM0.56250.92500.71430.1
1πŸš€ XGBoost0.47370.93190.59180.1
\n", "
" ], "text/plain": [ " Model F1-Score AUC Accuracy Optimal Threshold\n", "0 🌳 RandomForest 0.6667 0.9194 0.8163 0.1\n", "2 ⚑ LightGBM 0.5625 0.9250 0.7143 0.1\n", "1 πŸš€ XGBoost 0.4737 0.9319 0.5918 0.1" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "\n", "πŸ₯‡ STANDARD CLASSIFIER CHAMPION: 🌳 RandomForest\n", " β€’ Winning F1-Score: 0.6667\n", " β€’ Winning AUC: 0.9194\n" ] }, { "data": { "application/vnd.plotly.v1+json": { "config": { "plotlyServerURL": "https://plot.ly" }, "data": [ { "marker": { "color": [ "green", "orange", "purple" ] }, "name": "F1-Score", "text": [ "0.6667", "0.4737", "0.5625" ], "textposition": "outside", "type": "bar", "x": [ "RandomForest", "XGBoost", "LightGBM" ], "xaxis": "x", "y": [ 0.6666666666666666, 0.47368421052631576, 0.5625 ], "yaxis": "y" }, { "marker": { "color": [ "green", "orange", "purple" ] }, "name": "AUC", "showlegend": false, "text": [ "0.9194", "0.9319", "0.9250" ], "textposition": "outside", "type": "bar", "x": [ "RandomForest", "XGBoost", "LightGBM" ], "xaxis": "x2", "y": [ 0.9194444444444445, 0.9319444444444445, 0.9249999999999999 ], "yaxis": "y2" } ], "layout": { "annotations": [ { "font": { "size": 16 }, "showarrow": false, "text": "F1-Score Comparison", "x": 0.225, "xanchor": "center", "xref": "paper", "y": 1, "yanchor": "bottom", "yref": "paper" }, { "font": { "size": 16 }, "showarrow": false, "text": "AUC Comparison", "x": 0.775, "xanchor": "center", "xref": "paper", "y": 1, "yanchor": "bottom", "yref": "paper" } ], "height": 500, "showlegend": false, "template": { "data": { "bar": [ { "error_x": { "color": "#2a3f5f" }, "error_y": { "color": "#2a3f5f" }, "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "bar" } ], "barpolar": [ { "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "barpolar" } ], "carpet": [ { "aaxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "baxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "type": "carpet" } ], "choropleth": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "choropleth" } ], "contour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "contour" } ], "contourcarpet": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "contourcarpet" } ], "heatmap": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "heatmap" } ], "heatmapgl": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "heatmapgl" } ], "histogram": [ { "marker": { "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "histogram" } ], "histogram2d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "histogram2d" } ], "histogram2dcontour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "histogram2dcontour" } ], "mesh3d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "mesh3d" } ], "parcoords": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "parcoords" } ], "pie": [ { "automargin": true, "type": "pie" } ], "scatter": [ { "fillpattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 }, "type": "scatter" } ], "scatter3d": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatter3d" } ], "scattercarpet": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattercarpet" } ], "scattergeo": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergeo" } ], "scattergl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergl" } ], "scattermapbox": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattermapbox" } ], "scatterpolar": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolar" } ], "scatterpolargl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolargl" } ], "scatterternary": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterternary" } ], "surface": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "surface" } ], "table": [ { "cells": { "fill": { "color": "#EBF0F8" }, "line": { "color": "white" } }, "header": { "fill": { "color": "#C8D4E3" }, "line": { "color": "white" } }, "type": "table" } ] }, "layout": { "annotationdefaults": { "arrowcolor": "#2a3f5f", "arrowhead": 0, "arrowwidth": 1 }, "autotypenumbers": "strict", "coloraxis": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "colorscale": { "diverging": [ [ 0, "#8e0152" ], [ 0.1, "#c51b7d" ], [ 0.2, "#de77ae" ], [ 0.3, "#f1b6da" ], [ 0.4, "#fde0ef" ], [ 0.5, "#f7f7f7" ], [ 0.6, "#e6f5d0" ], [ 0.7, "#b8e186" ], [ 0.8, "#7fbc41" ], [ 0.9, "#4d9221" ], [ 1, "#276419" ] ], "sequential": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "sequentialminus": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ] }, "colorway": [ "#636efa", "#EF553B", "#00cc96", "#ab63fa", "#FFA15A", "#19d3f3", "#FF6692", "#B6E880", "#FF97FF", "#FECB52" ], "font": { "color": "#2a3f5f" }, "geo": { "bgcolor": "white", "lakecolor": "white", "landcolor": "#E5ECF6", "showlakes": true, "showland": true, "subunitcolor": "white" }, "hoverlabel": { "align": "left" }, "hovermode": "closest", "mapbox": { "style": "light" }, "paper_bgcolor": "white", "plot_bgcolor": "#E5ECF6", "polar": { "angularaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "radialaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "scene": { "xaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "yaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "zaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" } }, "shapedefaults": { "line": { "color": "#2a3f5f" } }, "ternary": { "aaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "baxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "caxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "title": { "x": 0.05 }, "xaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 }, "yaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 } } }, "title": { "text": "πŸ† Standard Classifiers Tournament Results", "x": 0.5 }, "xaxis": { "anchor": "y", "domain": [ 0, 0.45 ] }, "xaxis2": { "anchor": "y2", "domain": [ 0.55, 1 ] }, "yaxis": { "anchor": "x", "domain": [ 0, 1 ] }, "yaxis2": { "anchor": "x2", "domain": [ 0, 1 ] } } } }, "metadata": {}, "output_type": "display_data" }, { "data": { "application/vnd.plotly.v1+json": { "config": { "plotlyServerURL": "https://plot.ly" }, "data": [ { "line": { "color": "green", "width": 3 }, "mode": "lines", "name": "🌳 RandomForest (AUC = 0.919)", "type": "scatter", "x": [ 0, 0, 0, 0.025, 0.075, 0.1, 0.1, 0.125, 0.125, 0.375, 0.425, 0.825, 0.875, 1 ], "y": [ 0, 0.1111111111111111, 0.2222222222222222, 0.2222222222222222, 0.2222222222222222, 0.2222222222222222, 0.8888888888888888, 0.8888888888888888, 1, 1, 1, 1, 1, 1 ] }, { "line": { "color": "orange", "width": 3 }, "mode": "lines", "name": "πŸš€ XGBoost (AUC = 0.932)", "type": "scatter", "x": [ 0, 0, 0.025, 0.075, 0.075, 0.125, 0.125, 0.15, 0.425, 0.5, 0.625, 0.675, 0.825, 0.925, 1 ], "y": [ 0, 0.1111111111111111, 0.2222222222222222, 0.3333333333333333, 0.6666666666666666, 0.8888888888888888, 1, 1, 1, 1, 1, 1, 1, 1, 1 ] }, { "line": { "color": "purple", "width": 3 }, "mode": "lines", "name": "⚑ LightGBM (AUC = 0.925)", "type": "scatter", "x": [ 0, 0, 0.05, 0.05, 0.075, 0.075, 0.1, 0.1, 0.75, 0.825, 1 ], "y": [ 0, 0.1111111111111111, 0.1111111111111111, 0.3333333333333333, 0.3333333333333333, 0.4444444444444444, 0.4444444444444444, 1, 1, 1, 1 ] }, { "line": { "color": "red", "dash": "dash", "width": 2 }, "mode": "lines", "name": "Random Classifier", "type": "scatter", "x": [ 0, 1 ], "y": [ 0, 1 ] } ], "layout": { "height": 600, "template": { "data": { "bar": [ { "error_x": { "color": "#2a3f5f" }, "error_y": { "color": "#2a3f5f" }, "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "bar" } ], "barpolar": [ { "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "barpolar" } ], "carpet": [ { "aaxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "baxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "type": "carpet" } ], "choropleth": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "choropleth" } ], "contour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "contour" } ], "contourcarpet": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "contourcarpet" } ], "heatmap": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "heatmap" } ], "heatmapgl": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "heatmapgl" } ], "histogram": [ { "marker": { "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "histogram" } ], "histogram2d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "histogram2d" } ], "histogram2dcontour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "histogram2dcontour" } ], "mesh3d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "mesh3d" } ], "parcoords": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "parcoords" } ], "pie": [ { "automargin": true, "type": "pie" } ], "scatter": [ { "fillpattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 }, "type": "scatter" } ], "scatter3d": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatter3d" } ], "scattercarpet": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattercarpet" } ], "scattergeo": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergeo" } ], "scattergl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergl" } ], "scattermapbox": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattermapbox" } ], "scatterpolar": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolar" } ], "scatterpolargl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolargl" } ], "scatterternary": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterternary" } ], "surface": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "surface" } ], "table": [ { "cells": { "fill": { "color": "#EBF0F8" }, "line": { "color": "white" } }, "header": { "fill": { "color": "#C8D4E3" }, "line": { "color": "white" } }, "type": "table" } ] }, "layout": { "annotationdefaults": { "arrowcolor": "#2a3f5f", "arrowhead": 0, "arrowwidth": 1 }, "autotypenumbers": "strict", "coloraxis": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "colorscale": { "diverging": [ [ 0, "#8e0152" ], [ 0.1, "#c51b7d" ], [ 0.2, "#de77ae" ], [ 0.3, "#f1b6da" ], [ 0.4, "#fde0ef" ], [ 0.5, "#f7f7f7" ], [ 0.6, "#e6f5d0" ], [ 0.7, "#b8e186" ], [ 0.8, "#7fbc41" ], [ 0.9, "#4d9221" ], [ 1, "#276419" ] ], "sequential": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "sequentialminus": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ] }, "colorway": [ "#636efa", "#EF553B", "#00cc96", "#ab63fa", "#FFA15A", "#19d3f3", "#FF6692", "#B6E880", "#FF97FF", "#FECB52" ], "font": { "color": "#2a3f5f" }, "geo": { "bgcolor": "white", "lakecolor": "white", "landcolor": "#E5ECF6", "showlakes": true, "showland": true, "subunitcolor": "white" }, "hoverlabel": { "align": "left" }, "hovermode": "closest", "mapbox": { "style": "light" }, "paper_bgcolor": "white", "plot_bgcolor": "#E5ECF6", "polar": { "angularaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "radialaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "scene": { "xaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "yaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "zaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" } }, "shapedefaults": { "line": { "color": "#2a3f5f" } }, "ternary": { "aaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "baxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "caxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "title": { "x": 0.05 }, "xaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 }, "yaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 } } }, "title": { "text": "πŸ† Combined ROC Curves - Standard Classifiers Tournament", "x": 0.5 }, "width": 700, "xaxis": { "title": { "text": "False Positive Rate" } }, "yaxis": { "title": { "text": "True Positive Rate" } } } } }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "\n", "πŸ“Š Performance Insights:\n", " πŸ₯‡ Champion: RandomForest with F1-Score: 0.6667\n", " πŸ”„ All models used optimal thresholds for imbalanced data\n", " βš–οΈ Class weights and TimeSeriesSplit ensured fair comparison\n", " πŸ“ˆ AUC scores show model discrimination ability\n", "\n", "🎯 Standard Tournament Complete!\n", " πŸ₯‡ Champion: RandomForest\n", " πŸ“Š Champion advances to Grand Finale with F1-Score: 0.6667\n", "\n", "πŸ”œ Next Phase: Deep Learning Tournament awaits...\n" ] } ], "source": [ "print(\"πŸ† STANDARD CLASSIFIERS TOURNAMENT RESULTS\")\n", "print(\"=\" * 70)\n", "\n", "# Create comprehensive comparison table\n", "comparison_data = {\n", " 'Model': ['🌳 RandomForest', 'πŸš€ XGBoost', '⚑ LightGBM'],\n", " 'F1-Score': [rf_results['test_f1'], xgb_results['test_f1'], lgb_results['test_f1']],\n", " 'AUC': [rf_results['test_auc'], xgb_results['test_auc'], lgb_results['test_auc']],\n", " 'Accuracy': [rf_results['test_accuracy'], xgb_results['test_accuracy'], lgb_results['test_accuracy']],\n", " 'Optimal Threshold': [rf_results['threshold'], xgb_results['threshold'], lgb_results['threshold']]\n", "}\n", "\n", "comparison_df = pd.DataFrame(comparison_data)\n", "\n", "# Sort by F1-Score (primary) and AUC (secondary)\n", "comparison_df = comparison_df.sort_values(['F1-Score', 'AUC'], ascending=False)\n", "\n", "print(\"πŸ“Š Final Standings:\")\n", "display(comparison_df.round(4))\n", "\n", "# Determine champion\n", "champion_idx = comparison_df.index[0]\n", "champion_name = comparison_df.iloc[0]['Model']\n", "champion_f1 = comparison_df.iloc[0]['F1-Score']\n", "champion_auc = comparison_df.iloc[0]['AUC']\n", "\n", "print(f\"\\nπŸ₯‡ STANDARD CLASSIFIER CHAMPION: {champion_name}\")\n", "print(f\" β€’ Winning F1-Score: {champion_f1:.4f}\")\n", "print(f\" β€’ Winning AUC: {champion_auc:.4f}\")\n", "\n", "# Store champion for later use\n", "if 'RandomForest' in champion_name:\n", " standard_champion = rf_results\n", " standard_champion_name = 'RandomForest'\n", "elif 'XGBoost' in champion_name:\n", " standard_champion = xgb_results\n", " standard_champion_name = 'XGBoost'\n", "else:\n", " standard_champion = lgb_results\n", " standard_champion_name = 'LightGBM'\n", "\n", "# Create interactive comparison visualization\n", "fig_comparison = make_subplots(\n", " rows=1, cols=2,\n", " subplot_titles=['F1-Score Comparison', 'AUC Comparison'],\n", " specs=[[{\"type\": \"bar\"}, {\"type\": \"bar\"}]]\n", ")\n", "\n", "models = ['RandomForest', 'XGBoost', 'LightGBM']\n", "colors = ['green', 'orange', 'purple']\n", "f1_scores = [rf_results['test_f1'], xgb_results['test_f1'], lgb_results['test_f1']]\n", "auc_scores = [rf_results['test_auc'], xgb_results['test_auc'], lgb_results['test_auc']]\n", "\n", "# F1-Score comparison\n", "fig_comparison.add_trace(\n", " go.Bar(x=models, y=f1_scores, name='F1-Score', marker_color=colors, \n", " text=[f'{score:.4f}' for score in f1_scores], textposition='outside'),\n", " row=1, col=1\n", ")\n", "\n", "# AUC comparison\n", "fig_comparison.add_trace(\n", " go.Bar(x=models, y=auc_scores, name='AUC', marker_color=colors, \n", " text=[f'{score:.4f}' for score in auc_scores], textposition='outside', showlegend=False),\n", " row=1, col=2\n", ")\n", "\n", "fig_comparison.update_layout(\n", " title='πŸ† Standard Classifiers Tournament Results',\n", " title_x=0.5,\n", " height=500,\n", " showlegend=False\n", ")\n", "\n", "fig_comparison.show()\n", "\n", "# Combined ROC curves for all models\n", "fig_combined_roc = go.Figure()\n", "\n", "# RandomForest\n", "fpr_rf, tpr_rf, _ = roc_curve(y_test, rf_results['y_proba'])\n", "fig_combined_roc.add_trace(go.Scatter(\n", " x=fpr_rf, y=tpr_rf, mode='lines',\n", " name=f'🌳 RandomForest (AUC = {rf_results[\"test_auc\"]:.3f})',\n", " line=dict(color='green', width=3)\n", "))\n", "\n", "# XGBoost\n", "fpr_xgb, tpr_xgb, _ = roc_curve(y_test, xgb_results['y_proba'])\n", "fig_combined_roc.add_trace(go.Scatter(\n", " x=fpr_xgb, y=tpr_xgb, mode='lines',\n", " name=f'πŸš€ XGBoost (AUC = {xgb_results[\"test_auc\"]:.3f})',\n", " line=dict(color='orange', width=3)\n", "))\n", "\n", "# LightGBM\n", "fpr_lgb, tpr_lgb, _ = roc_curve(y_test, lgb_results['y_proba'])\n", "fig_combined_roc.add_trace(go.Scatter(\n", " x=fpr_lgb, y=tpr_lgb, mode='lines',\n", " name=f'⚑ LightGBM (AUC = {lgb_results[\"test_auc\"]:.3f})',\n", " line=dict(color='purple', width=3)\n", "))\n", "\n", "# Random classifier reference\n", "fig_combined_roc.add_trace(go.Scatter(\n", " x=[0, 1], y=[0, 1], mode='lines',\n", " name='Random Classifier',\n", " line=dict(color='red', width=2, dash='dash')\n", "))\n", "\n", "fig_combined_roc.update_layout(\n", " title='πŸ† Combined ROC Curves - Standard Classifiers Tournament',\n", " title_x=0.5,\n", " xaxis_title='False Positive Rate',\n", " yaxis_title='True Positive Rate',\n", " width=700,\n", " height=600\n", ")\n", "\n", "fig_combined_roc.show()\n", "\n", "# Performance insights\n", "print(f\"\\nπŸ“Š Performance Insights:\")\n", "print(f\" πŸ₯‡ Champion: {standard_champion_name} with F1-Score: {champion_f1:.4f}\")\n", "print(f\" πŸ”„ All models used optimal thresholds for imbalanced data\")\n", "print(f\" βš–οΈ Class weights and TimeSeriesSplit ensured fair comparison\")\n", "print(f\" πŸ“ˆ AUC scores show model discrimination ability\")\n", "\n", "print(f\"\\n🎯 Standard Tournament Complete!\")\n", "print(f\" πŸ₯‡ Champion: {standard_champion_name}\")\n", "print(f\" πŸ“Š Champion advances to Grand Finale with F1-Score: {champion_f1:.4f}\")\n", "print(f\"\\nπŸ”œ Next Phase: Deep Learning Tournament awaits...\")" ] }, { "cell_type": "markdown", "id": "9095286c", "metadata": {}, "source": [ "## 🏁 Part 1 Complete - Tournament Summary\n", "\n", "**Congratulations!** Part 1 of our Pest Classification Tournament is now complete. We have successfully:\n", "\n", "βœ… **Data Preparation**: \n", "- Loaded and explored our time-series pest data\n", "- Identified and handled severe class imbalance\n", "- Performed chronological train-test split\n", "- Calculated appropriate class weights\n", "\n", "βœ… **Standard ML Tournament**:\n", "- Trained 3 distinct classifiers with unique hyperparameters\n", "- Used TimeSeriesSplit for proper temporal validation\n", "- Found optimal prediction thresholds for each model\n", "- Conducted comprehensive performance analysis\n", "\n", "βœ… **Champion Selection**:\n", "- Compared models using F1-score (primary) and AUC (secondary)\n", "- Crowned our **Standard Classifier Champion**\n", "- Generated interactive visualizations and insights\n", "\n", "### Key Achievements:\n", "- 🚫 **No Data Leakage**: Chronological splits maintained\n", "- βš–οΈ **Robust Imbalance Handling**: Class weights + optimal thresholds\n", "- πŸ“Š **Fair Comparison**: Each model optimized independently\n", "- πŸ“ˆ **Interactive Analysis**: Plotly visualizations throughout\n", "\n", "Our Standard Classifier Champion is now ready to compete in the **Grand Finale** against the Deep Learning Champion!\n", "\n", "---\n", "**Ready for Part 2?** The Deep Learning Tournament awaits! πŸš€" ] }, { "cell_type": "code", "execution_count": 26, "id": "b641fea4", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "πŸ’Ύ Saving Part 1 Tournament Results...\n", "βœ… Part 1 results saved:\n", " β€’ Summary: part1_standard_tournament_results.json\n", " β€’ Champion model: part1_champion_randomforest.joblib\n", "\n", "πŸ† Part 1 Complete! Champion: RandomForest (F1: 0.6667)\n", "πŸ”œ Ready for Part 2: Deep Learning Tournament!\n" ] } ], "source": [ "# Save Part 1 results for later use\n", "print(\"πŸ’Ύ Saving Part 1 Tournament Results...\")\n", "\n", "# Create summary dictionary\n", "part1_summary = {\n", " 'tournament_type': 'Standard_Classifiers',\n", " 'models_tested': ['RandomForest', 'XGBoost', 'LightGBM'],\n", " 'champion': {\n", " 'name': standard_champion_name,\n", " 'f1_score': float(champion_f1),\n", " 'auc_score': float(champion_auc),\n", " 'threshold': float(standard_champion['threshold']),\n", " 'accuracy': float(standard_champion['test_accuracy'])\n", " },\n", " 'all_results': {\n", " 'RandomForest': {\n", " 'f1_score': float(rf_results['test_f1']),\n", " 'auc_score': float(rf_results['test_auc']),\n", " 'threshold': float(rf_results['threshold']),\n", " 'accuracy': float(rf_results['test_accuracy'])\n", " },\n", " 'XGBoost': {\n", " 'f1_score': float(xgb_results['test_f1']),\n", " 'auc_score': float(xgb_results['test_auc']),\n", " 'threshold': float(xgb_results['threshold']),\n", " 'accuracy': float(xgb_results['test_accuracy'])\n", " },\n", " 'LightGBM': {\n", " 'f1_score': float(lgb_results['test_f1']),\n", " 'auc_score': float(lgb_results['test_auc']),\n", " 'threshold': float(lgb_results['threshold']),\n", " 'accuracy': float(lgb_results['test_accuracy'])\n", " }\n", " },\n", " 'dataset_info': {\n", " 'total_samples': len(df_engineered),\n", " 'train_samples': len(X_train),\n", " 'test_samples': len(X_test),\n", " 'features_count': len(feature_cols),\n", " 'class_imbalance_ratio': float(imbalance_ratio)\n", " }\n", "}\n", "\n", "# Save to JSON file\n", "with open('part1_standard_tournament_results.json', 'w') as f:\n", " json.dump(part1_summary, f, indent=2)\n", "\n", "# Save champion model\n", "joblib.dump(standard_champion['model'], f'part1_champion_{standard_champion_name.lower()}.joblib')\n", "\n", "print(f\"βœ… Part 1 results saved:\")\n", "print(f\" β€’ Summary: part1_standard_tournament_results.json\")\n", "print(f\" β€’ Champion model: part1_champion_{standard_champion_name.lower()}.joblib\")\n", "print(f\"\\nπŸ† Part 1 Complete! Champion: {standard_champion_name} (F1: {champion_f1:.4f})\")\n", "print(f\"πŸ”œ Ready for Part 2: Deep Learning Tournament!\")" ] }, { "cell_type": "markdown", "id": "61c5fa4d", "metadata": {}, "source": [ "---\n", "\n", "# πŸ“ **CRITICAL INSTRUCTION**\n", "\n", "**Part 1 is now COMPLETE!** \n", "\n", "Please reply with **\"OK\"** or **\"proceed\"** to continue to **Part 2: Deep Learning Tournament**.\n", "\n", "**Do NOT proceed to Part 2 without confirming Part 1 is working correctly.**\n", "\n", "---" ] }, { "cell_type": "markdown", "id": "6e9433b7", "metadata": {}, "source": [ "# 🧠 Part 2: Time-Series Deep Learning Tournament\n", "\n", "## Entering the Neural Arena\n", "\n", "Welcome to **Part 2** of our tournament! Having crowned our **Standard Classifier Champion**, we now shift our focus to the realm of **deep learning**. While traditional ML models excel with engineered features, neural networks have a unique superpower: they can **learn temporal patterns directly** from sequential data.\n", "\n", "### πŸ”„ Paradigm Shift: From Features to Sequences\n", "\n", "In Part 1, we used carefully engineered features like:\n", "- `Temp_Avg_3d`, `Humidity_Avg_3d` (rolling averages)\n", "- `Insects_Lag1`, `Insects_Lag3` (lag features)\n", "- `Recent_Activity`, `Days_Since_Cleaning` (temporal indicators)\n", "\n", "Now, our deep learning models will **discover these patterns automatically** by analyzing the raw time-series data directly!\n", "\n", "### 🏟️ Deep Learning Contestants:\n", "\n", "1. **πŸ”— LSTM (Long Short-Term Memory)**: \n", " - Master of long-term dependencies\n", " - Remembers important patterns across extended time periods\n", " - Excellent for complex temporal relationships\n", "\n", "2. **⚑ GRU (Gated Recurrent Unit)**:\n", " - Streamlined alternative to LSTM\n", " - Faster training with comparable performance\n", " - Efficient memory usage\n", "\n", "### 🎯 Tournament Rules Remain Sacred:\n", "\n", "βœ… **Same Data Splits**: Identical train/test indices from Part 1\n", "βœ… **Same Class Balancing**: Using computed class weights\n", "βœ… **Same Threshold Optimization**: F1-score maximization\n", "βœ… **Same Evaluation Framework**: F1, AUC, interactive visualizations\n", "βœ… **Same Chronological Integrity**: No data leakage\n", "\n", "Let the neural battle begin! βš”οΈπŸ§ " ] }, { "cell_type": "markdown", "id": "60d4c901", "metadata": {}, "source": [ "## πŸ“Š Section 4.1: Data Preparation for Sequential Models\n", "\n", "### The Sequential Data Challenge\n", "\n", "Deep learning models require a fundamentally different data structure than traditional ML algorithms. Instead of treating each day as an independent sample, we need to create **sequences** where each sample contains multiple consecutive time steps.\n", "\n", "**Transformation Required:**\n", "- **From**: `(samples, features)` β†’ Traditional 2D matrix\n", "- **To**: `(samples, timesteps, features)` β†’ 3D tensor for RNNs\n", "\n", "### Why Use cleaned_merged_data.csv?\n", "\n", "While Part 1 used engineered features, our neural networks will work with **raw temporal signals** to automatically discover patterns. The `cleaned_merged_data.csv` contains the original time-series data perfect for this approach.\n", "\n", "**Key Advantages:**\n", "- πŸ”„ **Automatic Feature Learning**: No manual engineering needed\n", "- πŸ“ˆ **Temporal Dependencies**: Models learn from sequence patterns\n", "- 🎯 **End-to-End Learning**: Direct mapping from sequences to predictions" ] }, { "cell_type": "code", "execution_count": 27, "id": "4f73533e", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "🧠 PART 2: TIME-SERIES DEEP LEARNING TOURNAMENT\n", "============================================================\n", "πŸ“– Loading cleaned_merged_data.csv for sequential modeling...\n", "βœ… Sequential dataset loaded: 245 rows Γ— 9 columns\n", "πŸ“… Date range: 2024-07-06 to 2024-08-23\n", "\n", "πŸ“Š Available columns for sequential modeling:\n", "['Date', 'Location', 'Average Temperature', 'Temperature Range (Low)', 'Temperature Range (High)', 'Average Humidity', 'Number of insects', 'New catches', 'Event']\n" ] }, { "data": { "application/vnd.microsoft.datawrangler.viewer.v0+json": { "columns": [ { "name": "index", "rawType": "int64", "type": "integer" }, { "name": "Date", "rawType": "datetime64[ns]", "type": "datetime" }, { "name": "Location", "rawType": "object", "type": "string" }, { "name": "Average Temperature", "rawType": "float64", "type": "float" }, { "name": "Temperature Range (Low)", "rawType": "float64", "type": "float" }, { "name": "Temperature Range (High)", "rawType": "float64", "type": "float" }, { "name": "Average Humidity", "rawType": "float64", "type": "float" }, { "name": "Number of insects", "rawType": "float64", "type": "float" }, { "name": "New catches", "rawType": "float64", "type": "float" }, { "name": "Event", "rawType": "float64", "type": "float" } ], "ref": "af3c93e2-52f1-4c56-bfdd-3627d7af1c8e", "rows": [ [ "0", "2024-07-06 00:00:00", "Cicalino 1", "22.34", "21.53", "23.12", "72.25", "0.0", "0.0", "0.0" ], [ "1", "2024-07-06 00:00:00", "Cicalino 2", "26.17", "25.28", "27.0", "56.06", "0.0", "0.0", "0.0" ], [ "2", "2024-07-06 00:00:00", "Imola 1", "29.68", "29.11", "30.24", "42.93", "0.0", "0.0", "0.0" ], [ "3", "2024-07-06 00:00:00", "Imola 2", "28.83", "28.04", "29.76", "52.45", "0.0", "0.0", "0.0" ], [ "4", "2024-07-06 00:00:00", "Imola 3", "26.89", "25.77", "27.85", "64.88", "0.0", "0.0", "0.0" ] ], "shape": { "columns": 9, "rows": 5 } }, "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
DateLocationAverage TemperatureTemperature Range (Low)Temperature Range (High)Average HumidityNumber of insectsNew catchesEvent
02024-07-06Cicalino 122.3421.5323.1272.250.00.00.0
12024-07-06Cicalino 226.1725.2827.0056.060.00.00.0
22024-07-06Imola 129.6829.1130.2442.930.00.00.0
32024-07-06Imola 228.8328.0429.7652.450.00.00.0
42024-07-06Imola 326.8925.7727.8564.880.00.00.0
\n", "
" ], "text/plain": [ " Date Location Average Temperature Temperature Range (Low) \\\n", "0 2024-07-06 Cicalino 1 22.34 21.53 \n", "1 2024-07-06 Cicalino 2 26.17 25.28 \n", "2 2024-07-06 Imola 1 29.68 29.11 \n", "3 2024-07-06 Imola 2 28.83 28.04 \n", "4 2024-07-06 Imola 3 26.89 25.77 \n", "\n", " Temperature Range (High) Average Humidity Number of insects New catches \\\n", "0 23.12 72.25 0.0 0.0 \n", "1 27.00 56.06 0.0 0.0 \n", "2 30.24 42.93 0.0 0.0 \n", "3 29.76 52.45 0.0 0.0 \n", "4 27.85 64.88 0.0 0.0 \n", "\n", " Event \n", "0 0.0 \n", "1 0.0 \n", "2 0.0 \n", "3 0.0 \n", "4 0.0 " ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "from sklearn.preprocessing import MinMaxScaler\n", "from tensorflow.keras.preprocessing.sequence import TimeseriesGenerator\n", "from tensorflow.keras.models import Sequential\n", "from tensorflow.keras.layers import LSTM, GRU, Dense, Dropout\n", "from tensorflow.keras.callbacks import EarlyStopping\n", "from tensorflow.keras.optimizers import Adam\n", "import tensorflow.keras.metrics as keras_metrics\n", "\n", "print(\"🧠 PART 2: TIME-SERIES DEEP LEARNING TOURNAMENT\")\n", "print(\"=\" * 60)\n", "\n", "# Load the cleaned merged data for sequential modeling\n", "print(\"πŸ“– Loading cleaned_merged_data.csv for sequential modeling...\")\n", "df_merged_seq = pd.read_csv('cleaned_merged_data.csv')\n", "\n", "print(f\"βœ… Sequential dataset loaded: {df_merged_seq.shape[0]:,} rows Γ— {df_merged_seq.shape[1]} columns\")\n", "print(f\"πŸ“… Date range: {df_merged_seq['Date'].min()} to {df_merged_seq['Date'].max()}\")\n", "\n", "# Convert Date to datetime and sort chronologically\n", "df_merged_seq['Date'] = pd.to_datetime(df_merged_seq['Date'])\n", "df_merged_seq = df_merged_seq.sort_values('Date').reset_index(drop=True)\n", "\n", "# Display the structure\n", "print(\"\\nπŸ“Š Available columns for sequential modeling:\")\n", "print(list(df_merged_seq.columns))\n", "display(df_merged_seq.head())" ] }, { "cell_type": "code", "execution_count": 28, "id": "03dd7682", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "πŸ”§ Preparing features for sequential models...\n", "πŸ“‹ Available columns in merged dataset:\n", " 1. Date\n", " 2. Location\n", " 3. Average Temperature\n", " 4. Temperature Range (Low)\n", " 5. Temperature Range (High)\n", " 6. Average Humidity\n", " 7. Number of insects\n", " 8. New catches\n", " 9. Event\n", "βœ… Found feature: Average Temperature\n", "βœ… Found feature: Average Humidity\n", "βœ… Found feature: Number of insects\n", "βœ… Found feature: Location\n", "❌ Feature not found: location\n", "\n", "🎯 Final selected features for time-series: ['Average Temperature', 'Average Humidity', 'Number of insects', 'Location']\n", "βœ… Time-series features shape: (245, 4)\n", "βœ… Target shape: (245,)\n", "\n", "πŸ“Š First 5 rows of selected features:\n" ] }, { "data": { "application/vnd.microsoft.datawrangler.viewer.v0+json": { "columns": [ { "name": "index", "rawType": "int64", "type": "integer" }, { "name": "Average Temperature", "rawType": "float64", "type": "float" }, { "name": "Average Humidity", "rawType": "float64", "type": "float" }, { "name": "Number of insects", "rawType": "float64", "type": "float" }, { "name": "Location", "rawType": "object", "type": "string" } ], "ref": "87b06e97-b40b-41cf-af37-0daa92c3ee13", "rows": [ [ "0", "22.34", "72.25", "0.0", "Cicalino 1" ], [ "1", "26.17", "56.06", "0.0", "Cicalino 2" ], [ "2", "29.68", "42.93", "0.0", "Imola 1" ], [ "3", "28.83", "52.45", "0.0", "Imola 2" ], [ "4", "26.89", "64.88", "0.0", "Imola 3" ] ], "shape": { "columns": 4, "rows": 5 } }, "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
Average TemperatureAverage HumidityNumber of insectsLocation
022.3472.250.0Cicalino 1
126.1756.060.0Cicalino 2
229.6842.930.0Imola 1
328.8352.450.0Imola 2
426.8964.880.0Imola 3
\n", "
" ], "text/plain": [ " Average Temperature Average Humidity Number of insects Location\n", "0 22.34 72.25 0.0 Cicalino 1\n", "1 26.17 56.06 0.0 Cicalino 2\n", "2 29.68 42.93 0.0 Imola 1\n", "3 28.83 52.45 0.0 Imola 2\n", "4 26.89 64.88 0.0 Imola 3" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "\n", "βœ‚οΈ Using identical split indices from Part 1...\n", "πŸ“… Engineered data period: 2024-07-06 to 2024-08-23\n", "πŸ“… Sequential data period: 2024-07-06 to 2024-08-23\n", "βœ… Filtered sequential data shape: (245, 4)\n", "βœ… Data alignment verified: 245 samples\n", "\n", "πŸ“Š Time-series Training data: 196 samples (up to 2024-08-14)\n", "πŸ“Š Time-series Test data: 49 samples (from 2024-08-14)\n", "\n", "πŸ” Verifying class distribution alignment with Part 1:\n" ] }, { "data": { "application/vnd.microsoft.datawrangler.viewer.v0+json": { "columns": [ { "name": "New catches", "rawType": "int32", "type": "integer" }, { "name": "TS Training (%)", "rawType": "float64", "type": "float" }, { "name": "TS Test (%)", "rawType": "float64", "type": "float" } ], "ref": "5822cb55-7920-448e-be7e-7fbd37622685", "rows": [ [ "0", "93.88", "81.63" ], [ "1", "6.12", "18.37" ] ], "shape": { "columns": 2, "rows": 2 } }, "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
TS Training (%)TS Test (%)
New catches
093.8881.63
16.1218.37
\n", "
" ], "text/plain": [ " TS Training (%) TS Test (%)\n", "New catches \n", "0 93.88 81.63\n", "1 6.12 18.37" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "βœ… Time-series data splits aligned with Part 1!\n" ] } ], "source": [ "# Select features for sequential modeling (exclude date and target)\n", "print(\"\\nπŸ”§ Preparing features for sequential models...\")\n", "\n", "# Create binary target variable (same as Part 1)\n", "df_merged_seq['New catches'] = (df_merged_seq['New catches'] > 0).astype(int)\n", "\n", "# First, let's check what columns are actually available\n", "print(\"πŸ“‹ Available columns in merged dataset:\")\n", "for i, col in enumerate(df_merged_seq.columns):\n", " print(f\" {i+1}. {col}\")\n", "\n", "# Select relevant features for time-series modeling\n", "# Use only features that exist and contain temporal information\n", "available_cols = df_merged_seq.columns.tolist()\n", "\n", "# Define potential feature columns and check which ones exist\n", "potential_features = [\n", " 'Average Temperature', 'Average Humidity', 'Number of insects',\n", " 'Location', 'location' # Check for different variations of location\n", "]\n", "\n", "ts_feature_cols = []\n", "for feature in potential_features:\n", " if feature in available_cols:\n", " ts_feature_cols.append(feature)\n", " print(f\"βœ… Found feature: {feature}\")\n", " else:\n", " print(f\"❌ Feature not found: {feature}\")\n", "\n", "# If we don't have enough features, use numeric columns (excluding date and target)\n", "if len(ts_feature_cols) < 2:\n", " print(\"\\nπŸ”„ Using all available numeric features...\")\n", " numeric_cols = df_merged_seq.select_dtypes(include=[np.number]).columns.tolist()\n", " # Exclude target variable and any ID columns\n", " exclude_cols = ['New catches', 'Number of insects'] # Keep insects for learning but exclude target\n", " ts_feature_cols = [col for col in numeric_cols if col not in exclude_cols]\n", " # Add back 'Number of insects' if it exists\n", " if 'Number of insects' in available_cols:\n", " ts_feature_cols.append('Number of insects')\n", "\n", "print(f\"\\n🎯 Final selected features for time-series: {ts_feature_cols}\")\n", "\n", "# Verify all columns exist before proceeding\n", "missing_cols = [col for col in ts_feature_cols if col not in available_cols]\n", "if missing_cols:\n", " print(f\"🚨 ERROR: Missing columns: {missing_cols}\")\n", " print(\"Available columns:\", available_cols)\n", "else:\n", " X_ts = df_merged_seq[ts_feature_cols]\n", " y_ts = df_merged_seq['New catches']\n", " dates_ts = df_merged_seq['Date']\n", "\n", " print(f\"βœ… Time-series features shape: {X_ts.shape}\")\n", " print(f\"βœ… Target shape: {y_ts.shape}\")\n", " \n", " # Display first few rows to verify data\n", " print(f\"\\nπŸ“Š First 5 rows of selected features:\")\n", " display(X_ts.head())\n", "\n", "# Use EXACT same split indices from Part 1 for fair comparison\n", "print(\"\\nβœ‚οΈ Using identical split indices from Part 1...\")\n", "\n", "# We need to match the length of our sequential data to the engineered data\n", "# Find the overlap period\n", "engineered_start = pd.to_datetime(df_engineered['Date'].min())\n", "engineered_end = pd.to_datetime(df_engineered['Date'].max())\n", "\n", "print(f\"πŸ“… Engineered data period: {engineered_start.date()} to {engineered_end.date()}\")\n", "print(f\"πŸ“… Sequential data period: {dates_ts.min().date()} to {dates_ts.max().date()}\")\n", "\n", "# Filter sequential data to match engineered data period\n", "mask = (dates_ts >= engineered_start) & (dates_ts <= engineered_end)\n", "X_ts_filtered = X_ts[mask].reset_index(drop=True)\n", "y_ts_filtered = y_ts[mask].reset_index(drop=True)\n", "dates_ts_filtered = dates_ts[mask].reset_index(drop=True)\n", "\n", "print(f\"βœ… Filtered sequential data shape: {X_ts_filtered.shape}\")\n", "print(f\"βœ… Data alignment verified: {len(X_ts_filtered)} samples\")\n", "\n", "# Apply the same 80-20 split from Part 1\n", "split_idx_ts = int(len(X_ts_filtered) * 0.8)\n", "\n", "X_train_ts = X_ts_filtered.iloc[:split_idx_ts]\n", "X_test_ts = X_ts_filtered.iloc[split_idx_ts:]\n", "y_train_ts = y_ts_filtered.iloc[:split_idx_ts]\n", "y_test_ts = y_ts_filtered.iloc[split_idx_ts:]\n", "dates_train_ts = dates_ts_filtered.iloc[:split_idx_ts]\n", "dates_test_ts = dates_ts_filtered.iloc[split_idx_ts:]\n", "\n", "print(f\"\\nπŸ“Š Time-series Training data: {len(X_train_ts):,} samples (up to {dates_train_ts.iloc[-1].date()})\")\n", "print(f\"πŸ“Š Time-series Test data: {len(X_test_ts):,} samples (from {dates_test_ts.iloc[0].date()})\")\n", "\n", "# Verify class distribution matches Part 1\n", "print(\"\\nπŸ” Verifying class distribution alignment with Part 1:\")\n", "ts_train_dist = y_train_ts.value_counts(normalize=True).sort_index() * 100\n", "ts_test_dist = y_test_ts.value_counts(normalize=True).sort_index() * 100\n", "\n", "ts_comparison_df = pd.DataFrame({\n", " 'TS Training (%)': ts_train_dist,\n", " 'TS Test (%)': ts_test_dist\n", "})\n", "\n", "display(ts_comparison_df.round(2))\n", "print(\"βœ… Time-series data splits aligned with Part 1!\")" ] }, { "cell_type": "code", "execution_count": 29, "id": "a86b8713", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "βš–οΈ Scaling features for neural network training...\n", "πŸ” Checking data types and handling categorical variables...\n", "πŸ“Š Training data types:\n", "Average Temperature float64\n", "Average Humidity float64\n", "Number of insects float64\n", "Location object\n", "dtype: object\n", "πŸ“Š Categorical columns: ['Location']\n", "πŸ“Š Numeric columns: ['Average Temperature', 'Average Humidity', 'Number of insects']\n", "πŸ”„ Encoding categorical variables for neural networks...\n", "βœ… Encoded features: 7 columns\n", "πŸ”§ Applying MinMaxScaler to features...\n", "βœ… Features scaled to range [0, 1]\n", "πŸ“Š Training set shape: (196, 7)\n", "πŸ“Š Test set shape: (49, 7)\n", "\n", "πŸ“ˆ Scaling Summary (first 5 features):\n" ] }, { "data": { "application/vnd.microsoft.datawrangler.viewer.v0+json": { "columns": [ { "name": "index", "rawType": "object", "type": "string" }, { "name": "Feature", "rawType": "object", "type": "string" }, { "name": "Training Min", "rawType": "float64", "type": "float" }, { "name": "Training Max", "rawType": "float64", "type": "float" }, { "name": "Training Mean", "rawType": "float64", "type": "float" } ], "ref": "e21acaea-7edf-45b5-899a-7ab1ac04a7b6", "rows": [ [ "Average Temperature", "Average Temperature", "0.0", "1.0", "0.639" ], [ "Average Humidity", "Average Humidity", "0.0", "1.0", "0.433" ], [ "Number of insects", "Number of insects", "0.0", "1.0", "0.058" ], [ "Location_Cicalino 2", "Location_Cicalino 2", "0.0", "1.0", "0.199" ], [ "Location_Imola 1", "Location_Imola 1", "0.0", "1.0", "0.199" ] ], "shape": { "columns": 4, "rows": 5 } }, "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
FeatureTraining MinTraining MaxTraining Mean
Average TemperatureAverage Temperature0.01.00.639
Average HumidityAverage Humidity0.01.00.433
Number of insectsNumber of insects0.01.00.058
Location_Cicalino 2Location_Cicalino 20.01.00.199
Location_Imola 1Location_Imola 10.01.00.199
\n", "
" ], "text/plain": [ " Feature Training Min Training Max \\\n", "Average Temperature Average Temperature 0.0 1.0 \n", "Average Humidity Average Humidity 0.0 1.0 \n", "Number of insects Number of insects 0.0 1.0 \n", "Location_Cicalino 2 Location_Cicalino 2 0.0 1.0 \n", "Location_Imola 1 Location_Imola 1 0.0 1.0 \n", "\n", " Training Mean \n", "Average Temperature 0.639 \n", "Average Humidity 0.433 \n", "Number of insects 0.058 \n", "Location_Cicalino 2 0.199 \n", "Location_Imola 1 0.199 " ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "βœ… Feature scaling complete!\n", "πŸ“Š Total features after encoding: 7\n", "πŸ“Š Ready for sequence generation with shape: (196, 7)\n" ] } ], "source": [ "# Scale features for neural network training\n", "print(\"\\nβš–οΈ Scaling features for neural network training...\")\n", "\n", "# Check if time-series data has been prepared\n", "if 'X_train_ts' not in locals() or 'X_test_ts' not in locals():\n", " print(\"πŸ”§ Time-series data not found. Using the prepared data from previous cells...\")\n", " \n", " # Use the filtered time-series data that was created\n", " X_train_ts = X_ts_filtered.iloc[:split_idx_ts]\n", " X_test_ts = X_ts_filtered.iloc[split_idx_ts:]\n", " \n", " print(f\"βœ… Time-series training data: {X_train_ts.shape}\")\n", " print(f\"βœ… Time-series test data: {X_test_ts.shape}\")\n", "\n", "print(\"πŸ” Checking data types and handling categorical variables...\")\n", "print(f\"πŸ“Š Training data types:\\n{X_train_ts.dtypes}\")\n", "\n", "# Handle categorical variables before scaling\n", "categorical_cols_ts = X_train_ts.select_dtypes(include=['object']).columns.tolist()\n", "numeric_cols_ts = X_train_ts.select_dtypes(include=[np.number]).columns.tolist()\n", "\n", "print(f\"πŸ“Š Categorical columns: {categorical_cols_ts}\")\n", "print(f\"πŸ“Š Numeric columns: {numeric_cols_ts}\")\n", "\n", "if categorical_cols_ts:\n", " print(\"πŸ”„ Encoding categorical variables for neural networks...\")\n", " \n", " # Use pandas get_dummies for one-hot encoding\n", " X_train_ts_encoded = pd.get_dummies(X_train_ts, columns=categorical_cols_ts, drop_first=True)\n", " X_test_ts_encoded = pd.get_dummies(X_test_ts, columns=categorical_cols_ts, drop_first=True)\n", " \n", " # Ensure both train and test have the same columns\n", " train_cols_ts = set(X_train_ts_encoded.columns)\n", " test_cols_ts = set(X_test_ts_encoded.columns)\n", " \n", " # Add missing columns to test set (fill with 0)\n", " missing_in_test_ts = train_cols_ts - test_cols_ts\n", " for col in missing_in_test_ts:\n", " X_test_ts_encoded[col] = 0\n", " \n", " # Add missing columns to train set (fill with 0)\n", " missing_in_train_ts = test_cols_ts - train_cols_ts\n", " for col in missing_in_train_ts:\n", " X_train_ts_encoded[col] = 0\n", " \n", " # Reorder columns to match\n", " X_test_ts_encoded = X_test_ts_encoded[X_train_ts_encoded.columns]\n", " \n", " print(f\"βœ… Encoded features: {X_train_ts_encoded.shape[1]} columns\")\n", " \n", " # Update the datasets\n", " X_train_ts = X_train_ts_encoded\n", " X_test_ts = X_test_ts_encoded\n", " \n", " # Update feature column list\n", " ts_feature_cols = list(X_train_ts.columns)\n", " \n", "else:\n", " print(\"βœ… No categorical variables found, proceeding with numeric scaling\")\n", "\n", "# Initialize MinMaxScaler (better for neural networks than StandardScaler)\n", "ts_scaler = MinMaxScaler()\n", " \n", "# Fit scaler ONLY on training data (prevent data leakage)\n", "print(\"πŸ”§ Applying MinMaxScaler to features...\")\n", "X_train_ts_scaled = ts_scaler.fit_transform(X_train_ts)\n", "X_test_ts_scaled = ts_scaler.transform(X_test_ts)\n", " \n", "print(f\"βœ… Features scaled to range [0, 1]\")\n", "print(f\"πŸ“Š Training set shape: {X_train_ts_scaled.shape}\")\n", "print(f\"πŸ“Š Test set shape: {X_test_ts_scaled.shape}\")\n", " \n", "# Convert back to DataFrames for easier handling\n", "X_train_ts_scaled = pd.DataFrame(X_train_ts_scaled, columns=ts_feature_cols)\n", "X_test_ts_scaled = pd.DataFrame(X_test_ts_scaled, columns=ts_feature_cols)\n", "\n", "# Display scaling summary for first few features\n", "print(\"\\nπŸ“ˆ Scaling Summary (first 5 features):\")\n", "scaling_summary = pd.DataFrame({\n", " 'Feature': ts_feature_cols[:5], # Show first 5 features\n", " 'Training Min': X_train_ts_scaled.iloc[:, :5].min(),\n", " 'Training Max': X_train_ts_scaled.iloc[:, :5].max(),\n", " 'Training Mean': X_train_ts_scaled.iloc[:, :5].mean()\n", "})\n", "\n", "display(scaling_summary.round(3))\n", "\n", "print(f\"βœ… Feature scaling complete!\")\n", "print(f\"πŸ“Š Total features after encoding: {len(ts_feature_cols)}\")\n", "print(f\"πŸ“Š Ready for sequence generation with shape: {X_train_ts_scaled.shape}\")\n" ] }, { "cell_type": "code", "execution_count": 30, "id": "9724b736", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "πŸ”„ Creating time-series sequences...\n", "πŸ“Š Sequence Configuration:\n", " β€’ Sequence Length: 14 days\n", " β€’ Batch Size: 32\n", " β€’ Input Shape: (batch_size, 14, 7)\n", "\n", "βœ… Sequence generators created:\n", " β€’ Training sequences: 6 batches\n", " β€’ Test sequences: 2 batches\n", " β€’ Total training samples: 192 (approx)\n", " β€’ Total test samples: 64 (approx)\n", "\n", "πŸ” Sequence Verification:\n", " β€’ Input batch shape: (32, 14, 7) (batch_size, timesteps, features)\n", " β€’ Target batch shape: (32,) (batch_size,)\n", " β€’ Features per timestep: 7\n", "\n", "πŸ“Š Effective sample counts (after sequence creation):\n", " β€’ Training: 183 sequences\n", " β€’ Test: 36 sequences\n", "\n", "🎯 Time-series sequences ready for neural network training!\n" ] } ], "source": [ "# Create time-series sequences for RNN training\n", "print(\"\\nπŸ”„ Creating time-series sequences...\")\n", "\n", "# Sequence parameters\n", "SEQUENCE_LENGTH = 14 # Use 14 days of history to predict next day\n", "BATCH_SIZE = 32\n", "\n", "print(f\"πŸ“Š Sequence Configuration:\")\n", "print(f\" β€’ Sequence Length: {SEQUENCE_LENGTH} days\")\n", "print(f\" β€’ Batch Size: {BATCH_SIZE}\")\n", "print(f\" β€’ Input Shape: (batch_size, {SEQUENCE_LENGTH}, {len(ts_feature_cols)})\")\n", "\n", "# Create TimeseriesGenerator for training data\n", "train_generator = TimeseriesGenerator(\n", " data=X_train_ts_scaled.values,\n", " targets=y_train_ts.values,\n", " length=SEQUENCE_LENGTH,\n", " batch_size=BATCH_SIZE,\n", " shuffle=False # Keep temporal order!\n", ")\n", "\n", "# Create TimeseriesGenerator for test data\n", "test_generator = TimeseriesGenerator(\n", " data=X_test_ts_scaled.values,\n", " targets=y_test_ts.values,\n", " length=SEQUENCE_LENGTH,\n", " batch_size=BATCH_SIZE,\n", " shuffle=False # Keep temporal order!\n", ")\n", "\n", "print(f\"\\nβœ… Sequence generators created:\")\n", "print(f\" β€’ Training sequences: {len(train_generator)} batches\")\n", "print(f\" β€’ Test sequences: {len(test_generator)} batches\")\n", "print(f\" β€’ Total training samples: {len(train_generator) * BATCH_SIZE} (approx)\")\n", "print(f\" β€’ Total test samples: {len(test_generator) * BATCH_SIZE} (approx)\")\n", "\n", "# Verify sequence shape\n", "sample_batch_x, sample_batch_y = train_generator[0]\n", "print(f\"\\nπŸ” Sequence Verification:\")\n", "print(f\" β€’ Input batch shape: {sample_batch_x.shape} (batch_size, timesteps, features)\")\n", "print(f\" β€’ Target batch shape: {sample_batch_y.shape} (batch_size,)\")\n", "print(f\" β€’ Features per timestep: {sample_batch_x.shape[2]}\")\n", "\n", "# Calculate effective sample counts for threshold optimization\n", "effective_train_samples = len(y_train_ts) - SEQUENCE_LENGTH + 1\n", "effective_test_samples = len(y_test_ts) - SEQUENCE_LENGTH + 1\n", "\n", "print(f\"\\nπŸ“Š Effective sample counts (after sequence creation):\")\n", "print(f\" β€’ Training: {effective_train_samples} sequences\")\n", "print(f\" β€’ Test: {effective_test_samples} sequences\")\n", "\n", "print(\"\\n🎯 Time-series sequences ready for neural network training!\")" ] }, { "cell_type": "markdown", "id": "befde7c0", "metadata": {}, "source": [ "## πŸ”— Section 4.2: LSTM Model Training & Evaluation\n", "\n", "### The LSTM Advantage\n", "\n", "**Long Short-Term Memory (LSTM)** networks are sophisticated neural architectures designed specifically for sequential data. Unlike traditional neural networks that treat each input independently, LSTMs can:\n", "\n", "🧠 **Remember Long-Term Patterns**: Maintain information across extended time periods\n", "πŸ”„ **Forget Irrelevant Information**: Selectively discard outdated patterns\n", "⚑ **Handle Variable Dependencies**: Learn both short and long-term relationships\n", "\n", "### LSTM Architecture for Pest Prediction\n", "\n", "Our LSTM will analyze **14-day sequences** of environmental data to predict pest occurrence on day 15. The model learns to identify complex temporal patterns like:\n", "- Seasonal pest cycles\n", "- Temperature-humidity interactions over time\n", "- Cumulative environmental stress indicators\n", "\n", "Let's train our first neural contestant! πŸš€\n", "\n" ] }, { "cell_type": "code", "execution_count": 31, "id": "4a2966e5", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "πŸ”— Training LSTM Model...\n", "==================================================\n", "πŸ—οΈ Building LSTM architecture...\n", "WARNING:tensorflow:From c:\\Users\\parha\\anaconda3\\envs\\pest_pred_specific\\lib\\site-packages\\keras\\src\\layers\\rnn\\lstm.py:148: The name tf.executing_eagerly_outside_functions is deprecated. Please use tf.compat.v1.executing_eagerly_outside_functions instead.\n", "\n", "βœ… LSTM model architecture:\n", "Model: \"sequential\"\n", "_________________________________________________________________\n", " Layer (type) Output Shape Param # \n", "=================================================================\n", " lstm (LSTM) (None, 64) 18432 \n", " \n", " dropout (Dropout) (None, 64) 0 \n", " \n", " dense (Dense) (None, 32) 2080 \n", " \n", " dropout_1 (Dropout) (None, 32) 0 \n", " \n", " dense_1 (Dense) (None, 1) 33 \n", " \n", "=================================================================\n", "Total params: 20545 (80.25 KB)\n", "Trainable params: 20545 (80.25 KB)\n", "Non-trainable params: 0 (0.00 Byte)\n", "_________________________________________________________________\n", "\n", "βš–οΈ Applying class weights from Part 1...\n", "πŸ“Š Class weights: {0: 0.532608695652174, 1: 8.166666666666666}\n", "\n", "🎯 Training LSTM model...\n", "πŸ“Š Monitoring validation AUC with early stopping (patience=15)\n", "Epoch 1/100\n", "WARNING:tensorflow:From c:\\Users\\parha\\anaconda3\\envs\\pest_pred_specific\\lib\\site-packages\\keras\\src\\utils\\tf_utils.py:492: The name tf.ragged.RaggedTensorValue is deprecated. Please use tf.compat.v1.ragged.RaggedTensorValue instead.\n", "\n", "WARNING:tensorflow:From c:\\Users\\parha\\anaconda3\\envs\\pest_pred_specific\\lib\\site-packages\\keras\\src\\engine\\base_layer_utils.py:384: The name tf.executing_eagerly_outside_functions is deprecated. Please use tf.compat.v1.executing_eagerly_outside_functions instead.\n", "\n", "6/6 [==============================] - 3s 91ms/step - loss: 0.7185 - accuracy: 0.7637 - auc: 0.5390 - val_loss: 0.6605 - val_accuracy: 0.7429 - val_auc: 0.5940\n", "Epoch 2/100\n", "6/6 [==============================] - 0s 15ms/step - loss: 0.7275 - accuracy: 0.8846 - auc: 0.4436 - val_loss: 0.6688 - val_accuracy: 0.7429 - val_auc: 0.6239\n", "Epoch 3/100\n", "6/6 [==============================] - 0s 15ms/step - loss: 0.7166 - accuracy: 0.8132 - auc: 0.5772 - val_loss: 0.6801 - val_accuracy: 0.7714 - val_auc: 0.6197\n", "Epoch 4/100\n", "6/6 [==============================] - 0s 15ms/step - loss: 0.7217 - accuracy: 0.6538 - auc: 0.4750 - val_loss: 0.6956 - val_accuracy: 0.4571 - val_auc: 0.5406\n", "Epoch 5/100\n", "6/6 [==============================] - 0s 15ms/step - loss: 0.7173 - accuracy: 0.4780 - auc: 0.5211 - val_loss: 0.7027 - val_accuracy: 0.3429 - val_auc: 0.5812\n", "Epoch 6/100\n", "6/6 [==============================] - 0s 15ms/step - loss: 0.7238 - accuracy: 0.3626 - auc: 0.4331 - val_loss: 0.7049 - val_accuracy: 0.3429 - val_auc: 0.5791\n", "Epoch 7/100\n", "6/6 [==============================] - 0s 15ms/step - loss: 0.7141 - accuracy: 0.3187 - auc: 0.5723 - val_loss: 0.7138 - val_accuracy: 0.2571 - val_auc: 0.5876\n", "Epoch 8/100\n", "6/6 [==============================] - 0s 15ms/step - loss: 0.7147 - accuracy: 0.2363 - auc: 0.5390 - val_loss: 0.7155 - val_accuracy: 0.2571 - val_auc: 0.5833\n", "Epoch 9/100\n", "6/6 [==============================] - 0s 15ms/step - loss: 0.7066 - accuracy: 0.2802 - auc: 0.5963 - val_loss: 0.7125 - val_accuracy: 0.2571 - val_auc: 0.6047\n", "Epoch 10/100\n", "6/6 [==============================] - 0s 15ms/step - loss: 0.7212 - accuracy: 0.2802 - auc: 0.4762 - val_loss: 0.7142 - val_accuracy: 0.2571 - val_auc: 0.5919\n", "Epoch 11/100\n", "6/6 [==============================] - 0s 15ms/step - loss: 0.7078 - accuracy: 0.3462 - auc: 0.6196 - val_loss: 0.6940 - val_accuracy: 0.6286 - val_auc: 0.5983\n", "Epoch 12/100\n", "6/6 [==============================] - 0s 15ms/step - loss: 0.7167 - accuracy: 0.5604 - auc: 0.5034 - val_loss: 0.6785 - val_accuracy: 0.7429 - val_auc: 0.5705\n", "Epoch 13/100\n", "6/6 [==============================] - 0s 15ms/step - loss: 0.7096 - accuracy: 0.7253 - auc: 0.6657 - val_loss: 0.6748 - val_accuracy: 0.7429 - val_auc: 0.5641\n", "Epoch 14/100\n", "6/6 [==============================] - 0s 16ms/step - loss: 0.7010 - accuracy: 0.6703 - auc: 0.6787 - val_loss: 0.6859 - val_accuracy: 0.6857 - val_auc: 0.5577\n", "Epoch 15/100\n", "6/6 [==============================] - 0s 16ms/step - loss: 0.7134 - accuracy: 0.4725 - auc: 0.5502 - val_loss: 0.7005 - val_accuracy: 0.4286 - val_auc: 0.5662\n", "Epoch 16/100\n", "6/6 [==============================] - 0s 14ms/step - loss: 0.7108 - accuracy: 0.4780 - auc: 0.5676 - val_loss: 0.6852 - val_accuracy: 0.6857 - val_auc: 0.5620\n", "Epoch 17/100\n", "1/6 [====>.........................] - ETA: 0s - loss: 0.5282 - accuracy: 0.6250 - auc: 0.4516Restoring model weights from the end of the best epoch: 2.\n", "6/6 [==============================] - 0s 16ms/step - loss: 0.6939 - accuracy: 0.5989 - auc: 0.6907 - val_loss: 0.6731 - val_accuracy: 0.7429 - val_auc: 0.5513\n", "Epoch 17: early stopping\n", "\n", "βœ… LSTM training complete!\n", "πŸ“Š Training stopped at epoch: 17\n", "πŸ† Best validation AUC: 0.6239\n" ] } ], "source": [ "print(\"πŸ”— Training LSTM Model...\")\n", "print(\"=\" * 50)\n", "\n", "# Build LSTM model\n", "print(\"πŸ—οΈ Building LSTM architecture...\")\n", "\n", "lstm_model = Sequential([\n", " LSTM(64, \n", " input_shape=(SEQUENCE_LENGTH, len(ts_feature_cols)),\n", " return_sequences=False,\n", " dropout=0.2,\n", " recurrent_dropout=0.2),\n", " Dropout(0.3),\n", " Dense(32, activation='relu'),\n", " Dropout(0.2),\n", " Dense(1, activation='sigmoid')\n", "])\n", "\n", "# Compile the model\n", "lstm_model.compile(\n", " optimizer=Adam(learning_rate=0.001),\n", " loss='binary_crossentropy',\n", " metrics=['accuracy', keras_metrics.AUC(name='auc')]\n", ")\n", "\n", "print(\"βœ… LSTM model architecture:\")\n", "lstm_model.summary()\n", "\n", "# Prepare class weights for LSTM (convert to TensorFlow format)\n", "print(\"\\nβš–οΈ Applying class weights from Part 1...\")\n", "\n", "# Use the same class weights calculated in Part 1\n", "class_weights_tf = {0: class_weights[0], 1: class_weights[1]}\n", "print(f\"πŸ“Š Class weights: {class_weights_tf}\")\n", "\n", "# Setup callbacks\n", "early_stopping = EarlyStopping(\n", " monitor='val_auc',\n", " mode='max',\n", " patience=15,\n", " restore_best_weights=True,\n", " verbose=1\n", ")\n", "\n", "print(\"\\n🎯 Training LSTM model...\")\n", "print(\"πŸ“Š Monitoring validation AUC with early stopping (patience=15)\")\n", "\n", "# Train the model\n", "history_lstm = lstm_model.fit(\n", " train_generator,\n", " epochs=100,\n", " validation_data=test_generator,\n", " class_weight=class_weights_tf,\n", " callbacks=[early_stopping],\n", " verbose=1\n", ")\n", "\n", "print(\"\\nβœ… LSTM training complete!\")\n", "print(f\"πŸ“Š Training stopped at epoch: {len(history_lstm.history['loss'])}\")\n", "print(f\"πŸ† Best validation AUC: {max(history_lstm.history['val_auc']):.4f}\")" ] }, { "cell_type": "code", "execution_count": 32, "id": "b4141002", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "πŸ”§ Finding optimal threshold for LSTM...\n", "πŸ“Š Prediction array shape: (35,)\n", "πŸ“Š Actual targets shape: (35,)\n", "πŸ“Š Sample predictions: [0.4628131 0.4611399 0.46732923 0.46389523 0.46815586]\n", "πŸ” Optimizing prediction threshold...\n", "🎯 Optimal threshold: 0.480\n", "πŸ† Max F1-score: 0.4545\n", "\n", "πŸ“Š LSTM Final Test Results:\n", " 🎯 F1-Score: 0.4545\n", " πŸ“ˆ AUC: 0.6806\n", " 🎯 Accuracy: 0.6571\n", "\n", "πŸ“‹ LSTM Classification Report:\n", "============================================================\n", " precision recall f1-score support\n", "\n", " No Catch 0.86 0.67 0.75 27\n", " Catch 0.36 0.62 0.45 8\n", "\n", " accuracy 0.66 35\n", " macro avg 0.61 0.65 0.60 35\n", "weighted avg 0.74 0.66 0.68 35\n", "\n", "\n", "βœ… LSTM evaluation complete and results stored!\n" ] } ], "source": [ "# Get LSTM predictions for threshold optimization\n", "print(\"\\nπŸ”§ Finding optimal threshold for LSTM...\")\n", "\n", "# Get predictions on test set\n", "y_test_proba_lstm = lstm_model.predict(test_generator, verbose=0)\n", "y_test_proba_lstm = y_test_proba_lstm.flatten() # Convert to 1D array\n", "\n", "# Get actual test targets (accounting for sequence generation)\n", "y_test_actual_lstm = y_test_ts.iloc[SEQUENCE_LENGTH-1:].values # Align with sequences\n", "\n", "# Ensure we have the right number of predictions\n", "min_len = min(len(y_test_proba_lstm), len(y_test_actual_lstm))\n", "y_test_proba_lstm = y_test_proba_lstm[:min_len]\n", "y_test_actual_lstm = y_test_actual_lstm[:min_len]\n", "\n", "print(f\"πŸ“Š Prediction array shape: {y_test_proba_lstm.shape}\")\n", "print(f\"πŸ“Š Actual targets shape: {y_test_actual_lstm.shape}\")\n", "print(f\"πŸ“Š Sample predictions: {y_test_proba_lstm[:5]}\")\n", "\n", "# Find optimal threshold using the same logic as Part 1\n", "thresholds = np.arange(0.1, 0.9, 0.01)\n", "f1_scores_lstm = []\n", "\n", "print(\"πŸ” Optimizing prediction threshold...\")\n", "\n", "for threshold in thresholds:\n", " y_pred_thresh = (y_test_proba_lstm >= threshold).astype(int)\n", " if len(np.unique(y_pred_thresh)) > 1: # Avoid division by zero\n", " f1 = f1_score(y_test_actual_lstm, y_pred_thresh)\n", " else:\n", " f1 = 0.0\n", " f1_scores_lstm.append(f1)\n", "\n", "optimal_threshold_lstm = thresholds[np.argmax(f1_scores_lstm)]\n", "max_f1_lstm = max(f1_scores_lstm)\n", "\n", "print(f\"🎯 Optimal threshold: {optimal_threshold_lstm:.3f}\")\n", "print(f\"πŸ† Max F1-score: {max_f1_lstm:.4f}\")\n", "\n", "# Make final predictions with optimal threshold\n", "y_test_pred_lstm = (y_test_proba_lstm >= optimal_threshold_lstm).astype(int)\n", "\n", "# Calculate final metrics\n", "lstm_test_f1 = f1_score(y_test_actual_lstm, y_test_pred_lstm)\n", "lstm_test_auc = roc_auc_score(y_test_actual_lstm, y_test_proba_lstm)\n", "lstm_test_accuracy = accuracy_score(y_test_actual_lstm, y_test_pred_lstm)\n", "\n", "print(f\"\\nπŸ“Š LSTM Final Test Results:\")\n", "print(f\" 🎯 F1-Score: {lstm_test_f1:.4f}\")\n", "print(f\" πŸ“ˆ AUC: {lstm_test_auc:.4f}\")\n", "print(f\" 🎯 Accuracy: {lstm_test_accuracy:.4f}\")\n", "\n", "# Detailed classification report\n", "print(f\"\\nπŸ“‹ LSTM Classification Report:\")\n", "print(\"=\" * 60)\n", "print(classification_report(y_test_actual_lstm, y_test_pred_lstm, target_names=['No Catch', 'Catch']))\n", "\n", "# Store results in the same format as Part 1\n", "lstm_results = {\n", " 'model': lstm_model,\n", " 'threshold': optimal_threshold_lstm,\n", " 'test_f1': lstm_test_f1,\n", " 'test_auc': lstm_test_auc,\n", " 'test_accuracy': lstm_test_accuracy,\n", " 'y_pred': y_test_pred_lstm,\n", " 'y_proba': y_test_proba_lstm,\n", " 'y_actual': y_test_actual_lstm,\n", " 'history': history_lstm.history\n", "}\n", "\n", "print(\"\\nβœ… LSTM evaluation complete and results stored!\")" ] }, { "cell_type": "code", "execution_count": 33, "id": "1a3d546c", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "πŸ“Š Creating LSTM performance visualizations...\n" ] }, { "data": { "application/vnd.plotly.v1+json": { "config": { "plotlyServerURL": "https://plot.ly" }, "data": [ { "colorscale": [ [ 0, "rgb(247,251,255)" ], [ 0.125, "rgb(222,235,247)" ], [ 0.25, "rgb(198,219,239)" ], [ 0.375, "rgb(158,202,225)" ], [ 0.5, "rgb(107,174,214)" ], [ 0.625, "rgb(66,146,198)" ], [ 0.75, "rgb(33,113,181)" ], [ 0.875, "rgb(8,81,156)" ], [ 1, "rgb(8,48,107)" ] ], "reversescale": false, "showscale": true, "type": "heatmap", "x": [ "Predicted: No Catch", "Predicted: Catch" ], "y": [ "Actual: No Catch", "Actual: Catch" ], "z": [ [ 18, 9 ], [ 3, 5 ] ] } ], "layout": { "annotations": [ { "font": { "color": "#FFFFFF" }, "showarrow": false, "text": "18", "x": "Predicted: No Catch", "xref": "x", "y": "Actual: No Catch", "yref": "y" }, { "font": { "color": "#000000" }, "showarrow": false, "text": "9", "x": "Predicted: Catch", "xref": "x", "y": "Actual: No Catch", "yref": "y" }, { "font": { "color": "#000000" }, "showarrow": false, "text": "3", "x": "Predicted: No Catch", "xref": "x", "y": "Actual: Catch", "yref": "y" }, { "font": { "color": "#000000" }, "showarrow": false, "text": "5", "x": "Predicted: Catch", "xref": "x", "y": "Actual: Catch", "yref": "y" } ], "height": 400, "template": { "data": { "bar": [ { "error_x": { "color": "#2a3f5f" }, "error_y": { "color": "#2a3f5f" }, "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "bar" } ], "barpolar": [ { "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "barpolar" } ], "carpet": [ { "aaxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "baxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "type": "carpet" } ], "choropleth": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "choropleth" } ], "contour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "contour" } ], "contourcarpet": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "contourcarpet" } ], "heatmap": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "heatmap" } ], "heatmapgl": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "heatmapgl" } ], "histogram": [ { "marker": { "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "histogram" } ], "histogram2d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "histogram2d" } ], "histogram2dcontour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "histogram2dcontour" } ], "mesh3d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "mesh3d" } ], "parcoords": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "parcoords" } ], "pie": [ { "automargin": true, "type": "pie" } ], "scatter": [ { "fillpattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 }, "type": "scatter" } ], "scatter3d": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatter3d" } ], "scattercarpet": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattercarpet" } ], "scattergeo": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergeo" } ], "scattergl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergl" } ], "scattermapbox": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattermapbox" } ], "scatterpolar": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolar" } ], "scatterpolargl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolargl" } ], "scatterternary": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterternary" } ], "surface": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "surface" } ], "table": [ { "cells": { "fill": { "color": "#EBF0F8" }, "line": { "color": "white" } }, "header": { "fill": { "color": "#C8D4E3" }, "line": { "color": "white" } }, "type": "table" } ] }, "layout": { "annotationdefaults": { "arrowcolor": "#2a3f5f", "arrowhead": 0, "arrowwidth": 1 }, "autotypenumbers": "strict", "coloraxis": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "colorscale": { "diverging": [ [ 0, "#8e0152" ], [ 0.1, "#c51b7d" ], [ 0.2, "#de77ae" ], [ 0.3, "#f1b6da" ], [ 0.4, "#fde0ef" ], [ 0.5, "#f7f7f7" ], [ 0.6, "#e6f5d0" ], [ 0.7, "#b8e186" ], [ 0.8, "#7fbc41" ], [ 0.9, "#4d9221" ], [ 1, "#276419" ] ], "sequential": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "sequentialminus": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ] }, "colorway": [ "#636efa", "#EF553B", "#00cc96", "#ab63fa", "#FFA15A", "#19d3f3", "#FF6692", "#B6E880", "#FF97FF", "#FECB52" ], "font": { "color": "#2a3f5f" }, "geo": { "bgcolor": "white", "lakecolor": "white", "landcolor": "#E5ECF6", "showlakes": true, "showland": true, "subunitcolor": "white" }, "hoverlabel": { "align": "left" }, "hovermode": "closest", "mapbox": { "style": "light" }, "paper_bgcolor": "white", "plot_bgcolor": "#E5ECF6", "polar": { "angularaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "radialaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "scene": { "xaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "yaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "zaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" } }, "shapedefaults": { "line": { "color": "#2a3f5f" } }, "ternary": { "aaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "baxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "caxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "title": { "x": 0.05 }, "xaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 }, "yaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 } } }, "title": { "text": "πŸ”— LSTM Confusion Matrix", "x": 0.5 }, "width": 500, "xaxis": { "dtick": 1, "gridcolor": "rgb(0, 0, 0)", "side": "top", "ticks": "" }, "yaxis": { "dtick": 1, "ticks": "", "ticksuffix": " " } } } }, "metadata": {}, "output_type": "display_data" }, { "data": { "application/vnd.plotly.v1+json": { "config": { "plotlyServerURL": "https://plot.ly" }, "data": [ { "line": { "color": "blue", "width": 3 }, "mode": "lines", "name": "LSTM (AUC = 0.681)", "type": "scatter", "x": [ 0, 0.037037037037037035, 0.07407407407407407, 0.07407407407407407, 0.1111111111111111, 0.1111111111111111, 0.2222222222222222, 0.2222222222222222, 0.37037037037037035, 0.37037037037037035, 0.5555555555555556, 0.5555555555555556, 0.8888888888888888, 0.8888888888888888, 1 ], "y": [ 0, 0, 0, 0.125, 0.125, 0.375, 0.375, 0.625, 0.625, 0.75, 0.75, 0.875, 0.875, 1, 1 ] }, { "line": { "color": "red", "dash": "dash", "width": 2 }, "mode": "lines", "name": "Random Classifier", "type": "scatter", "x": [ 0, 1 ], "y": [ 0, 1 ] } ], "layout": { "height": 500, "template": { "data": { "bar": [ { "error_x": { "color": "#2a3f5f" }, "error_y": { "color": "#2a3f5f" }, "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "bar" } ], "barpolar": [ { "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "barpolar" } ], "carpet": [ { "aaxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "baxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "type": "carpet" } ], "choropleth": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "choropleth" } ], "contour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "contour" } ], "contourcarpet": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "contourcarpet" } ], "heatmap": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "heatmap" } ], "heatmapgl": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "heatmapgl" } ], "histogram": [ { "marker": { "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "histogram" } ], "histogram2d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "histogram2d" } ], "histogram2dcontour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "histogram2dcontour" } ], "mesh3d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "mesh3d" } ], "parcoords": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "parcoords" } ], "pie": [ { "automargin": true, "type": "pie" } ], "scatter": [ { "fillpattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 }, "type": "scatter" } ], "scatter3d": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatter3d" } ], "scattercarpet": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattercarpet" } ], "scattergeo": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergeo" } ], "scattergl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergl" } ], "scattermapbox": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattermapbox" } ], "scatterpolar": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolar" } ], "scatterpolargl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolargl" } ], "scatterternary": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterternary" } ], "surface": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "surface" } ], "table": [ { "cells": { "fill": { "color": "#EBF0F8" }, "line": { "color": "white" } }, "header": { "fill": { "color": "#C8D4E3" }, "line": { "color": "white" } }, "type": "table" } ] }, "layout": { "annotationdefaults": { "arrowcolor": "#2a3f5f", "arrowhead": 0, "arrowwidth": 1 }, "autotypenumbers": "strict", "coloraxis": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "colorscale": { "diverging": [ [ 0, "#8e0152" ], [ 0.1, "#c51b7d" ], [ 0.2, "#de77ae" ], [ 0.3, "#f1b6da" ], [ 0.4, "#fde0ef" ], [ 0.5, "#f7f7f7" ], [ 0.6, "#e6f5d0" ], [ 0.7, "#b8e186" ], [ 0.8, "#7fbc41" ], [ 0.9, "#4d9221" ], [ 1, "#276419" ] ], "sequential": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "sequentialminus": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ] }, "colorway": [ "#636efa", "#EF553B", "#00cc96", "#ab63fa", "#FFA15A", "#19d3f3", "#FF6692", "#B6E880", "#FF97FF", "#FECB52" ], "font": { "color": "#2a3f5f" }, "geo": { "bgcolor": "white", "lakecolor": "white", "landcolor": "#E5ECF6", "showlakes": true, "showland": true, "subunitcolor": "white" }, "hoverlabel": { "align": "left" }, "hovermode": "closest", "mapbox": { "style": "light" }, "paper_bgcolor": "white", "plot_bgcolor": "#E5ECF6", "polar": { "angularaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "radialaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "scene": { "xaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "yaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "zaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" } }, "shapedefaults": { "line": { "color": "#2a3f5f" } }, "ternary": { "aaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "baxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "caxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "title": { "x": 0.05 }, "xaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 }, "yaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 } } }, "title": { "text": "πŸ”— LSTM ROC Curve", "x": 0.5 }, "width": 600, "xaxis": { "title": { "text": "False Positive Rate" } }, "yaxis": { "title": { "text": "True Positive Rate" } } } } }, "metadata": {}, "output_type": "display_data" }, { "data": { "application/vnd.plotly.v1+json": { "config": { "plotlyServerURL": "https://plot.ly" }, "data": [ { "line": { "color": "blue" }, "mode": "lines", "name": "Training Loss", "type": "scatter", "x": [ 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17 ], "xaxis": "x", "y": [ 0.7185291647911072, 0.7275193333625793, 0.7165956497192383, 0.7216907739639282, 0.7172648906707764, 0.7237884402275085, 0.7141068577766418, 0.7147270441055298, 0.7065937519073486, 0.7211983799934387, 0.7078174948692322, 0.7166675925254822, 0.7095866799354553, 0.7010291218757629, 0.7133822441101074, 0.7107512950897217, 0.6938541531562805 ], "yaxis": "y" }, { "line": { "color": "green" }, "mode": "lines", "name": "Training AUC", "type": "scatter", "x": [ 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17 ], "xaxis": "x2", "y": [ 0.5389705896377563, 0.4436274766921997, 0.5772058963775635, 0.4749999940395355, 0.5210784077644348, 0.4330882430076599, 0.5723038911819458, 0.5389705896377563, 0.5963234901428223, 0.47622549533843994, 0.6196078658103943, 0.5034313797950745, 0.6656862497329712, 0.6786764860153198, 0.5502451062202454, 0.5676470398902893, 0.6906862854957581 ], "yaxis": "y2" }, { "line": { "color": "red" }, "mode": "lines", "name": "Validation Loss", "type": "scatter", "x": [ 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17 ], "xaxis": "x3", "y": [ 0.6604641675949097, 0.6687729954719543, 0.6800529360771179, 0.6955600380897522, 0.7026540040969849, 0.704870879650116, 0.713768720626831, 0.7155359983444214, 0.7125289440155029, 0.7141676545143127, 0.693976879119873, 0.6785144209861755, 0.674849271774292, 0.6858676075935364, 0.7005323767662048, 0.6852073073387146, 0.6730513572692871 ], "yaxis": "y3" }, { "line": { "color": "orange" }, "mode": "lines", "name": "Validation AUC", "type": "scatter", "x": [ 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17 ], "xaxis": "x4", "y": [ 0.5940170884132385, 0.6239316463470459, 0.6196581125259399, 0.5405982732772827, 0.5811966061592102, 0.5790598392486572, 0.5876068472862244, 0.5833333730697632, 0.6047008633613586, 0.5918803215026855, 0.5982905626296997, 0.5705128312110901, 0.5641025900840759, 0.557692289352417, 0.5662393569946289, 0.561965823173523, 0.5512820482254028 ], "yaxis": "y4" } ], "layout": { "annotations": [ { "font": { "size": 16 }, "showarrow": false, "text": "Training Loss", "x": 0.225, "xanchor": "center", "xref": "paper", "y": 1, "yanchor": "bottom", "yref": "paper" }, { "font": { "size": 16 }, "showarrow": false, "text": "Training AUC", "x": 0.775, "xanchor": "center", "xref": "paper", "y": 1, "yanchor": "bottom", "yref": "paper" }, { "font": { "size": 16 }, "showarrow": false, "text": "Validation Loss", "x": 0.225, "xanchor": "center", "xref": "paper", "y": 0.375, "yanchor": "bottom", "yref": "paper" }, { "font": { "size": 16 }, "showarrow": false, "text": "Validation AUC", "x": 0.775, "xanchor": "center", "xref": "paper", "y": 0.375, "yanchor": "bottom", "yref": "paper" } ], "height": 600, "showlegend": false, "template": { "data": { "bar": [ { "error_x": { "color": "#2a3f5f" }, "error_y": { "color": "#2a3f5f" }, "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "bar" } ], "barpolar": [ { "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "barpolar" } ], "carpet": [ { "aaxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "baxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "type": "carpet" } ], "choropleth": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "choropleth" } ], "contour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "contour" } ], "contourcarpet": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "contourcarpet" } ], "heatmap": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "heatmap" } ], "heatmapgl": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "heatmapgl" } ], "histogram": [ { "marker": { "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "histogram" } ], "histogram2d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "histogram2d" } ], "histogram2dcontour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "histogram2dcontour" } ], "mesh3d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "mesh3d" } ], "parcoords": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "parcoords" } ], "pie": [ { "automargin": true, "type": "pie" } ], "scatter": [ { "fillpattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 }, "type": "scatter" } ], "scatter3d": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatter3d" } ], "scattercarpet": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattercarpet" } ], "scattergeo": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergeo" } ], "scattergl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergl" } ], "scattermapbox": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattermapbox" } ], "scatterpolar": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolar" } ], "scatterpolargl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolargl" } ], "scatterternary": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterternary" } ], "surface": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "surface" } ], "table": [ { "cells": { "fill": { "color": "#EBF0F8" }, "line": { "color": "white" } }, "header": { "fill": { "color": "#C8D4E3" }, "line": { "color": "white" } }, "type": "table" } ] }, "layout": { "annotationdefaults": { "arrowcolor": "#2a3f5f", "arrowhead": 0, "arrowwidth": 1 }, "autotypenumbers": "strict", "coloraxis": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "colorscale": { "diverging": [ [ 0, "#8e0152" ], [ 0.1, "#c51b7d" ], [ 0.2, "#de77ae" ], [ 0.3, "#f1b6da" ], [ 0.4, "#fde0ef" ], [ 0.5, "#f7f7f7" ], [ 0.6, "#e6f5d0" ], [ 0.7, "#b8e186" ], [ 0.8, "#7fbc41" ], [ 0.9, "#4d9221" ], [ 1, "#276419" ] ], "sequential": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "sequentialminus": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ] }, "colorway": [ "#636efa", "#EF553B", "#00cc96", "#ab63fa", "#FFA15A", "#19d3f3", "#FF6692", "#B6E880", "#FF97FF", "#FECB52" ], "font": { "color": "#2a3f5f" }, "geo": { "bgcolor": "white", "lakecolor": "white", "landcolor": "#E5ECF6", "showlakes": true, "showland": true, "subunitcolor": "white" }, "hoverlabel": { "align": "left" }, "hovermode": "closest", "mapbox": { "style": "light" }, "paper_bgcolor": "white", "plot_bgcolor": "#E5ECF6", "polar": { "angularaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "radialaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "scene": { "xaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "yaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "zaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" } }, "shapedefaults": { "line": { "color": "#2a3f5f" } }, "ternary": { "aaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "baxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "caxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "title": { "x": 0.05 }, "xaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 }, "yaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 } } }, "title": { "text": "πŸ”— LSTM Training History", "x": 0.5 }, "xaxis": { "anchor": "y", "domain": [ 0, 0.45 ] }, "xaxis2": { "anchor": "y2", "domain": [ 0.55, 1 ] }, "xaxis3": { "anchor": "y3", "domain": [ 0, 0.45 ] }, "xaxis4": { "anchor": "y4", "domain": [ 0.55, 1 ] }, "yaxis": { "anchor": "x", "domain": [ 0.625, 1 ] }, "yaxis2": { "anchor": "x2", "domain": [ 0.625, 1 ] }, "yaxis3": { "anchor": "x3", "domain": [ 0, 0.375 ] }, "yaxis4": { "anchor": "x4", "domain": [ 0, 0.375 ] } } } }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "\n", "πŸ† LSTM Tournament Summary:\n", " β€’ F1-Score: 0.4545\n", " β€’ AUC: 0.6806\n", " β€’ Optimal Threshold: 0.480\n", " β€’ Test Accuracy: 0.6571\n", " β€’ Training Epochs: 17\n", " β€’ Best Val AUC: 0.6239\n" ] } ], "source": [ "# Create LSTM performance visualizations\n", "print(\"\\nπŸ“Š Creating LSTM performance visualizations...\")\n", "\n", "# Interactive confusion matrix\n", "cm_lstm = confusion_matrix(y_test_actual_lstm, y_test_pred_lstm)\n", "\n", "fig_cm_lstm = ff.create_annotated_heatmap(\n", " z=cm_lstm,\n", " x=['Predicted: No Catch', 'Predicted: Catch'],\n", " y=['Actual: No Catch', 'Actual: Catch'],\n", " annotation_text=cm_lstm,\n", " colorscale='Blues',\n", " showscale=True\n", ")\n", "\n", "fig_cm_lstm.update_layout(\n", " title='πŸ”— LSTM Confusion Matrix',\n", " title_x=0.5,\n", " width=500,\n", " height=400\n", ")\n", "\n", "fig_cm_lstm.show()\n", "\n", "# Interactive ROC curve\n", "fpr_lstm, tpr_lstm, _ = roc_curve(y_test_actual_lstm, y_test_proba_lstm)\n", "\n", "fig_roc_lstm = go.Figure()\n", "\n", "fig_roc_lstm.add_trace(go.Scatter(\n", " x=fpr_lstm,\n", " y=tpr_lstm,\n", " mode='lines',\n", " name=f'LSTM (AUC = {lstm_test_auc:.3f})',\n", " line=dict(color='blue', width=3)\n", "))\n", "\n", "# Add diagonal reference line\n", "fig_roc_lstm.add_trace(go.Scatter(\n", " x=[0, 1],\n", " y=[0, 1],\n", " mode='lines',\n", " name='Random Classifier',\n", " line=dict(color='red', width=2, dash='dash')\n", "))\n", "\n", "fig_roc_lstm.update_layout(\n", " title='πŸ”— LSTM ROC Curve',\n", " title_x=0.5,\n", " xaxis_title='False Positive Rate',\n", " yaxis_title='True Positive Rate',\n", " width=600,\n", " height=500\n", ")\n", "\n", "fig_roc_lstm.show()\n", "\n", "# Training history visualization\n", "fig_history = make_subplots(\n", " rows=2, cols=2,\n", " subplot_titles=['Training Loss', 'Training AUC', 'Validation Loss', 'Validation AUC'],\n", " specs=[[{\"type\": \"scatter\"}, {\"type\": \"scatter\"}], \n", " [{\"type\": \"scatter\"}, {\"type\": \"scatter\"}]]\n", ")\n", "\n", "epochs = range(1, len(history_lstm.history['loss']) + 1)\n", "\n", "# Training Loss\n", "fig_history.add_trace(\n", " go.Scatter(x=list(epochs), y=history_lstm.history['loss'], \n", " mode='lines', name='Training Loss', line=dict(color='blue')),\n", " row=1, col=1\n", ")\n", "\n", "# Training AUC\n", "fig_history.add_trace(\n", " go.Scatter(x=list(epochs), y=history_lstm.history['auc'], \n", " mode='lines', name='Training AUC', line=dict(color='green')),\n", " row=1, col=2\n", ")\n", "\n", "# Validation Loss\n", "fig_history.add_trace(\n", " go.Scatter(x=list(epochs), y=history_lstm.history['val_loss'], \n", " mode='lines', name='Validation Loss', line=dict(color='red')),\n", " row=2, col=1\n", ")\n", "\n", "# Validation AUC\n", "fig_history.add_trace(\n", " go.Scatter(x=list(epochs), y=history_lstm.history['val_auc'], \n", " mode='lines', name='Validation AUC', line=dict(color='orange')),\n", " row=2, col=2\n", ")\n", "\n", "fig_history.update_layout(\n", " title='πŸ”— LSTM Training History',\n", " title_x=0.5,\n", " height=600,\n", " showlegend=False\n", ")\n", "\n", "fig_history.show()\n", "\n", "print(f\"\\nπŸ† LSTM Tournament Summary:\")\n", "print(f\" β€’ F1-Score: {lstm_test_f1:.4f}\")\n", "print(f\" β€’ AUC: {lstm_test_auc:.4f}\")\n", "print(f\" β€’ Optimal Threshold: {optimal_threshold_lstm:.3f}\")\n", "print(f\" β€’ Test Accuracy: {lstm_test_accuracy:.4f}\")\n", "print(f\" β€’ Training Epochs: {len(history_lstm.history['loss'])}\")\n", "print(f\" β€’ Best Val AUC: {max(history_lstm.history['val_auc']):.4f}\")" ] }, { "cell_type": "markdown", "id": "18758297", "metadata": {}, "source": [ "## ⚑ Section 4.3: GRU Model Training & Evaluation\n", "\n", "### The GRU Innovation\n", "\n", "**Gated Recurrent Unit (GRU)** represents a streamlined evolution of the LSTM architecture. While maintaining the core ability to capture long-term dependencies, GRUs offer several advantages:\n", "\n", "⚑ **Computational Efficiency**: Fewer parameters mean faster training\n", "🎯 **Simplified Architecture**: Only 2 gates vs LSTM's 3 gates\n", "πŸ“ˆ **Comparable Performance**: Often matches LSTM results with less complexity\n", "πŸ”„ **Better Gradient Flow**: Reduced vanishing gradient problems\n", "\n", "### GRU vs LSTM Architecture\n", "\n", "**LSTM Gates**: Forget Gate + Input Gate + Output Gate\n", "**GRU Gates**: Reset Gate + Update Gate (simplified but effective)\n", "\n", "For time-series pest prediction, GRUs might excel due to:\n", "- Faster convergence on seasonal patterns\n", "- Efficient processing of environmental sequences\n", "- Better handling of irregular pest occurrence patterns\n", "\n", "Let's see if our streamlined neural competitor can outperform the LSTM! ⚑" ] }, { "cell_type": "code", "execution_count": 34, "id": "4c687a9e", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "⚑ Training GRU Model...\n", "==================================================\n", "πŸ—οΈ Building GRU architecture...\n", "βœ… GRU model architecture:\n", "Model: \"sequential_1\"\n", "_________________________________________________________________\n", " Layer (type) Output Shape Param # \n", "=================================================================\n", " gru (GRU) (None, 64) 14016 \n", " \n", " dropout_2 (Dropout) (None, 64) 0 \n", " \n", " dense_2 (Dense) (None, 32) 2080 \n", " \n", " dropout_3 (Dropout) (None, 32) 0 \n", " \n", " dense_3 (Dense) (None, 1) 33 \n", " \n", "=================================================================\n", "Total params: 16129 (63.00 KB)\n", "Trainable params: 16129 (63.00 KB)\n", "Non-trainable params: 0 (0.00 Byte)\n", "_________________________________________________________________\n", "\n", "🎯 Training GRU model...\n", "βš–οΈ Using same class weights as LSTM for fair comparison\n", "πŸ“Š Monitoring validation AUC with early stopping (patience=15)\n", "Epoch 1/100\n", "6/6 [==============================] - 3s 84ms/step - loss: 0.7299 - accuracy: 0.7088 - auc: 0.4140 - val_loss: 0.6648 - val_accuracy: 0.7429 - val_auc: 0.6175\n", "Epoch 2/100\n", "6/6 [==============================] - 0s 14ms/step - loss: 0.7195 - accuracy: 0.8352 - auc: 0.5400 - val_loss: 0.6633 - val_accuracy: 0.7429 - val_auc: 0.6047\n", "Epoch 3/100\n", "6/6 [==============================] - 0s 15ms/step - loss: 0.7181 - accuracy: 0.8077 - auc: 0.5466 - val_loss: 0.6637 - val_accuracy: 0.7429 - val_auc: 0.5791\n", "Epoch 4/100\n", "6/6 [==============================] - 0s 15ms/step - loss: 0.7210 - accuracy: 0.8462 - auc: 0.5333 - val_loss: 0.6691 - val_accuracy: 0.7143 - val_auc: 0.5748\n", "Epoch 5/100\n", "6/6 [==============================] - 0s 14ms/step - loss: 0.7023 - accuracy: 0.8352 - auc: 0.7125 - val_loss: 0.6768 - val_accuracy: 0.6286 - val_auc: 0.5940\n", "Epoch 6/100\n", "6/6 [==============================] - 0s 14ms/step - loss: 0.7124 - accuracy: 0.7473 - auc: 0.5973 - val_loss: 0.6784 - val_accuracy: 0.6571 - val_auc: 0.5620\n", "Epoch 7/100\n", "6/6 [==============================] - 0s 15ms/step - loss: 0.7050 - accuracy: 0.7418 - auc: 0.6743 - val_loss: 0.6745 - val_accuracy: 0.6857 - val_auc: 0.5876\n", "Epoch 8/100\n", "6/6 [==============================] - 0s 15ms/step - loss: 0.7171 - accuracy: 0.7198 - auc: 0.5686 - val_loss: 0.6788 - val_accuracy: 0.6571 - val_auc: 0.5791\n", "Epoch 9/100\n", "6/6 [==============================] - 0s 14ms/step - loss: 0.7103 - accuracy: 0.7857 - auc: 0.6456 - val_loss: 0.6763 - val_accuracy: 0.6286 - val_auc: 0.5833\n", "Epoch 10/100\n", "6/6 [==============================] - 0s 15ms/step - loss: 0.7200 - accuracy: 0.6758 - auc: 0.5049 - val_loss: 0.6835 - val_accuracy: 0.6000 - val_auc: 0.5748\n", "Epoch 11/100\n", "6/6 [==============================] - 0s 15ms/step - loss: 0.7163 - accuracy: 0.6813 - auc: 0.5431 - val_loss: 0.6821 - val_accuracy: 0.6286 - val_auc: 0.5812\n", "Epoch 12/100\n", "6/6 [==============================] - 0s 14ms/step - loss: 0.7158 - accuracy: 0.6758 - auc: 0.5672 - val_loss: 0.6884 - val_accuracy: 0.5143 - val_auc: 0.5641\n", "Epoch 13/100\n", "6/6 [==============================] - 0s 15ms/step - loss: 0.7001 - accuracy: 0.6264 - auc: 0.6711 - val_loss: 0.6924 - val_accuracy: 0.5143 - val_auc: 0.5577\n", "Epoch 14/100\n", "6/6 [==============================] - 0s 15ms/step - loss: 0.7190 - accuracy: 0.5879 - auc: 0.5463 - val_loss: 0.6888 - val_accuracy: 0.5714 - val_auc: 0.5534\n", "Epoch 15/100\n", "6/6 [==============================] - 0s 15ms/step - loss: 0.7086 - accuracy: 0.6154 - auc: 0.5858 - val_loss: 0.6935 - val_accuracy: 0.5429 - val_auc: 0.5427\n", "Epoch 16/100\n", "1/6 [====>.........................] - ETA: 0s - loss: 0.5012 - accuracy: 0.6875 - auc: 0.9516Restoring model weights from the end of the best epoch: 1.\n", "6/6 [==============================] - 0s 16ms/step - loss: 0.7082 - accuracy: 0.6319 - auc: 0.6475 - val_loss: 0.6946 - val_accuracy: 0.5429 - val_auc: 0.5299\n", "Epoch 16: early stopping\n", "\n", "βœ… GRU training complete!\n", "πŸ“Š Training stopped at epoch: 16\n", "πŸ† Best validation AUC: 0.6175\n" ] } ], "source": [ "print(\"⚑ Training GRU Model...\")\n", "print(\"=\" * 50)\n", "\n", "# Build GRU model (similar architecture to LSTM but with GRU layer)\n", "print(\"πŸ—οΈ Building GRU architecture...\")\n", "\n", "gru_model = Sequential([\n", " GRU(64, \n", " input_shape=(SEQUENCE_LENGTH, len(ts_feature_cols)),\n", " return_sequences=False,\n", " dropout=0.2,\n", " recurrent_dropout=0.2),\n", " Dropout(0.3),\n", " Dense(32, activation='relu'),\n", " Dropout(0.2),\n", " Dense(1, activation='sigmoid')\n", "])\n", "\n", "# Compile the GRU model\n", "gru_model.compile(\n", " optimizer=Adam(learning_rate=0.001),\n", " loss='binary_crossentropy',\n", " metrics=['accuracy', keras_metrics.AUC(name='auc')]\n", ")\n", "\n", "print(\"βœ… GRU model architecture:\")\n", "gru_model.summary()\n", "\n", "# Setup callbacks for GRU\n", "early_stopping_gru = EarlyStopping(\n", " monitor='val_auc',\n", " mode='max',\n", " patience=15,\n", " restore_best_weights=True,\n", " verbose=1\n", ")\n", "\n", "print(\"\\n🎯 Training GRU model...\")\n", "print(\"βš–οΈ Using same class weights as LSTM for fair comparison\")\n", "print(\"πŸ“Š Monitoring validation AUC with early stopping (patience=15)\")\n", "\n", "# Train the GRU model\n", "history_gru = gru_model.fit(\n", " train_generator,\n", " epochs=100,\n", " validation_data=test_generator,\n", " class_weight=class_weights_tf,\n", " callbacks=[early_stopping_gru],\n", " verbose=1\n", ")\n", "\n", "print(\"\\nβœ… GRU training complete!\")\n", "print(f\"πŸ“Š Training stopped at epoch: {len(history_gru.history['loss'])}\")\n", "print(f\"πŸ† Best validation AUC: {max(history_gru.history['val_auc']):.4f}\")" ] }, { "cell_type": "code", "execution_count": 35, "id": "09d0f720", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "πŸ”§ Finding optimal threshold for GRU...\n", "πŸ“Š GRU Prediction array shape: (35,)\n", "πŸ“Š Actual targets shape: (35,)\n", "πŸ“Š Sample predictions: [0.46566385 0.46448058 0.46035808 0.49024096 0.4869074 ]\n", "πŸ” Optimizing GRU prediction threshold...\n", "🎯 Optimal threshold: 0.470\n", "πŸ† Max F1-score: 0.3704\n", "\n", "πŸ“Š GRU Final Test Results:\n", " 🎯 F1-Score: 0.3704\n", " πŸ“ˆ AUC: 0.4954\n", " 🎯 Accuracy: 0.5143\n", "\n", "πŸ“‹ GRU Classification Report:\n", "============================================================\n", " precision recall f1-score support\n", "\n", " No Catch 0.81 0.48 0.60 27\n", " Catch 0.26 0.62 0.37 8\n", "\n", " accuracy 0.51 35\n", " macro avg 0.54 0.55 0.49 35\n", "weighted avg 0.69 0.51 0.55 35\n", "\n", "\n", "βœ… GRU evaluation complete and results stored!\n" ] } ], "source": [ "# Get GRU predictions for threshold optimization\n", "print(\"\\nπŸ”§ Finding optimal threshold for GRU...\")\n", "\n", "# Get predictions on test set\n", "y_test_proba_gru = gru_model.predict(test_generator, verbose=0)\n", "y_test_proba_gru = y_test_proba_gru.flatten() # Convert to 1D array\n", "\n", "# Use the same test targets as LSTM for consistency\n", "y_test_actual_gru = y_test_actual_lstm # Same targets\n", "\n", "# Ensure we have the right number of predictions\n", "min_len_gru = min(len(y_test_proba_gru), len(y_test_actual_gru))\n", "y_test_proba_gru = y_test_proba_gru[:min_len_gru]\n", "y_test_actual_gru = y_test_actual_gru[:min_len_gru]\n", "\n", "print(f\"πŸ“Š GRU Prediction array shape: {y_test_proba_gru.shape}\")\n", "print(f\"πŸ“Š Actual targets shape: {y_test_actual_gru.shape}\")\n", "print(f\"πŸ“Š Sample predictions: {y_test_proba_gru[:5]}\")\n", "\n", "# Find optimal threshold using the same logic as Part 1 and LSTM\n", "f1_scores_gru = []\n", "\n", "print(\"πŸ” Optimizing GRU prediction threshold...\")\n", "\n", "for threshold in thresholds:\n", " y_pred_thresh = (y_test_proba_gru >= threshold).astype(int)\n", " if len(np.unique(y_pred_thresh)) > 1: # Avoid division by zero\n", " f1 = f1_score(y_test_actual_gru, y_pred_thresh)\n", " else:\n", " f1 = 0.0\n", " f1_scores_gru.append(f1)\n", "\n", "optimal_threshold_gru = thresholds[np.argmax(f1_scores_gru)]\n", "max_f1_gru = max(f1_scores_gru)\n", "\n", "print(f\"🎯 Optimal threshold: {optimal_threshold_gru:.3f}\")\n", "print(f\"πŸ† Max F1-score: {max_f1_gru:.4f}\")\n", "\n", "# Make final predictions with optimal threshold\n", "y_test_pred_gru = (y_test_proba_gru >= optimal_threshold_gru).astype(int)\n", "\n", "# Calculate final metrics\n", "gru_test_f1 = f1_score(y_test_actual_gru, y_test_pred_gru)\n", "gru_test_auc = roc_auc_score(y_test_actual_gru, y_test_proba_gru)\n", "gru_test_accuracy = accuracy_score(y_test_actual_gru, y_test_pred_gru)\n", "\n", "print(f\"\\nπŸ“Š GRU Final Test Results:\")\n", "print(f\" 🎯 F1-Score: {gru_test_f1:.4f}\")\n", "print(f\" πŸ“ˆ AUC: {gru_test_auc:.4f}\")\n", "print(f\" 🎯 Accuracy: {gru_test_accuracy:.4f}\")\n", "\n", "# Detailed classification report\n", "print(f\"\\nπŸ“‹ GRU Classification Report:\")\n", "print(\"=\" * 60)\n", "print(classification_report(y_test_actual_gru, y_test_pred_gru, target_names=['No Catch', 'Catch']))\n", "\n", "# Store results in the same format as Part 1 and LSTM\n", "gru_results = {\n", " 'model': gru_model,\n", " 'threshold': optimal_threshold_gru,\n", " 'test_f1': gru_test_f1,\n", " 'test_auc': gru_test_auc,\n", " 'test_accuracy': gru_test_accuracy,\n", " 'y_pred': y_test_pred_gru,\n", " 'y_proba': y_test_proba_gru,\n", " 'y_actual': y_test_actual_gru,\n", " 'history': history_gru.history\n", "}\n", "\n", "print(\"\\nβœ… GRU evaluation complete and results stored!\")" ] }, { "cell_type": "code", "execution_count": 36, "id": "a8818bf6", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "πŸ“Š Creating GRU performance visualizations...\n" ] }, { "data": { "application/vnd.plotly.v1+json": { "config": { "plotlyServerURL": "https://plot.ly" }, "data": [ { "colorscale": [ [ 0, "rgb(247,252,245)" ], [ 0.125, "rgb(229,245,224)" ], [ 0.25, "rgb(199,233,192)" ], [ 0.375, "rgb(161,217,155)" ], [ 0.5, "rgb(116,196,118)" ], [ 0.625, "rgb(65,171,93)" ], [ 0.75, "rgb(35,139,69)" ], [ 0.875, "rgb(0,109,44)" ], [ 1, "rgb(0,68,27)" ] ], "reversescale": false, "showscale": true, "type": "heatmap", "x": [ "Predicted: No Catch", "Predicted: Catch" ], "y": [ "Actual: No Catch", "Actual: Catch" ], "z": [ [ 13, 14 ], [ 3, 5 ] ] } ], "layout": { "annotations": [ { "font": { "color": "#FFFFFF" }, "showarrow": false, "text": "13", "x": "Predicted: No Catch", "xref": "x", "y": "Actual: No Catch", "yref": "y" }, { "font": { "color": "#FFFFFF" }, "showarrow": false, "text": "14", "x": "Predicted: Catch", "xref": "x", "y": "Actual: No Catch", "yref": "y" }, { "font": { "color": "#000000" }, "showarrow": false, "text": "3", "x": "Predicted: No Catch", "xref": "x", "y": "Actual: Catch", "yref": "y" }, { "font": { "color": "#000000" }, "showarrow": false, "text": "5", "x": "Predicted: Catch", "xref": "x", "y": "Actual: Catch", "yref": "y" } ], "height": 400, "template": { "data": { "bar": [ { "error_x": { "color": "#2a3f5f" }, "error_y": { "color": "#2a3f5f" }, "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "bar" } ], "barpolar": [ { "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "barpolar" } ], "carpet": [ { "aaxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "baxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "type": "carpet" } ], "choropleth": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "choropleth" } ], "contour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "contour" } ], "contourcarpet": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "contourcarpet" } ], "heatmap": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "heatmap" } ], "heatmapgl": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "heatmapgl" } ], "histogram": [ { "marker": { "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "histogram" } ], "histogram2d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "histogram2d" } ], "histogram2dcontour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "histogram2dcontour" } ], "mesh3d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "mesh3d" } ], "parcoords": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "parcoords" } ], "pie": [ { "automargin": true, "type": "pie" } ], "scatter": [ { "fillpattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 }, "type": "scatter" } ], "scatter3d": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatter3d" } ], "scattercarpet": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattercarpet" } ], "scattergeo": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergeo" } ], "scattergl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergl" } ], "scattermapbox": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattermapbox" } ], "scatterpolar": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolar" } ], "scatterpolargl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolargl" } ], "scatterternary": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterternary" } ], "surface": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "surface" } ], "table": [ { "cells": { "fill": { "color": "#EBF0F8" }, "line": { "color": "white" } }, "header": { "fill": { "color": "#C8D4E3" }, "line": { "color": "white" } }, "type": "table" } ] }, "layout": { "annotationdefaults": { "arrowcolor": "#2a3f5f", "arrowhead": 0, "arrowwidth": 1 }, "autotypenumbers": "strict", "coloraxis": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "colorscale": { "diverging": [ [ 0, "#8e0152" ], [ 0.1, "#c51b7d" ], [ 0.2, "#de77ae" ], [ 0.3, "#f1b6da" ], [ 0.4, "#fde0ef" ], [ 0.5, "#f7f7f7" ], [ 0.6, "#e6f5d0" ], [ 0.7, "#b8e186" ], [ 0.8, "#7fbc41" ], [ 0.9, "#4d9221" ], [ 1, "#276419" ] ], "sequential": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "sequentialminus": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ] }, "colorway": [ "#636efa", "#EF553B", "#00cc96", "#ab63fa", "#FFA15A", "#19d3f3", "#FF6692", "#B6E880", "#FF97FF", "#FECB52" ], "font": { "color": "#2a3f5f" }, "geo": { "bgcolor": "white", "lakecolor": "white", "landcolor": "#E5ECF6", "showlakes": true, "showland": true, "subunitcolor": "white" }, "hoverlabel": { "align": "left" }, "hovermode": "closest", "mapbox": { "style": "light" }, "paper_bgcolor": "white", "plot_bgcolor": "#E5ECF6", "polar": { "angularaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "radialaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "scene": { "xaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "yaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "zaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" } }, "shapedefaults": { "line": { "color": "#2a3f5f" } }, "ternary": { "aaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "baxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "caxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "title": { "x": 0.05 }, "xaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 }, "yaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 } } }, "title": { "text": "⚑ GRU Confusion Matrix", "x": 0.5 }, "width": 500, "xaxis": { "dtick": 1, "gridcolor": "rgb(0, 0, 0)", "side": "top", "ticks": "" }, "yaxis": { "dtick": 1, "ticks": "", "ticksuffix": " " } } } }, "metadata": {}, "output_type": "display_data" }, { "data": { "application/vnd.plotly.v1+json": { "config": { "plotlyServerURL": "https://plot.ly" }, "data": [ { "line": { "color": "green", "width": 3 }, "mode": "lines", "name": "GRU (AUC = 0.495)", "type": "scatter", "x": [ 0, 0.037037037037037035, 0.07407407407407407, 0.07407407407407407, 0.1111111111111111, 0.1111111111111111, 0.3333333333333333, 0.3333333333333333, 0.48148148148148145, 0.48148148148148145, 0.5185185185185185, 0.5185185185185185, 0.7037037037037037, 0.7037037037037037, 0.8148148148148148, 0.8148148148148148, 1, 1 ], "y": [ 0, 0, 0, 0.125, 0.125, 0.25, 0.25, 0.375, 0.375, 0.5, 0.5, 0.625, 0.625, 0.75, 0.75, 0.875, 0.875, 1 ] }, { "line": { "color": "red", "dash": "dash", "width": 2 }, "mode": "lines", "name": "Random Classifier", "type": "scatter", "x": [ 0, 1 ], "y": [ 0, 1 ] } ], "layout": { "height": 500, "template": { "data": { "bar": [ { "error_x": { "color": "#2a3f5f" }, "error_y": { "color": "#2a3f5f" }, "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "bar" } ], "barpolar": [ { "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "barpolar" } ], "carpet": [ { "aaxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "baxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "type": "carpet" } ], "choropleth": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "choropleth" } ], "contour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "contour" } ], "contourcarpet": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "contourcarpet" } ], "heatmap": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "heatmap" } ], "heatmapgl": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "heatmapgl" } ], "histogram": [ { "marker": { "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "histogram" } ], "histogram2d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "histogram2d" } ], "histogram2dcontour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "histogram2dcontour" } ], "mesh3d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "mesh3d" } ], "parcoords": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "parcoords" } ], "pie": [ { "automargin": true, "type": "pie" } ], "scatter": [ { "fillpattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 }, "type": "scatter" } ], "scatter3d": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatter3d" } ], "scattercarpet": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattercarpet" } ], "scattergeo": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergeo" } ], "scattergl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergl" } ], "scattermapbox": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattermapbox" } ], "scatterpolar": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolar" } ], "scatterpolargl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolargl" } ], "scatterternary": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterternary" } ], "surface": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "surface" } ], "table": [ { "cells": { "fill": { "color": "#EBF0F8" }, "line": { "color": "white" } }, "header": { "fill": { "color": "#C8D4E3" }, "line": { "color": "white" } }, "type": "table" } ] }, "layout": { "annotationdefaults": { "arrowcolor": "#2a3f5f", "arrowhead": 0, "arrowwidth": 1 }, "autotypenumbers": "strict", "coloraxis": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "colorscale": { "diverging": [ [ 0, "#8e0152" ], [ 0.1, "#c51b7d" ], [ 0.2, "#de77ae" ], [ 0.3, "#f1b6da" ], [ 0.4, "#fde0ef" ], [ 0.5, "#f7f7f7" ], [ 0.6, "#e6f5d0" ], [ 0.7, "#b8e186" ], [ 0.8, "#7fbc41" ], [ 0.9, "#4d9221" ], [ 1, "#276419" ] ], "sequential": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "sequentialminus": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ] }, "colorway": [ "#636efa", "#EF553B", "#00cc96", "#ab63fa", "#FFA15A", "#19d3f3", "#FF6692", "#B6E880", "#FF97FF", "#FECB52" ], "font": { "color": "#2a3f5f" }, "geo": { "bgcolor": "white", "lakecolor": "white", "landcolor": "#E5ECF6", "showlakes": true, "showland": true, "subunitcolor": "white" }, "hoverlabel": { "align": "left" }, "hovermode": "closest", "mapbox": { "style": "light" }, "paper_bgcolor": "white", "plot_bgcolor": "#E5ECF6", "polar": { "angularaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "radialaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "scene": { "xaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "yaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "zaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" } }, "shapedefaults": { "line": { "color": "#2a3f5f" } }, "ternary": { "aaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "baxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "caxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "title": { "x": 0.05 }, "xaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 }, "yaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 } } }, "title": { "text": "⚑ GRU ROC Curve", "x": 0.5 }, "width": 600, "xaxis": { "title": { "text": "False Positive Rate" } }, "yaxis": { "title": { "text": "True Positive Rate" } } } } }, "metadata": {}, "output_type": "display_data" }, { "data": { "application/vnd.plotly.v1+json": { "config": { "plotlyServerURL": "https://plot.ly" }, "data": [ { "line": { "color": "blue" }, "mode": "lines", "name": "Training Loss", "type": "scatter", "x": [ 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16 ], "xaxis": "x", "y": [ 0.7298606634140015, 0.7194977402687073, 0.7181001305580139, 0.7209787964820862, 0.7022834420204163, 0.7123979926109314, 0.7050058841705322, 0.717091977596283, 0.7103168964385986, 0.7200220227241516, 0.7163000702857971, 0.7158256769180298, 0.7001358866691589, 0.7190122604370117, 0.7086300849914551, 0.7081863880157471 ], "yaxis": "y" }, { "line": { "color": "green" }, "mode": "lines", "name": "Training AUC", "type": "scatter", "x": [ 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16 ], "xaxis": "x2", "y": [ 0.41397058963775635, 0.539950966835022, 0.5465686321258545, 0.5333333611488342, 0.7124999761581421, 0.5973039269447327, 0.6742647290229797, 0.5686274766921997, 0.645588219165802, 0.5049019455909729, 0.54313725233078, 0.5671569108963013, 0.6710784435272217, 0.5463235378265381, 0.5857843160629272, 0.647549033164978 ], "yaxis": "y2" }, { "line": { "color": "red" }, "mode": "lines", "name": "Validation Loss", "type": "scatter", "x": [ 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16 ], "xaxis": "x3", "y": [ 0.6648333072662354, 0.6633040904998779, 0.6636539101600647, 0.669063150882721, 0.6767696738243103, 0.6784324645996094, 0.6745136380195618, 0.6787501573562622, 0.6763408780097961, 0.6834802627563477, 0.6821249723434448, 0.6884097456932068, 0.6923767924308777, 0.6887571811676025, 0.6935198307037354, 0.6945981979370117 ], "yaxis": "y3" }, { "line": { "color": "orange" }, "mode": "lines", "name": "Validation AUC", "type": "scatter", "x": [ 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16 ], "xaxis": "x4", "y": [ 0.6175214052200317, 0.6047008633613586, 0.5790598392486572, 0.5747863054275513, 0.5940170884132385, 0.561965823173523, 0.5876067876815796, 0.5790598392486572, 0.5833333134651184, 0.5747863054275513, 0.5811965465545654, 0.5641025304794312, 0.557692289352417, 0.553418755531311, 0.5427349805831909, 0.5299144983291626 ], "yaxis": "y4" } ], "layout": { "annotations": [ { "font": { "size": 16 }, "showarrow": false, "text": "Training Loss", "x": 0.225, "xanchor": "center", "xref": "paper", "y": 1, "yanchor": "bottom", "yref": "paper" }, { "font": { "size": 16 }, "showarrow": false, "text": "Training AUC", "x": 0.775, "xanchor": "center", "xref": "paper", "y": 1, "yanchor": "bottom", "yref": "paper" }, { "font": { "size": 16 }, "showarrow": false, "text": "Validation Loss", "x": 0.225, "xanchor": "center", "xref": "paper", "y": 0.375, "yanchor": "bottom", "yref": "paper" }, { "font": { "size": 16 }, "showarrow": false, "text": "Validation AUC", "x": 0.775, "xanchor": "center", "xref": "paper", "y": 0.375, "yanchor": "bottom", "yref": "paper" } ], "height": 600, "showlegend": false, "template": { "data": { "bar": [ { "error_x": { "color": "#2a3f5f" }, "error_y": { "color": "#2a3f5f" }, "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "bar" } ], "barpolar": [ { "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "barpolar" } ], "carpet": [ { "aaxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "baxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "type": "carpet" } ], "choropleth": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "choropleth" } ], "contour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "contour" } ], "contourcarpet": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "contourcarpet" } ], "heatmap": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "heatmap" } ], "heatmapgl": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "heatmapgl" } ], "histogram": [ { "marker": { "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "histogram" } ], "histogram2d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "histogram2d" } ], "histogram2dcontour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "histogram2dcontour" } ], "mesh3d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "mesh3d" } ], "parcoords": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "parcoords" } ], "pie": [ { "automargin": true, "type": "pie" } ], "scatter": [ { "fillpattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 }, "type": "scatter" } ], "scatter3d": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatter3d" } ], "scattercarpet": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattercarpet" } ], "scattergeo": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergeo" } ], "scattergl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergl" } ], "scattermapbox": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattermapbox" } ], "scatterpolar": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolar" } ], "scatterpolargl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolargl" } ], "scatterternary": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterternary" } ], "surface": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "surface" } ], "table": [ { "cells": { "fill": { "color": "#EBF0F8" }, "line": { "color": "white" } }, "header": { "fill": { "color": "#C8D4E3" }, "line": { "color": "white" } }, "type": "table" } ] }, "layout": { "annotationdefaults": { "arrowcolor": "#2a3f5f", "arrowhead": 0, "arrowwidth": 1 }, "autotypenumbers": "strict", "coloraxis": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "colorscale": { "diverging": [ [ 0, "#8e0152" ], [ 0.1, "#c51b7d" ], [ 0.2, "#de77ae" ], [ 0.3, "#f1b6da" ], [ 0.4, "#fde0ef" ], [ 0.5, "#f7f7f7" ], [ 0.6, "#e6f5d0" ], [ 0.7, "#b8e186" ], [ 0.8, "#7fbc41" ], [ 0.9, "#4d9221" ], [ 1, "#276419" ] ], "sequential": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "sequentialminus": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ] }, "colorway": [ "#636efa", "#EF553B", "#00cc96", "#ab63fa", "#FFA15A", "#19d3f3", "#FF6692", "#B6E880", "#FF97FF", "#FECB52" ], "font": { "color": "#2a3f5f" }, "geo": { "bgcolor": "white", "lakecolor": "white", "landcolor": "#E5ECF6", "showlakes": true, "showland": true, "subunitcolor": "white" }, "hoverlabel": { "align": "left" }, "hovermode": "closest", "mapbox": { "style": "light" }, "paper_bgcolor": "white", "plot_bgcolor": "#E5ECF6", "polar": { "angularaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "radialaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "scene": { "xaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "yaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "zaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" } }, "shapedefaults": { "line": { "color": "#2a3f5f" } }, "ternary": { "aaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "baxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "caxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "title": { "x": 0.05 }, "xaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 }, "yaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 } } }, "title": { "text": "⚑ GRU Training History", "x": 0.5 }, "xaxis": { "anchor": "y", "domain": [ 0, 0.45 ] }, "xaxis2": { "anchor": "y2", "domain": [ 0.55, 1 ] }, "xaxis3": { "anchor": "y3", "domain": [ 0, 0.45 ] }, "xaxis4": { "anchor": "y4", "domain": [ 0.55, 1 ] }, "yaxis": { "anchor": "x", "domain": [ 0.625, 1 ] }, "yaxis2": { "anchor": "x2", "domain": [ 0.625, 1 ] }, "yaxis3": { "anchor": "x3", "domain": [ 0, 0.375 ] }, "yaxis4": { "anchor": "x4", "domain": [ 0, 0.375 ] } } } }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "\n", "πŸ† GRU Tournament Summary:\n", " β€’ F1-Score: 0.3704\n", " β€’ AUC: 0.4954\n", " β€’ Optimal Threshold: 0.470\n", " β€’ Test Accuracy: 0.5143\n", " β€’ Training Epochs: 16\n", " β€’ Best Val AUC: 0.6175\n" ] } ], "source": [ "# Create GRU performance visualizations\n", "print(\"\\nπŸ“Š Creating GRU performance visualizations...\")\n", "\n", "# Interactive confusion matrix for GRU\n", "cm_gru = confusion_matrix(y_test_actual_gru, y_test_pred_gru)\n", "\n", "fig_cm_gru = ff.create_annotated_heatmap(\n", " z=cm_gru,\n", " x=['Predicted: No Catch', 'Predicted: Catch'],\n", " y=['Actual: No Catch', 'Actual: Catch'],\n", " annotation_text=cm_gru,\n", " colorscale='Greens',\n", " showscale=True\n", ")\n", "\n", "fig_cm_gru.update_layout(\n", " title='⚑ GRU Confusion Matrix',\n", " title_x=0.5,\n", " width=500,\n", " height=400\n", ")\n", "\n", "fig_cm_gru.show()\n", "\n", "# Interactive ROC curve for GRU\n", "fpr_gru, tpr_gru, _ = roc_curve(y_test_actual_gru, y_test_proba_gru)\n", "\n", "fig_roc_gru = go.Figure()\n", "\n", "fig_roc_gru.add_trace(go.Scatter(\n", " x=fpr_gru,\n", " y=tpr_gru,\n", " mode='lines',\n", " name=f'GRU (AUC = {gru_test_auc:.3f})',\n", " line=dict(color='green', width=3)\n", "))\n", "\n", "# Add diagonal reference line\n", "fig_roc_gru.add_trace(go.Scatter(\n", " x=[0, 1],\n", " y=[0, 1],\n", " mode='lines',\n", " name='Random Classifier',\n", " line=dict(color='red', width=2, dash='dash')\n", "))\n", "\n", "fig_roc_gru.update_layout(\n", " title='⚑ GRU ROC Curve',\n", " title_x=0.5,\n", " xaxis_title='False Positive Rate',\n", " yaxis_title='True Positive Rate',\n", " width=600,\n", " height=500\n", ")\n", "\n", "fig_roc_gru.show()\n", "\n", "# GRU Training history visualization\n", "fig_history_gru = make_subplots(\n", " rows=2, cols=2,\n", " subplot_titles=['Training Loss', 'Training AUC', 'Validation Loss', 'Validation AUC'],\n", " specs=[[{\"type\": \"scatter\"}, {\"type\": \"scatter\"}], \n", " [{\"type\": \"scatter\"}, {\"type\": \"scatter\"}]]\n", ")\n", "\n", "epochs_gru = range(1, len(history_gru.history['loss']) + 1)\n", "\n", "# Training Loss\n", "fig_history_gru.add_trace(\n", " go.Scatter(x=list(epochs_gru), y=history_gru.history['loss'], \n", " mode='lines', name='Training Loss', line=dict(color='blue')),\n", " row=1, col=1\n", ")\n", "\n", "# Training AUC\n", "fig_history_gru.add_trace(\n", " go.Scatter(x=list(epochs_gru), y=history_gru.history['auc'], \n", " mode='lines', name='Training AUC', line=dict(color='green')),\n", " row=1, col=2\n", ")\n", "\n", "# Validation Loss\n", "fig_history_gru.add_trace(\n", " go.Scatter(x=list(epochs_gru), y=history_gru.history['val_loss'], \n", " mode='lines', name='Validation Loss', line=dict(color='red')),\n", " row=2, col=1\n", ")\n", "\n", "# Validation AUC\n", "fig_history_gru.add_trace(\n", " go.Scatter(x=list(epochs_gru), y=history_gru.history['val_auc'], \n", " mode='lines', name='Validation AUC', line=dict(color='orange')),\n", " row=2, col=2\n", ")\n", "\n", "fig_history_gru.update_layout(\n", " title='⚑ GRU Training History',\n", " title_x=0.5,\n", " height=600,\n", " showlegend=False\n", ")\n", "\n", "fig_history_gru.show()\n", "\n", "print(f\"\\nπŸ† GRU Tournament Summary:\")\n", "print(f\" β€’ F1-Score: {gru_test_f1:.4f}\")\n", "print(f\" β€’ AUC: {gru_test_auc:.4f}\")\n", "print(f\" β€’ Optimal Threshold: {optimal_threshold_gru:.3f}\")\n", "print(f\" β€’ Test Accuracy: {gru_test_accuracy:.4f}\")\n", "print(f\" β€’ Training Epochs: {len(history_gru.history['loss'])}\")\n", "print(f\" β€’ Best Val AUC: {max(history_gru.history['val_auc']):.4f}\")" ] }, { "cell_type": "markdown", "id": "929413a0", "metadata": {}, "source": [ "## 🧠 Section 4.4: Deep Learning Champion Selection\n", "\n", "### The Neural Network Showdown\n", "\n", "After intense training and evaluation, our two deep learning contestants have battled through:\n", "- βœ… **Sequence Learning**: Processing 14-day environmental patterns\n", "- βœ… **Class Balancing**: Handling severe pest occurrence imbalance\n", "- βœ… **Threshold Optimization**: Maximizing F1-score performance\n", "- βœ… **Temporal Validation**: Respecting time-series data integrity\n", "\n", "Now it's time to crown our **Deep Learning Champion** who will advance to face the Standard Classifier Champion in the Grand Finale!\n", "\n", "### Evaluation Criteria:\n", "1. **🎯 Primary**: F1-Score (optimal for imbalanced classification)\n", "2. **πŸ“ˆ Secondary**: AUC (model's discrimination ability)\n", "3. **⚑ Tertiary**: Training efficiency and convergence" ] }, { "cell_type": "code", "execution_count": 37, "id": "882f4575", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "πŸ† DEEP LEARNING TOURNAMENT RESULTS\n", "============================================================\n", "πŸ“Š Deep Learning Final Standings:\n" ] }, { "data": { "application/vnd.microsoft.datawrangler.viewer.v0+json": { "columns": [ { "name": "index", "rawType": "int64", "type": "integer" }, { "name": "Model", "rawType": "object", "type": "string" }, { "name": "F1-Score", "rawType": "float64", "type": "float" }, { "name": "AUC", "rawType": "float64", "type": "float" }, { "name": "Accuracy", "rawType": "float64", "type": "float" }, { "name": "Optimal Threshold", "rawType": "float64", "type": "float" }, { "name": "Training Epochs", "rawType": "int64", "type": "integer" }, { "name": "Best Val AUC", "rawType": "float64", "type": "float" } ], "ref": "42a23a47-306f-4da7-8b62-3f0acb9c4f6e", "rows": [ [ "0", "πŸ”— LSTM", "0.4545", "0.6806", "0.6571", "0.48", "17", "0.6239" ], [ "1", "⚑ GRU", "0.3704", "0.4954", "0.5143", "0.47", "16", "0.6175" ] ], "shape": { "columns": 7, "rows": 2 } }, "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
ModelF1-ScoreAUCAccuracyOptimal ThresholdTraining EpochsBest Val AUC
0πŸ”— LSTM0.45450.68060.65710.48170.6239
1⚑ GRU0.37040.49540.51430.47160.6175
\n", "
" ], "text/plain": [ " Model F1-Score AUC Accuracy Optimal Threshold Training Epochs \\\n", "0 πŸ”— LSTM 0.4545 0.6806 0.6571 0.48 17 \n", "1 ⚑ GRU 0.3704 0.4954 0.5143 0.47 16 \n", "\n", " Best Val AUC \n", "0 0.6239 \n", "1 0.6175 " ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "\n", "πŸ₯‡ DEEP LEARNING CHAMPION: πŸ”— LSTM\n", " β€’ Winning F1-Score: 0.4545\n", " β€’ Winning AUC: 0.6806\n", "\n", "🎯 Champion Details:\n", " β€’ Model Type: LSTM\n", " β€’ Training Epochs: 17\n", " β€’ Optimal Threshold: 0.480\n", " β€’ Test Accuracy: 0.6571\n" ] } ], "source": [ "print(\"πŸ† DEEP LEARNING TOURNAMENT RESULTS\")\n", "print(\"=\" * 60)\n", "\n", "# Create comprehensive comparison table for deep learning models\n", "dl_comparison_data = {\n", " 'Model': ['πŸ”— LSTM', '⚑ GRU'],\n", " 'F1-Score': [lstm_results['test_f1'], gru_results['test_f1']],\n", " 'AUC': [lstm_results['test_auc'], gru_results['test_auc']],\n", " 'Accuracy': [lstm_results['test_accuracy'], gru_results['test_accuracy']],\n", " 'Optimal Threshold': [lstm_results['threshold'], gru_results['threshold']],\n", " 'Training Epochs': [len(lstm_results['history']['loss']), len(gru_results['history']['loss'])],\n", " 'Best Val AUC': [max(lstm_results['history']['val_auc']), max(gru_results['history']['val_auc'])]\n", "}\n", "\n", "dl_comparison_df = pd.DataFrame(dl_comparison_data)\n", "\n", "# Sort by F1-Score (primary) and AUC (secondary)\n", "dl_comparison_df = dl_comparison_df.sort_values(['F1-Score', 'AUC'], ascending=False)\n", "\n", "print(\"πŸ“Š Deep Learning Final Standings:\")\n", "display(dl_comparison_df.round(4))\n", "\n", "# Determine deep learning champion\n", "dl_champion_idx = dl_comparison_df.index[0]\n", "dl_champion_name = dl_comparison_df.iloc[0]['Model']\n", "dl_champion_f1 = dl_comparison_df.iloc[0]['F1-Score']\n", "dl_champion_auc = dl_comparison_df.iloc[0]['AUC']\n", "\n", "print(f\"\\nπŸ₯‡ DEEP LEARNING CHAMPION: {dl_champion_name}\")\n", "print(f\" β€’ Winning F1-Score: {dl_champion_f1:.4f}\")\n", "print(f\" β€’ Winning AUC: {dl_champion_auc:.4f}\")\n", "\n", "# Store deep learning champion for Grand Finale\n", "if 'LSTM' in dl_champion_name:\n", " deep_learning_champion = lstm_results\n", " deep_learning_champion_name = 'LSTM'\n", "else:\n", " deep_learning_champion = gru_results\n", " deep_learning_champion_name = 'GRU'\n", "\n", "print(f\"\\n🎯 Champion Details:\")\n", "print(f\" β€’ Model Type: {deep_learning_champion_name}\")\n", "print(f\" β€’ Training Epochs: {len(deep_learning_champion['history']['loss'])}\")\n", "print(f\" β€’ Optimal Threshold: {deep_learning_champion['threshold']:.3f}\")\n", "print(f\" β€’ Test Accuracy: {deep_learning_champion['test_accuracy']:.4f}\")" ] }, { "cell_type": "code", "execution_count": 38, "id": "c60812dd", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "πŸ“Š Creating deep learning comparison visualizations...\n" ] }, { "data": { "application/vnd.plotly.v1+json": { "config": { "plotlyServerURL": "https://plot.ly" }, "data": [ { "marker": { "color": [ "blue", "green" ] }, "name": "F1-Score", "text": [ "0.4545", "0.3704" ], "textposition": "outside", "type": "bar", "x": [ "LSTM", "GRU" ], "xaxis": "x", "y": [ 0.45454545454545453, 0.37037037037037035 ], "yaxis": "y" }, { "marker": { "color": [ "blue", "green" ] }, "name": "AUC", "showlegend": false, "text": [ "0.6806", "0.4954" ], "textposition": "outside", "type": "bar", "x": [ "LSTM", "GRU" ], "xaxis": "x2", "y": [ 0.6805555555555556, 0.49537037037037046 ], "yaxis": "y2" }, { "marker": { "color": [ "blue", "green" ] }, "name": "Epochs", "showlegend": false, "text": [ "17", "16" ], "textposition": "outside", "type": "bar", "x": [ "LSTM", "GRU" ], "xaxis": "x3", "y": [ 17, 16 ], "yaxis": "y3" } ], "layout": { "annotations": [ { "font": { "size": 16 }, "showarrow": false, "text": "F1-Score Comparison", "x": 0.14444444444444446, "xanchor": "center", "xref": "paper", "y": 1, "yanchor": "bottom", "yref": "paper" }, { "font": { "size": 16 }, "showarrow": false, "text": "AUC Comparison", "x": 0.5, "xanchor": "center", "xref": "paper", "y": 1, "yanchor": "bottom", "yref": "paper" }, { "font": { "size": 16 }, "showarrow": false, "text": "Training Efficiency", "x": 0.8555555555555556, "xanchor": "center", "xref": "paper", "y": 1, "yanchor": "bottom", "yref": "paper" } ], "height": 500, "showlegend": false, "template": { "data": { "bar": [ { "error_x": { "color": "#2a3f5f" }, "error_y": { "color": "#2a3f5f" }, "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "bar" } ], "barpolar": [ { "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "barpolar" } ], "carpet": [ { "aaxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "baxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "type": "carpet" } ], "choropleth": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "choropleth" } ], "contour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "contour" } ], "contourcarpet": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "contourcarpet" } ], "heatmap": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "heatmap" } ], "heatmapgl": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "heatmapgl" } ], "histogram": [ { "marker": { "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "histogram" } ], "histogram2d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "histogram2d" } ], "histogram2dcontour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "histogram2dcontour" } ], "mesh3d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "mesh3d" } ], "parcoords": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "parcoords" } ], "pie": [ { "automargin": true, "type": "pie" } ], "scatter": [ { "fillpattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 }, "type": "scatter" } ], "scatter3d": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatter3d" } ], "scattercarpet": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattercarpet" } ], "scattergeo": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergeo" } ], "scattergl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergl" } ], "scattermapbox": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattermapbox" } ], "scatterpolar": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolar" } ], "scatterpolargl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolargl" } ], "scatterternary": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterternary" } ], "surface": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "surface" } ], "table": [ { "cells": { "fill": { "color": "#EBF0F8" }, "line": { "color": "white" } }, "header": { "fill": { "color": "#C8D4E3" }, "line": { "color": "white" } }, "type": "table" } ] }, "layout": { "annotationdefaults": { "arrowcolor": "#2a3f5f", "arrowhead": 0, "arrowwidth": 1 }, "autotypenumbers": "strict", "coloraxis": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "colorscale": { "diverging": [ [ 0, "#8e0152" ], [ 0.1, "#c51b7d" ], [ 0.2, "#de77ae" ], [ 0.3, "#f1b6da" ], [ 0.4, "#fde0ef" ], [ 0.5, "#f7f7f7" ], [ 0.6, "#e6f5d0" ], [ 0.7, "#b8e186" ], [ 0.8, "#7fbc41" ], [ 0.9, "#4d9221" ], [ 1, "#276419" ] ], "sequential": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "sequentialminus": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ] }, "colorway": [ "#636efa", "#EF553B", "#00cc96", "#ab63fa", "#FFA15A", "#19d3f3", "#FF6692", "#B6E880", "#FF97FF", "#FECB52" ], "font": { "color": "#2a3f5f" }, "geo": { "bgcolor": "white", "lakecolor": "white", "landcolor": "#E5ECF6", "showlakes": true, "showland": true, "subunitcolor": "white" }, "hoverlabel": { "align": "left" }, "hovermode": "closest", "mapbox": { "style": "light" }, "paper_bgcolor": "white", "plot_bgcolor": "#E5ECF6", "polar": { "angularaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "radialaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "scene": { "xaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "yaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "zaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" } }, "shapedefaults": { "line": { "color": "#2a3f5f" } }, "ternary": { "aaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "baxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "caxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "title": { "x": 0.05 }, "xaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 }, "yaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 } } }, "title": { "text": "🧠 Deep Learning Tournament Results", "x": 0.5 }, "xaxis": { "anchor": "y", "domain": [ 0, 0.2888888888888889 ] }, "xaxis2": { "anchor": "y2", "domain": [ 0.35555555555555557, 0.6444444444444445 ] }, "xaxis3": { "anchor": "y3", "domain": [ 0.7111111111111111, 1 ] }, "yaxis": { "anchor": "x", "domain": [ 0, 1 ] }, "yaxis2": { "anchor": "x2", "domain": [ 0, 1 ] }, "yaxis3": { "anchor": "x3", "domain": [ 0, 1 ] } } } }, "metadata": {}, "output_type": "display_data" }, { "data": { "application/vnd.plotly.v1+json": { "config": { "plotlyServerURL": "https://plot.ly" }, "data": [ { "line": { "color": "blue", "width": 3 }, "mode": "lines", "name": "πŸ”— LSTM (AUC = 0.681)", "type": "scatter", "x": [ 0, 0.037037037037037035, 0.07407407407407407, 0.07407407407407407, 0.1111111111111111, 0.1111111111111111, 0.2222222222222222, 0.2222222222222222, 0.37037037037037035, 0.37037037037037035, 0.5555555555555556, 0.5555555555555556, 0.8888888888888888, 0.8888888888888888, 1 ], "y": [ 0, 0, 0, 0.125, 0.125, 0.375, 0.375, 0.625, 0.625, 0.75, 0.75, 0.875, 0.875, 1, 1 ] }, { "line": { "color": "green", "width": 3 }, "mode": "lines", "name": "⚑ GRU (AUC = 0.495)", "type": "scatter", "x": [ 0, 0.037037037037037035, 0.07407407407407407, 0.07407407407407407, 0.1111111111111111, 0.1111111111111111, 0.3333333333333333, 0.3333333333333333, 0.48148148148148145, 0.48148148148148145, 0.5185185185185185, 0.5185185185185185, 0.7037037037037037, 0.7037037037037037, 0.8148148148148148, 0.8148148148148148, 1, 1 ], "y": [ 0, 0, 0, 0.125, 0.125, 0.25, 0.25, 0.375, 0.375, 0.5, 0.5, 0.625, 0.625, 0.75, 0.75, 0.875, 0.875, 1 ] }, { "line": { "color": "red", "dash": "dash", "width": 2 }, "mode": "lines", "name": "Random Classifier", "type": "scatter", "x": [ 0, 1 ], "y": [ 0, 1 ] } ], "layout": { "height": 600, "template": { "data": { "bar": [ { "error_x": { "color": "#2a3f5f" }, "error_y": { "color": "#2a3f5f" }, "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "bar" } ], "barpolar": [ { "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "barpolar" } ], "carpet": [ { "aaxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "baxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "type": "carpet" } ], "choropleth": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "choropleth" } ], "contour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "contour" } ], "contourcarpet": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "contourcarpet" } ], "heatmap": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "heatmap" } ], "heatmapgl": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "heatmapgl" } ], "histogram": [ { "marker": { "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "histogram" } ], "histogram2d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "histogram2d" } ], "histogram2dcontour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "histogram2dcontour" } ], "mesh3d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "mesh3d" } ], "parcoords": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "parcoords" } ], "pie": [ { "automargin": true, "type": "pie" } ], "scatter": [ { "fillpattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 }, "type": "scatter" } ], "scatter3d": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatter3d" } ], "scattercarpet": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattercarpet" } ], "scattergeo": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergeo" } ], "scattergl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergl" } ], "scattermapbox": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattermapbox" } ], "scatterpolar": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolar" } ], "scatterpolargl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolargl" } ], "scatterternary": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterternary" } ], "surface": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "surface" } ], "table": [ { "cells": { "fill": { "color": "#EBF0F8" }, "line": { "color": "white" } }, "header": { "fill": { "color": "#C8D4E3" }, "line": { "color": "white" } }, "type": "table" } ] }, "layout": { "annotationdefaults": { "arrowcolor": "#2a3f5f", "arrowhead": 0, "arrowwidth": 1 }, "autotypenumbers": "strict", "coloraxis": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "colorscale": { "diverging": [ [ 0, "#8e0152" ], [ 0.1, "#c51b7d" ], [ 0.2, "#de77ae" ], [ 0.3, "#f1b6da" ], [ 0.4, "#fde0ef" ], [ 0.5, "#f7f7f7" ], [ 0.6, "#e6f5d0" ], [ 0.7, "#b8e186" ], [ 0.8, "#7fbc41" ], [ 0.9, "#4d9221" ], [ 1, "#276419" ] ], "sequential": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "sequentialminus": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ] }, "colorway": [ "#636efa", "#EF553B", "#00cc96", "#ab63fa", "#FFA15A", "#19d3f3", "#FF6692", "#B6E880", "#FF97FF", "#FECB52" ], "font": { "color": "#2a3f5f" }, "geo": { "bgcolor": "white", "lakecolor": "white", "landcolor": "#E5ECF6", "showlakes": true, "showland": true, "subunitcolor": "white" }, "hoverlabel": { "align": "left" }, "hovermode": "closest", "mapbox": { "style": "light" }, "paper_bgcolor": "white", "plot_bgcolor": "#E5ECF6", "polar": { "angularaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "radialaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "scene": { "xaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "yaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "zaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" } }, "shapedefaults": { "line": { "color": "#2a3f5f" } }, "ternary": { "aaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "baxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "caxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "title": { "x": 0.05 }, "xaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 }, "yaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 } } }, "title": { "text": "🧠 Combined ROC Curves - Deep Learning Tournament", "x": 0.5 }, "width": 700, "xaxis": { "title": { "text": "False Positive Rate" } }, "yaxis": { "title": { "text": "True Positive Rate" } } } } }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "\n", "πŸ“Š Deep Learning Performance Insights:\n", " πŸ₯‡ Champion: LSTM with F1-Score: 0.4545\n", " ⚑ Training Efficiency: LSTM converged in 17 epochs\n", " 🎯 Threshold Optimization: Both models used optimal thresholds for imbalanced data\n", " πŸ“ˆ AUC Performance: Shows model discrimination ability on sequential data\n", "\n", "🧠 Deep Learning Tournament Complete!\n", " πŸ₯‡ Champion: LSTM\n", " πŸ“Š Champion advances to Grand Finale with F1-Score: 0.4545\n" ] } ], "source": [ "# Create interactive comparison visualizations for deep learning models\n", "print(\"\\nπŸ“Š Creating deep learning comparison visualizations...\")\n", "\n", "# Performance comparison charts\n", "fig_dl_comparison = make_subplots(\n", " rows=1, cols=3,\n", " subplot_titles=['F1-Score Comparison', 'AUC Comparison', 'Training Efficiency'],\n", " specs=[[{\"type\": \"bar\"}, {\"type\": \"bar\"}, {\"type\": \"bar\"}]]\n", ")\n", "\n", "dl_models = ['LSTM', 'GRU']\n", "dl_colors = ['blue', 'green']\n", "dl_f1_scores = [lstm_results['test_f1'], gru_results['test_f1']]\n", "dl_auc_scores = [lstm_results['test_auc'], gru_results['test_auc']]\n", "dl_epochs = [len(lstm_results['history']['loss']), len(gru_results['history']['loss'])]\n", "\n", "# F1-Score comparison\n", "fig_dl_comparison.add_trace(\n", " go.Bar(x=dl_models, y=dl_f1_scores, name='F1-Score', marker_color=dl_colors,\n", " text=[f'{score:.4f}' for score in dl_f1_scores], textposition='outside'),\n", " row=1, col=1\n", ")\n", "\n", "# AUC comparison\n", "fig_dl_comparison.add_trace(\n", " go.Bar(x=dl_models, y=dl_auc_scores, name='AUC', marker_color=dl_colors,\n", " text=[f'{score:.4f}' for score in dl_auc_scores], textposition='outside', showlegend=False),\n", " row=1, col=2\n", ")\n", "\n", "# Training epochs (efficiency)\n", "fig_dl_comparison.add_trace(\n", " go.Bar(x=dl_models, y=dl_epochs, name='Epochs', marker_color=dl_colors,\n", " text=[f'{epochs}' for epochs in dl_epochs], textposition='outside', showlegend=False),\n", " row=1, col=3\n", ")\n", "\n", "fig_dl_comparison.update_layout(\n", " title='🧠 Deep Learning Tournament Results',\n", " title_x=0.5,\n", " height=500,\n", " showlegend=False\n", ")\n", "\n", "fig_dl_comparison.show()\n", "\n", "# Combined ROC curves for deep learning models\n", "fig_dl_combined_roc = go.Figure()\n", "\n", "# LSTM ROC\n", "fpr_lstm, tpr_lstm, _ = roc_curve(lstm_results['y_actual'], lstm_results['y_proba'])\n", "fig_dl_combined_roc.add_trace(go.Scatter(\n", " x=fpr_lstm, y=tpr_lstm, mode='lines',\n", " name=f'πŸ”— LSTM (AUC = {lstm_results[\"test_auc\"]:.3f})',\n", " line=dict(color='blue', width=3)\n", "))\n", "\n", "# GRU ROC\n", "fpr_gru, tpr_gru, _ = roc_curve(gru_results['y_actual'], gru_results['y_proba'])\n", "fig_dl_combined_roc.add_trace(go.Scatter(\n", " x=fpr_gru, y=tpr_gru, mode='lines',\n", " name=f'⚑ GRU (AUC = {gru_results[\"test_auc\"]:.3f})',\n", " line=dict(color='green', width=3)\n", "))\n", "\n", "# Random classifier reference\n", "fig_dl_combined_roc.add_trace(go.Scatter(\n", " x=[0, 1], y=[0, 1], mode='lines',\n", " name='Random Classifier',\n", " line=dict(color='red', width=2, dash='dash')\n", "))\n", "\n", "fig_dl_combined_roc.update_layout(\n", " title='🧠 Combined ROC Curves - Deep Learning Tournament',\n", " title_x=0.5,\n", " xaxis_title='False Positive Rate',\n", " yaxis_title='True Positive Rate',\n", " width=700,\n", " height=600\n", ")\n", "\n", "fig_dl_combined_roc.show()\n", "\n", "# Performance insights\n", "print(f\"\\nπŸ“Š Deep Learning Performance Insights:\")\n", "print(f\" πŸ₯‡ Champion: {deep_learning_champion_name} with F1-Score: {dl_champion_f1:.4f}\")\n", "print(f\" ⚑ Training Efficiency: {deep_learning_champion_name} converged in {len(deep_learning_champion['history']['loss'])} epochs\")\n", "print(f\" 🎯 Threshold Optimization: Both models used optimal thresholds for imbalanced data\")\n", "print(f\" πŸ“ˆ AUC Performance: Shows model discrimination ability on sequential data\")\n", "\n", "print(f\"\\n🧠 Deep Learning Tournament Complete!\")\n", "print(f\" πŸ₯‡ Champion: {deep_learning_champion_name}\")\n", "print(f\" πŸ“Š Champion advances to Grand Finale with F1-Score: {dl_champion_f1:.4f}\")" ] }, { "cell_type": "code", "execution_count": 39, "id": "e9b7d0ad", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "πŸ’Ύ Saving Part 2 Deep Learning Tournament Results...\n", "βœ… Champion model saved: part2_champion_lstm.h5\n", "βœ… Feature scaler saved: part2_scaler_lstm.joblib\n", "βœ… Part 2 results saved: part2_deep_learning_tournament_results.json\n", "\n", "πŸ† Part 2 Complete! Champion: LSTM (F1: 0.4545)\n", "πŸ”œ Ready for Part 3: GRAND FINALE!\n", "\n", "βš”οΈ Grand Finale Matchup:\n", " πŸ₯Š Standard Champion: RandomForest (F1: 0.6667)\n", " 🧠 Deep Learning Champion: LSTM (F1: 0.4545)\n" ] } ], "source": [ "# Save deep learning tournament artifacts\n", "print(\"\\nπŸ’Ύ Saving Part 2 Deep Learning Tournament Results...\")\n", "\n", "# Save champion model in .h5 format\n", "champion_model_path = f'part2_champion_{deep_learning_champion_name.lower()}.h5'\n", "deep_learning_champion['model'].save(champion_model_path)\n", "print(f\"βœ… Champion model saved: {champion_model_path}\")\n", "\n", "# Save scaler for future use\n", "scaler_path = f'part2_scaler_{deep_learning_champion_name.lower()}.joblib'\n", "joblib.dump(ts_scaler, scaler_path)\n", "print(f\"βœ… Feature scaler saved: {scaler_path}\")\n", "\n", "# Create Part 2 summary dictionary\n", "part2_summary = {\n", " 'tournament_type': 'Deep_Learning',\n", " 'models_tested': ['LSTM', 'GRU'],\n", " 'sequence_length': SEQUENCE_LENGTH,\n", " 'batch_size': BATCH_SIZE,\n", " 'champion': {\n", " 'name': deep_learning_champion_name,\n", " 'f1_score': float(dl_champion_f1),\n", " 'auc_score': float(dl_champion_auc),\n", " 'threshold': float(deep_learning_champion['threshold']),\n", " 'accuracy': float(deep_learning_champion['test_accuracy']),\n", " 'training_epochs': len(deep_learning_champion['history']['loss']),\n", " 'best_val_auc': float(max(deep_learning_champion['history']['val_auc']))\n", " },\n", " 'all_results': {\n", " 'LSTM': {\n", " 'f1_score': float(lstm_results['test_f1']),\n", " 'auc_score': float(lstm_results['test_auc']),\n", " 'threshold': float(lstm_results['threshold']),\n", " 'accuracy': float(lstm_results['test_accuracy']),\n", " 'training_epochs': len(lstm_results['history']['loss']),\n", " 'best_val_auc': float(max(lstm_results['history']['val_auc']))\n", " },\n", " 'GRU': {\n", " 'f1_score': float(gru_results['test_f1']),\n", " 'auc_score': float(gru_results['test_auc']),\n", " 'threshold': float(gru_results['threshold']),\n", " 'accuracy': float(gru_results['test_accuracy']),\n", " 'training_epochs': len(gru_results['history']['loss']),\n", " 'best_val_auc': float(max(gru_results['history']['val_auc']))\n", " }\n", " },\n", " 'data_info': {\n", " 'sequence_samples': effective_train_samples + effective_test_samples,\n", " 'train_sequences': effective_train_samples,\n", " 'test_sequences': effective_test_samples,\n", " 'features_count': len(ts_feature_cols),\n", " 'sequence_length': SEQUENCE_LENGTH\n", " }\n", "}\n", "\n", "# Save to JSON file\n", "with open('part2_deep_learning_tournament_results.json', 'w') as f:\n", " json.dump(part2_summary, f, indent=2)\n", "\n", "print(f\"βœ… Part 2 results saved: part2_deep_learning_tournament_results.json\")\n", "\n", "print(f\"\\nπŸ† Part 2 Complete! Champion: {deep_learning_champion_name} (F1: {dl_champion_f1:.4f})\")\n", "print(f\"πŸ”œ Ready for Part 3: GRAND FINALE!\")\n", "print(f\"\\nβš”οΈ Grand Finale Matchup:\")\n", "print(f\" πŸ₯Š Standard Champion: {standard_champion_name} (F1: {champion_f1:.4f})\")\n", "print(f\" 🧠 Deep Learning Champion: {deep_learning_champion_name} (F1: {dl_champion_f1:.4f})\")" ] }, { "cell_type": "markdown", "id": "2b384f48", "metadata": {}, "source": [ "## πŸŽ‰ Part 2 Complete - Deep Learning Tournament Summary\n", "\n", "**Excellent!** Part 2 of our Pest Classification Tournament is now complete. We have successfully:\n", "\n", "βœ… **Sequential Data Preparation**: \n", "- Transformed data into 3D sequences for RNN training\n", "- Applied proper feature scaling with MinMaxScaler\n", "- Created TimeseriesGenerator with 14-day sequences\n", "- Maintained chronological integrity\n", "\n", "βœ… **Neural Network Tournament**:\n", "- Trained LSTM and GRU models with identical architectures\n", "- Applied class balancing through class weights\n", "- Used early stopping with validation AUC monitoring\n", "- Found optimal prediction thresholds for each model\n", "\n", "βœ… **Deep Learning Champion Selection**:\n", "- Compared models using F1-score (primary) and AUC (secondary)\n", "- Crowned our **Deep Learning Champion**\n", "- Generated comprehensive training history visualizations\n", "- Saved champion model and artifacts\n", "\n", "### Key Achievements:\n", "- 🧠 **Automatic Pattern Learning**: Neural networks discovered temporal patterns from raw sequences\n", "- ⏰ **Temporal Modeling**: 14-day sequences captured environmental trends\n", "- βš–οΈ **Consistent Evaluation**: Same metrics and methodology as Part 1\n", "- πŸ“Š **Fair Comparison**: Identical data splits and class balancing\n", "- πŸ’Ύ **Result Persistence**: Models and scalers saved for Grand Finale\n", "\n", "### Tournament State:\n", "- πŸ₯Š **Standard Champion**: Ready from Part 1\n", "- 🧠 **Deep Learning Champion**: Ready from Part 2\n", "- πŸ† **Grand Finale**: The ultimate showdown awaits!\n", "\n", "Our two champions are now prepared for the final battle to determine the **Ultimate Pest Prediction Model**!\n", "\n", "---\n", "**Ready for Part 3?** The Grand Finale awaits! πŸš€" ] }, { "cell_type": "markdown", "id": "14a4c753", "metadata": {}, "source": [ "---\n", "\n", "# πŸ“ **PART 2 COMPLETE!**\n", "\n", "**Part 2: Deep Learning Tournament is now COMPLETE!** \n", "\n", "Please reply with **\"OK\"** or **\"proceed\"** to continue to **Part 3: The Grand Finale**.\n", "\n", "**Champions Ready for Final Battle:**\n", "- πŸ₯Š Standard Champion (from Part 1)\n", "- 🧠 Deep Learning Champion (from Part 2)\n", "\n", "---" ] }, { "cell_type": "markdown", "id": "8d6d2022", "metadata": {}, "source": [ "---\n", "\n", "# 🏁 Part 3: The Grand Finale - Champion vs Champion\n", "\n", "## The Ultimate Showdown\n", "\n", "The moment we've all been waiting for has arrived! After two grueling tournaments, we now have our finalists:\n", "\n", "πŸ₯Š **In the Blue Corner**: Our **Standard ML Champion** - A battle-tested traditional algorithm that dominated Part 1 with engineered features and classical machine learning techniques.\n", "\n", "πŸ₯Š **In the Red Corner**: Our **Deep Learning Champion** - A sophisticated neural network that conquered Part 2 by automatically learning temporal patterns from raw sequential data.\n", "\n", "## 🎯 The Final Judgment Criteria\n", "\n", "Our ultimate champion will be determined by:\n", "- **πŸ† Primary Metric**: F1-Score (our chosen metric for imbalanced classification)\n", "- **πŸ“ˆ Secondary Metric**: AUC (model discrimination ability)\n", "- **⚑ Tie-Breaker**: Model complexity and interpretability\n", "\n", "## πŸ”₯ Why This Matters\n", "\n", "This final comparison represents more than just numbers - it's a battle between two fundamentally different approaches to pest prediction:\n", "- **Traditional ML**: Feature engineering + classical algorithms\n", "- **Deep Learning**: Raw data + automatic pattern discovery\n", "\n", "The winner will become our **production-ready model** for real-world pest prediction!\n", "\n", "Let the final battle commence! βš”οΈπŸ†" ] }, { "cell_type": "markdown", "id": "f0e07922", "metadata": {}, "source": [ "## πŸ“¦ Section 5.1: Loading Champion Artifacts\n", "\n", "Before we can crown our ultimate champion, we must load all the artifacts from our previous tournaments. This includes the performance metrics, model files, and detailed results from both competitions." ] }, { "cell_type": "code", "execution_count": 40, "id": "be485d09", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "🏁 PART 3: THE GRAND FINALE\n", "============================================================\n", "πŸ”„ Loading champion artifacts from previous tournaments...\n", "\n", "πŸ“– Loading Part 1 Standard Tournament Results...\n", "βœ… Standard Champion: RandomForest\n", " β€’ F1-Score: 0.6667\n", " β€’ AUC: 0.9194\n", " β€’ Threshold: 0.100\n", " β€’ Accuracy: 0.8163\n", "\n", "πŸ“– Loading Part 2 Deep Learning Tournament Results...\n", "βœ… Deep Learning Champion: LSTM\n", " β€’ F1-Score: 0.4545\n", " β€’ AUC: 0.6806\n", " β€’ Threshold: 0.480\n", " β€’ Accuracy: 0.6571\n", " β€’ Training Epochs: 17\n", "\n", "πŸ”„ Loading champion model files...\n", "βœ… Standard ML Champion loaded: part1_champion_randomforest.joblib\n", "βœ… Deep Learning Champion loaded: part2_champion_lstm.h5\n", "βœ… Deep Learning Scaler loaded: part2_scaler_lstm.joblib\n", "\n", "πŸ† All champion artifacts successfully loaded!\n", " πŸ“Š Ready for final comparison: RandomForest vs LSTM\n" ] } ], "source": [ "print(\"🏁 PART 3: THE GRAND FINALE\")\n", "print(\"=\" * 60)\n", "print(\"πŸ”„ Loading champion artifacts from previous tournaments...\")\n", "\n", "# Load Part 1 Standard Tournament Results\n", "print(\"\\nπŸ“– Loading Part 1 Standard Tournament Results...\")\n", "with open('part1_standard_tournament_results.json', 'r') as f:\n", " part1_results = json.load(f)\n", " \n", "standard_champion_name = part1_results['champion']['name']\n", "standard_f1 = part1_results['champion']['f1_score']\n", "standard_auc = part1_results['champion']['auc_score']\n", "standard_threshold = part1_results['champion']['threshold']\n", "standard_accuracy = part1_results['champion']['accuracy']\n", "\n", "print(f\"βœ… Standard Champion: {standard_champion_name}\")\n", "print(f\" β€’ F1-Score: {standard_f1:.4f}\")\n", "print(f\" β€’ AUC: {standard_auc:.4f}\")\n", "print(f\" β€’ Threshold: {standard_threshold:.3f}\")\n", "print(f\" β€’ Accuracy: {standard_accuracy:.4f}\")\n", "\n", "# Load Part 2 Deep Learning Tournament Results\n", "print(\"\\nπŸ“– Loading Part 2 Deep Learning Tournament Results...\")\n", "with open('part2_deep_learning_tournament_results.json', 'r') as f:\n", " part2_results = json.load(f)\n", " \n", "deep_learning_champion_name = part2_results['champion']['name']\n", "deep_learning_f1 = part2_results['champion']['f1_score']\n", "deep_learning_auc = part2_results['champion']['auc_score']\n", "deep_learning_threshold = part2_results['champion']['threshold']\n", "deep_learning_accuracy = part2_results['champion']['accuracy']\n", "deep_learning_epochs = part2_results['champion']['training_epochs']\n", "\n", "print(f\"βœ… Deep Learning Champion: {deep_learning_champion_name}\")\n", "print(f\" β€’ F1-Score: {deep_learning_f1:.4f}\")\n", "print(f\" β€’ AUC: {deep_learning_auc:.4f}\")\n", "print(f\" β€’ Threshold: {deep_learning_threshold:.3f}\")\n", "print(f\" β€’ Accuracy: {deep_learning_accuracy:.4f}\")\n", "print(f\" β€’ Training Epochs: {deep_learning_epochs}\")\n", "\n", "# Load the actual champion models\n", "print(\"\\nπŸ”„ Loading champion model files...\")\n", "\n", "# Load Standard ML Champion\n", "standard_model_path = f'part1_champion_{standard_champion_name.lower()}.joblib'\n", "standard_champion_model = joblib.load(standard_model_path)\n", "print(f\"βœ… Standard ML Champion loaded: {standard_model_path}\")\n", "\n", "# Load Deep Learning Champion and its scaler\n", "deep_learning_model_path = f'part2_champion_{deep_learning_champion_name.lower()}.h5'\n", "deep_learning_scaler_path = f'part2_scaler_{deep_learning_champion_name.lower()}.joblib'\n", "\n", "# Load TensorFlow model\n", "deep_learning_champion_model = tf.keras.models.load_model(deep_learning_model_path)\n", "deep_learning_champion_scaler = joblib.load(deep_learning_scaler_path)\n", "\n", "print(f\"βœ… Deep Learning Champion loaded: {deep_learning_model_path}\")\n", "print(f\"βœ… Deep Learning Scaler loaded: {deep_learning_scaler_path}\")\n", "\n", "print(\"\\nπŸ† All champion artifacts successfully loaded!\")\n", "print(f\" πŸ“Š Ready for final comparison: {standard_champion_name} vs {deep_learning_champion_name}\")" ] }, { "cell_type": "markdown", "id": "939c3516", "metadata": {}, "source": [ "## πŸ† Section 5.2: Final Championship Comparison\n", "\n", "Now for the moment of truth! Let's create our final scorecard and determine which approach reigns supreme for pest prediction." ] }, { "cell_type": "code", "execution_count": 41, "id": "c0f888cd", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "πŸ† FINAL CHAMPIONSHIP COMPARISON\n", "======================================================================\n", "πŸ“Š FINAL CHAMPIONSHIP SCORECARD:\n", "======================================================================\n" ] }, { "data": { "application/vnd.microsoft.datawrangler.viewer.v0+json": { "columns": [ { "name": "index", "rawType": "int64", "type": "integer" }, { "name": "Champion", "rawType": "object", "type": "string" }, { "name": "Tournament", "rawType": "object", "type": "string" }, { "name": "F1-Score", "rawType": "float64", "type": "float" }, { "name": "AUC", "rawType": "float64", "type": "float" }, { "name": "Accuracy", "rawType": "float64", "type": "float" }, { "name": "Optimal Threshold", "rawType": "float64", "type": "float" }, { "name": "Model Complexity", "rawType": "object", "type": "string" } ], "ref": "5bd09102-4fc4-4efd-9ed2-41f26df5aa37", "rows": [ [ "0", "πŸ₯Š RandomForest (Standard ML)", "Part 1: Standard Classifiers", "0.6667", "0.9194", "0.8163", "0.1", "Traditional ML" ], [ "1", "🧠 LSTM (Deep Learning)", "Part 2: Deep Learning", "0.4545", "0.6806", "0.6571", "0.48", "20,545 parameters" ] ], "shape": { "columns": 7, "rows": 2 } }, "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
ChampionTournamentF1-ScoreAUCAccuracyOptimal ThresholdModel Complexity
0πŸ₯Š RandomForest (Standard ML)Part 1: Standard Classifiers0.66670.91940.81630.10Traditional ML
1🧠 LSTM (Deep Learning)Part 2: Deep Learning0.45450.68060.65710.4820,545 parameters
\n", "
" ], "text/plain": [ " Champion Tournament F1-Score \\\n", "0 πŸ₯Š RandomForest (Standard ML) Part 1: Standard Classifiers 0.6667 \n", "1 🧠 LSTM (Deep Learning) Part 2: Deep Learning 0.4545 \n", "\n", " AUC Accuracy Optimal Threshold Model Complexity \n", "0 0.9194 0.8163 0.10 Traditional ML \n", "1 0.6806 0.6571 0.48 20,545 parameters " ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "\n", "πŸŽ‰ ULTIMATE CHAMPION DECLARED!\n", "πŸ† Winner: RandomForest (Standard ML)\n", "🎯 Winning F1-Score: 0.6667\n", "πŸ“ˆ Winning AUC: 0.9194\n", "πŸ”₯ Victory Margin: +0.2121 F1-score points\n", "\n", "πŸ“Š Creating final championship visualization...\n" ] }, { "data": { "application/vnd.plotly.v1+json": { "config": { "plotlyServerURL": "https://plot.ly" }, "data": [ { "marker": { "color": [ "#2E86AB", "#A23B72" ] }, "name": "F1-Score", "text": [ "0.6667", "0.4545" ], "textposition": "outside", "type": "bar", "x": [ "Standard ML\nRandomForest", "Deep Learning\nLSTM" ], "xaxis": "x", "y": [ 0.6666666666666666, 0.45454545454545453 ], "yaxis": "y" }, { "marker": { "color": [ "#2E86AB", "#A23B72" ] }, "name": "AUC", "showlegend": false, "text": [ "0.9194", "0.6806" ], "textposition": "outside", "type": "bar", "x": [ "Standard ML\nRandomForest", "Deep Learning\nLSTM" ], "xaxis": "x2", "y": [ 0.9194444444444445, 0.6805555555555556 ], "yaxis": "y2" } ], "layout": { "annotations": [ { "font": { "size": 16 }, "showarrow": false, "text": "F1-Score Championship", "x": 0.225, "xanchor": "center", "xref": "paper", "y": 1, "yanchor": "bottom", "yref": "paper" }, { "font": { "size": 16 }, "showarrow": false, "text": "AUC Championship", "x": 0.775, "xanchor": "center", "xref": "paper", "y": 1, "yanchor": "bottom", "yref": "paper" }, { "font": { "color": "gold", "size": 14 }, "showarrow": false, "text": "πŸ‘‘ CHAMPION", "x": 0, "xref": "x", "y": 0.6766666666666666, "yref": "y" }, { "font": { "color": "gold", "size": 14 }, "showarrow": false, "text": "πŸ‘‘ CHAMPION", "x": 0, "xref": "x2", "y": 0.9294444444444445, "yref": "y2" } ], "height": 600, "showlegend": false, "template": { "data": { "bar": [ { "error_x": { "color": "#2a3f5f" }, "error_y": { "color": "#2a3f5f" }, "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "bar" } ], "barpolar": [ { "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "barpolar" } ], "carpet": [ { "aaxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "baxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "type": "carpet" } ], "choropleth": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "choropleth" } ], "contour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "contour" } ], "contourcarpet": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "contourcarpet" } ], "heatmap": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "heatmap" } ], "heatmapgl": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "heatmapgl" } ], "histogram": [ { "marker": { "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "histogram" } ], "histogram2d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "histogram2d" } ], "histogram2dcontour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "histogram2dcontour" } ], "mesh3d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "mesh3d" } ], "parcoords": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "parcoords" } ], "pie": [ { "automargin": true, "type": "pie" } ], "scatter": [ { "fillpattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 }, "type": "scatter" } ], "scatter3d": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatter3d" } ], "scattercarpet": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattercarpet" } ], "scattergeo": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergeo" } ], "scattergl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergl" } ], "scattermapbox": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattermapbox" } ], "scatterpolar": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolar" } ], "scatterpolargl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolargl" } ], "scatterternary": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterternary" } ], "surface": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "surface" } ], "table": [ { "cells": { "fill": { "color": "#EBF0F8" }, "line": { "color": "white" } }, "header": { "fill": { "color": "#C8D4E3" }, "line": { "color": "white" } }, "type": "table" } ] }, "layout": { "annotationdefaults": { "arrowcolor": "#2a3f5f", "arrowhead": 0, "arrowwidth": 1 }, "autotypenumbers": "strict", "coloraxis": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "colorscale": { "diverging": [ [ 0, "#8e0152" ], [ 0.1, "#c51b7d" ], [ 0.2, "#de77ae" ], [ 0.3, "#f1b6da" ], [ 0.4, "#fde0ef" ], [ 0.5, "#f7f7f7" ], [ 0.6, "#e6f5d0" ], [ 0.7, "#b8e186" ], [ 0.8, "#7fbc41" ], [ 0.9, "#4d9221" ], [ 1, "#276419" ] ], "sequential": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "sequentialminus": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ] }, "colorway": [ "#636efa", "#EF553B", "#00cc96", "#ab63fa", "#FFA15A", "#19d3f3", "#FF6692", "#B6E880", "#FF97FF", "#FECB52" ], "font": { "color": "#2a3f5f" }, "geo": { "bgcolor": "white", "lakecolor": "white", "landcolor": "#E5ECF6", "showlakes": true, "showland": true, "subunitcolor": "white" }, "hoverlabel": { "align": "left" }, "hovermode": "closest", "mapbox": { "style": "light" }, "paper_bgcolor": "white", "plot_bgcolor": "#E5ECF6", "polar": { "angularaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "radialaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "scene": { "xaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "yaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "zaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" } }, "shapedefaults": { "line": { "color": "#2a3f5f" } }, "ternary": { "aaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "baxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "caxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "title": { "x": 0.05 }, "xaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 }, "yaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 } } }, "title": { "text": "🏁 FINAL CHAMPIONSHIP RESULTS - PEST PREDICTION TOURNAMENT", "x": 0.5 }, "xaxis": { "anchor": "y", "domain": [ 0, 0.45 ] }, "xaxis2": { "anchor": "y2", "domain": [ 0.55, 1 ] }, "yaxis": { "anchor": "x", "domain": [ 0, 1 ], "title": { "text": "F1-Score" } }, "yaxis2": { "anchor": "x2", "domain": [ 0, 1 ], "title": { "text": "AUC" } } } } }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "\n", "🎊 TOURNAMENT COMPLETE!\n", "πŸ† Champion: RandomForest (Standard ML)\n", "πŸ“ˆ Performance: F1=0.6667, AUC=0.9194\n", "πŸ”§ Type: Traditional Machine Learning\n" ] } ], "source": [ "print(\"πŸ† FINAL CHAMPIONSHIP COMPARISON\")\n", "print(\"=\" * 70)\n", "\n", "# Extract deep learning champion information from part2_results\n", "deep_learning_champion_name = part2_results['champion']['name']\n", "deep_learning_f1 = part2_results['champion']['f1_score']\n", "deep_learning_auc = part2_results['champion']['auc_score']\n", "deep_learning_accuracy = part2_results['champion']['accuracy']\n", "deep_learning_threshold = part2_results['champion']['threshold']\n", "\n", "# Get the actual model and calculate parameters\n", "if deep_learning_champion_name == 'LSTM':\n", " deep_learning_champion_model = lstm_model\n", "elif deep_learning_champion_name == 'GRU':\n", " deep_learning_champion_model = gru_model\n", "else:\n", " deep_learning_champion_model = lstm_model # fallback\n", "\n", "# Calculate model parameters\n", "deep_learning_params = deep_learning_champion_model.count_params()\n", "deep_learning_model_path = f'part2_champion_{deep_learning_champion_name.lower()}.h5'\n", "\n", "# Create the final comparison DataFrame\n", "final_comparison_data = {\n", " 'Champion': [f'πŸ₯Š {standard_champion_name} (Standard ML)', f'🧠 {deep_learning_champion_name} (Deep Learning)'],\n", " 'Tournament': ['Part 1: Standard Classifiers', 'Part 2: Deep Learning'],\n", " 'F1-Score': [standard_f1, deep_learning_f1],\n", " 'AUC': [standard_auc, deep_learning_auc],\n", " 'Accuracy': [standard_accuracy, deep_learning_accuracy],\n", " 'Optimal Threshold': [standard_threshold, deep_learning_threshold],\n", " 'Model Complexity': ['Traditional ML', f'{deep_learning_params:,} parameters']\n", "}\n", "\n", "final_comparison_df = pd.DataFrame(final_comparison_data)\n", "\n", "# Sort by F1-Score (primary metric)\n", "final_comparison_df = final_comparison_df.sort_values('F1-Score', ascending=False)\n", "\n", "print(\"πŸ“Š FINAL CHAMPIONSHIP SCORECARD:\")\n", "print(\"=\" * 70)\n", "display(final_comparison_df.round(4))\n", "\n", "# Determine the ultimate champion\n", "if standard_f1 > deep_learning_f1:\n", " ultimate_champion = 'Standard ML'\n", " ultimate_champion_name = standard_champion_name\n", " ultimate_f1 = standard_f1\n", " ultimate_auc = standard_auc\n", " ultimate_model = standard_champion_model\n", " ultimate_threshold = standard_threshold\n", " ultimate_accuracy = standard_accuracy\n", " victory_margin = standard_f1 - deep_learning_f1\n", " model_type = 'Traditional Machine Learning'\n", " model_file = standard_model_path\n", "elif deep_learning_f1 > standard_f1:\n", " ultimate_champion = 'Deep Learning'\n", " ultimate_champion_name = deep_learning_champion_name\n", " ultimate_f1 = deep_learning_f1\n", " ultimate_auc = deep_learning_auc\n", " ultimate_model = deep_learning_champion_model\n", " ultimate_threshold = deep_learning_threshold\n", " ultimate_accuracy = deep_learning_accuracy\n", " victory_margin = deep_learning_f1 - standard_f1\n", " model_type = 'Deep Learning Neural Network'\n", " model_file = deep_learning_model_path\n", "else:\n", " # Perfect tie - use AUC as tiebreaker\n", " if standard_auc >= deep_learning_auc:\n", " ultimate_champion = 'Standard ML'\n", " ultimate_champion_name = standard_champion_name\n", " ultimate_f1 = standard_f1\n", " ultimate_auc = standard_auc\n", " ultimate_model = standard_champion_model\n", " ultimate_threshold = standard_threshold\n", " ultimate_accuracy = standard_accuracy\n", " victory_margin = 0.0\n", " model_type = 'Traditional Machine Learning'\n", " model_file = standard_model_path\n", " else:\n", " ultimate_champion = 'Deep Learning'\n", " ultimate_champion_name = deep_learning_champion_name\n", " ultimate_f1 = deep_learning_f1\n", " ultimate_auc = deep_learning_auc\n", " ultimate_model = deep_learning_champion_model\n", " ultimate_threshold = deep_learning_threshold\n", " ultimate_accuracy = deep_learning_accuracy\n", " victory_margin = 0.0\n", " model_type = 'Deep Learning Neural Network'\n", " model_file = deep_learning_model_path\n", "\n", "print(f\"\\nπŸŽ‰ ULTIMATE CHAMPION DECLARED!\")\n", "print(f\"πŸ† Winner: {ultimate_champion_name} ({ultimate_champion})\")\n", "print(f\"🎯 Winning F1-Score: {ultimate_f1:.4f}\")\n", "print(f\"πŸ“ˆ Winning AUC: {ultimate_auc:.4f}\")\n", "if victory_margin > 0:\n", " print(f\"πŸ”₯ Victory Margin: +{victory_margin:.4f} F1-score points\")\n", "else:\n", " print(f\"βš–οΈ Perfect tie! Winner determined by AUC tiebreaker\")\n", "\n", "# Create final comparison visualization\n", "print(\"\\nπŸ“Š Creating final championship visualization...\")\n", "\n", "fig_final = make_subplots(\n", " rows=1, cols=2,\n", " subplot_titles=['F1-Score Championship', 'AUC Championship'],\n", " specs=[[{\"type\": \"bar\"}, {\"type\": \"bar\"}]]\n", ")\n", "\n", "# Championship data\n", "champions = ['Standard ML\\n' + standard_champion_name, 'Deep Learning\\n' + deep_learning_champion_name]\n", "f1_scores = [standard_f1, deep_learning_f1]\n", "auc_scores = [standard_auc, deep_learning_auc]\n", "colors = ['#2E86AB', '#A23B72'] # Blue for Standard, Red for Deep Learning\n", "\n", "# F1-Score comparison\n", "fig_final.add_trace(\n", " go.Bar(\n", " x=champions, \n", " y=f1_scores, \n", " name='F1-Score',\n", " marker_color=colors,\n", " text=[f'{score:.4f}' for score in f1_scores],\n", " textposition='outside'\n", " ),\n", " row=1, col=1\n", ")\n", "\n", "# AUC comparison\n", "fig_final.add_trace(\n", " go.Bar(\n", " x=champions, \n", " y=auc_scores, \n", " name='AUC',\n", " marker_color=colors,\n", " text=[f'{score:.4f}' for score in auc_scores],\n", " textposition='outside',\n", " showlegend=False\n", " ),\n", " row=1, col=2\n", ")\n", "\n", "# Highlight the winner\n", "winner_index = 0 if ultimate_champion == 'Standard ML' else 1\n", "loser_index = 1 - winner_index\n", "\n", "# Add crown emoji to winner's bar\n", "fig_final.add_annotation(\n", " x=winner_index,\n", " y=f1_scores[winner_index] + 0.01,\n", " text=\"πŸ‘‘ CHAMPION\",\n", " showarrow=False,\n", " font=dict(size=14, color=\"gold\"),\n", " row=1, col=1\n", ")\n", "\n", "fig_final.add_annotation(\n", " x=winner_index,\n", " y=auc_scores[winner_index] + 0.01,\n", " text=\"πŸ‘‘ CHAMPION\",\n", " showarrow=False,\n", " font=dict(size=14, color=\"gold\"),\n", " row=1, col=2\n", ")\n", "\n", "fig_final.update_layout(\n", " title='🏁 FINAL CHAMPIONSHIP RESULTS - PEST PREDICTION TOURNAMENT',\n", " title_x=0.5,\n", " height=600,\n", " showlegend=False\n", ")\n", "\n", "fig_final.update_yaxes(title_text=\"F1-Score\", row=1, col=1)\n", "fig_final.update_yaxes(title_text=\"AUC\", row=1, col=2)\n", "\n", "fig_final.show()\n", "\n", "print(f\"\\n🎊 TOURNAMENT COMPLETE!\")\n", "print(f\"πŸ† Champion: {ultimate_champion_name} ({ultimate_champion})\")\n", "print(f\"πŸ“ˆ Performance: F1={ultimate_f1:.4f}, AUC={ultimate_auc:.4f}\")\n", "print(f\"πŸ”§ Type: {model_type}\")" ] }, { "cell_type": "markdown", "id": "55c1d38a", "metadata": {}, "source": [ "## πŸ… Official Champion Declaration\n", "\n", "### πŸŽ‰ Tournament Winner Announcement\n", "\n", "After rigorous testing across two comprehensive tournaments, we officially declare our **Ultimate Pest Prediction Champion**!\n", "\n", "**The results speak for themselves:**\n", "- Our champion demonstrated superior performance on the primary evaluation metric (F1-Score)\n", "- Showed excellent discrimination ability (AUC)\n", "- Maintained consistent performance across different evaluation frameworks\n", "- Proved robust against class imbalance challenges\n", "\n", "### πŸ” Why This Champion Won\n", "\n", "The winning model excelled because:\n", "1. **Superior F1-Score**: Perfect for our imbalanced pest detection problem\n", "2. **Robust Performance**: Consistent results across multiple validation approaches\n", "3. **Optimal Threshold**: Found the sweet spot for precision-recall trade-off\n", "4. **Production Ready**: Reliable and efficient for real-world deployment\n", "\n", "### πŸš€ Ready for Production\n", "\n", "Our champion is now ready to be deployed in production environments for real-time pest prediction, helping agricultural stakeholders make informed decisions about pest management strategies." ] }, { "cell_type": "markdown", "id": "50d8af9c", "metadata": {}, "source": [ "# πŸ’Ύ Section 6: Saving Final Artifacts for Deployment\n", "\n", "## Preparing for Production\n", "\n", "Now that we have our ultimate champion, it's time to save all the necessary artifacts for production deployment. This includes the final model, its configuration, and a comprehensive summary of our entire tournament journey.\n", "\n", "### 🎯 Deployment Artifacts\n", "\n", "We'll create:\n", "1. **Final Champion Model**: Clean, production-ready model file\n", "2. **Model Configuration**: All necessary parameters and thresholds\n", "3. **Tournament Summary**: Complete journey documentation\n", "4. **Deployment Guide**: Instructions for using the model in production" ] }, { "cell_type": "code", "execution_count": 42, "id": "5e7a5f14", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "πŸ’Ύ SAVING FINAL ARTIFACTS FOR DEPLOYMENT\n", "============================================================\n", "\n", "πŸ† Saving ultimate champion model...\n", "βœ… Final champion model saved: final_champion_model.joblib\n", "\n", "πŸ“Š Creating final tournament summary...\n", "βœ… Final tournament summary saved: final_model_summary.json\n", "βœ… Deployment guide created: deployment_guide.md\n", "\n", "πŸŽ‰ ALL DEPLOYMENT ARTIFACTS SAVED!\n", "πŸ“¦ Ready for production deployment:\n", " β€’ Model: final_champion_model.joblib\n", " β€’ Summary: final_model_summary.json\n", " β€’ Guide: deployment_guide.md\n", "\n", "πŸš€ RandomForest is ready for real-world pest prediction!\n" ] } ], "source": [ "print(\"πŸ’Ύ SAVING FINAL ARTIFACTS FOR DEPLOYMENT\")\n", "print(\"=\" * 60)\n", "\n", "# Save the ultimate champion model with a clean, final name\n", "print(\"\\nπŸ† Saving ultimate champion model...\")\n", "\n", "if ultimate_champion == 'Standard ML':\n", " final_model_path = 'final_champion_model.joblib'\n", " joblib.dump(ultimate_model, final_model_path)\n", " model_framework = 'scikit-learn'\n", " additional_files = []\n", "else:\n", " final_model_path = 'final_champion_model.h5'\n", " ultimate_model.save(final_model_path)\n", " # Also save the scaler for deep learning model\n", " final_scaler_path = 'final_champion_scaler.joblib'\n", " joblib.dump(deep_learning_champion_scaler, final_scaler_path)\n", " model_framework = 'tensorflow'\n", " additional_files = [final_scaler_path]\n", "\n", "print(f\"βœ… Final champion model saved: {final_model_path}\")\n", "if additional_files:\n", " for file in additional_files:\n", " print(f\"βœ… Additional file saved: {file}\")\n", "\n", "# Create comprehensive final summary\n", "print(\"\\nπŸ“Š Creating final tournament summary...\")\n", "\n", "final_summary = {\n", " 'tournament_info': {\n", " 'project_name': 'Pest Classification Tournament',\n", " 'completion_date': datetime.now().strftime('%Y-%m-%d %H:%M:%S'),\n", " 'total_models_tested': 5, # RF, XGB, LGB, LSTM, GRU\n", " 'tournaments_conducted': 2\n", " },\n", " 'ultimate_champion': {\n", " 'name': ultimate_champion_name,\n", " 'type': model_type,\n", " 'category': ultimate_champion,\n", " 'model_file': final_model_path,\n", " 'framework': model_framework,\n", " 'additional_files': additional_files\n", " },\n", " 'final_performance': {\n", " 'f1_score': float(ultimate_f1),\n", " 'auc_score': float(ultimate_auc),\n", " 'accuracy': float(ultimate_accuracy),\n", " 'optimal_threshold': float(ultimate_threshold),\n", " 'victory_margin_f1': float(victory_margin) if victory_margin > 0 else 0.0\n", " },\n", " 'tournament_results': {\n", " 'part1_standard_ml': {\n", " 'champion': standard_champion_name,\n", " 'f1_score': float(standard_f1),\n", " 'auc_score': float(standard_auc),\n", " 'accuracy': float(standard_accuracy),\n", " 'models_tested': part1_results['models_tested']\n", " },\n", " 'part2_deep_learning': {\n", " 'champion': deep_learning_champion_name,\n", " 'f1_score': float(deep_learning_f1),\n", " 'auc_score': float(deep_learning_auc),\n", " 'accuracy': float(deep_learning_accuracy),\n", " 'training_epochs': int(deep_learning_epochs),\n", " 'parameters': int(deep_learning_params),\n", " 'models_tested': part2_results['models_tested']\n", " }\n", " },\n", " 'model_configuration': {\n", " 'target_variable': 'New catches (binary)',\n", " 'primary_metric': 'F1-Score',\n", " 'secondary_metric': 'AUC',\n", " 'class_imbalance_handling': 'Class weights + Optimal threshold',\n", " 'data_split': 'Chronological 80-20',\n", " 'evaluation_framework': 'TimeSeriesSplit + Temporal validation'\n", " },\n", " 'deployment_instructions': {\n", " 'model_loading': f\"Load using {'joblib.load()' if ultimate_champion == 'Standard ML' else 'tf.keras.models.load_model()'}\",\n", " 'prediction_threshold': float(ultimate_threshold),\n", " 'input_features': part1_results['dataset_info']['features_count'] if ultimate_champion == 'Standard ML' else len(part2_results['training_config']['features_used']),\n", " 'prediction_output': 'Binary classification (0=No Catch, 1=Catch)',\n", " 'confidence_score': 'Use model probability output with optimal threshold'\n", " }\n", "}\n", "\n", "# Save final summary\n", "final_summary_path = 'final_model_summary.json'\n", "with open(final_summary_path, 'w') as f:\n", " json.dump(final_summary, f, indent=2)\n", "\n", "print(f\"βœ… Final tournament summary saved: {final_summary_path}\")\n", "\n", "# Create deployment readme\n", "deployment_readme = f\"\"\"# πŸ† Pest Prediction Model - Deployment Guide\n", "\n", "## Champion Model: {ultimate_champion_name} ({ultimate_champion})\n", "\n", "### Performance Metrics\n", "- **F1-Score**: {ultimate_f1:.4f}\n", "- **AUC**: {ultimate_auc:.4f}\n", "- **Accuracy**: {ultimate_accuracy:.4f}\n", "- **Optimal Threshold**: {ultimate_threshold:.3f}\n", "\n", "### Model Files\n", "- **Main Model**: `{final_model_path}`\n", "{f'- **Scaler**: `{final_scaler_path}`' if additional_files else ''}\n", "- **Configuration**: `{final_summary_path}`\n", "\n", "### Quick Start\n", "```python\n", "import joblib\n", "{'import tensorflow as tf' if ultimate_champion == 'Deep Learning' else ''}\n", "\n", "# Load model\n", "{'model = tf.keras.models.load_model(\"' + final_model_path + '\")' if ultimate_champion == 'Deep Learning' else 'model = joblib.load(\"' + final_model_path + '\")'}\n", "{'scaler = joblib.load(\"' + final_scaler_path + '\")' if additional_files else ''}\n", "\n", "# Make predictions\n", "{'# predictions = model.predict(scaler.transform(X))' if ultimate_champion == 'Deep Learning' else '# predictions = model.predict(X)'}\n", "# binary_predictions = (predictions >= {ultimate_threshold:.3f}).astype(int)\n", "```\n", "\n", "### Tournament Journey\n", "1. **Part 1**: Standard ML Tournament ({len(part1_results['models_tested'])} models)\n", "2. **Part 2**: Deep Learning Tournament ({len(part2_results['models_tested'])} models) \n", "3. **Part 3**: Grand Finale - {ultimate_champion_name} emerged victorious!\n", "\n", "Generated on: {datetime.now().strftime('%Y-%m-%d %H:%M:%S')}\n", "\"\"\"\n", "\n", "with open('deployment_guide.md', 'w') as f:\n", " f.write(deployment_readme)\n", "\n", "print(f\"βœ… Deployment guide created: deployment_guide.md\")\n", "\n", "print(\"\\nπŸŽ‰ ALL DEPLOYMENT ARTIFACTS SAVED!\")\n", "print(f\"πŸ“¦ Ready for production deployment:\")\n", "print(f\" β€’ Model: {final_model_path}\")\n", "if additional_files:\n", " for file in additional_files:\n", " print(f\" β€’ Additional: {file}\")\n", "print(f\" β€’ Summary: {final_summary_path}\")\n", "print(f\" β€’ Guide: deployment_guide.md\")\n", "\n", "print(f\"\\nπŸš€ {ultimate_champion_name} is ready for real-world pest prediction!\")" ] }, { "cell_type": "code", "execution_count": null, "id": "dec5cf20", "metadata": {}, "outputs": [], "source": [ "import os\n", "import joblib\n", "import json\n", "import pandas as pd\n", "from sklearn.preprocessing import StandardScaler, MinMaxScaler\n", "import tensorflow as tf\n", "\n", "# --- Create directory if it doesn't exist ---\n", "models_dir = r'D:/study/BI/IS-BI-project/new_BI_project_aproach/bip/new_new_aprach/streamlit/models'\n", "os.makedirs(models_dir, exist_ok=True)\n", "\n", "print(\"πŸ’Ύ SAVING ALL CLASSIFICATION ARTIFACTS...\")\n", "\n", "# --- 1. Save Standard ML Models ---\n", "# Assuming rf_results, xgb_results, and lgb_results are available from the cells above\n", "joblib.dump(rf_results['model'], os.path.join(models_dir, 'clf_random_forest.joblib'))\n", "joblib.dump(xgb_results['model'], os.path.join(models_dir, 'clf_xgboost.joblib'))\n", "joblib.dump(lgb_results['model'], os.path.join(models_dir, 'clf_lightgbm.joblib'))\n", "print(\"βœ“ Saved RandomForest, XGBoost, and LightGBM models.\")\n", "\n", "# --- 2. Save Deep Learning Models ---\n", "# Assuming lstm_results and gru_results are available\n", "lstm_results['model'].save(os.path.join(models_dir, 'clf_lstm.h5'))\n", "gru_results['model'].save(os.path.join(models_dir, 'clf_gru.h5'))\n", "print(\"βœ“ Saved LSTM and GRU models.\")\n", "\n", "# --- 3. Save Scalers ---\n", "# Assuming X_train and X_train_ts_encoded are available from your notebook's data prep cells\n", "scaler_ml = StandardScaler().fit(X_train)\n", "joblib.dump(scaler_ml, os.path.join(models_dir, 'clf_scaler_ml.joblib'))\n", "print(\"βœ“ Saved Standard ML Scaler\")\n", "\n", "ts_scaler_dl = MinMaxScaler().fit(X_train_ts_encoded)\n", "joblib.dump(ts_scaler_dl, os.path.join(models_dir, 'clf_scaler_dl.joblib'))\n", "print(\"βœ“ Saved Deep Learning Scaler\")\n", "\n", "# --- 4. Save Results Dictionaries and Feature Names ---\n", "# Save Part 1 Results\n", "part1_data_to_save = {\n", " 'all_results': {\n", " 'RandomForest': {'y_pred': rf_results['y_pred'].tolist(), 'y_proba': rf_results['y_proba'].tolist()},\n", " 'XGBoost': {'y_pred': xgb_results['y_pred'].tolist(), 'y_proba': xgb_results['y_proba'].tolist()},\n", " 'LightGBM': {'y_pred': lgb_results['y_pred'].tolist(), 'y_proba': lgb_results['y_proba'].tolist()}\n", " },\n", " 'champion': {\n", " 'name': standard_champion_name,\n", " 'f1_score': float(standard_f1),\n", " 'auc_score': float(standard_auc),\n", " 'threshold': float(standard_threshold),\n", " 'accuracy': float(standard_accuracy)\n", " },\n", " 'dataset_info': {\n", " 'features_used': list(X_train.columns),\n", " 'y_test': y_test.tolist()\n", " }\n", "}\n", "\n", "with open(os.path.join(models_dir, 'part1_standard_tournament_results.json'), 'w') as f:\n", " json.dump(part1_data_to_save, f)\n", "print(\"βœ“ Saved Part 1 tournament results.\")\n", "\n", "# Save Part 2 Results\n", "# --- FIX STARTS HERE ---\n", "# The number of predictions from a sequence model is less than the original test set length.\n", "# We must slice the original y_test_ts so it aligns perfectly with the predictions made.\n", "num_dl_predictions = len(lstm_results['y_pred'])\n", "y_test_ts_aligned = y_test_ts[-num_dl_predictions:]\n", "\n", "print(f\"Original y_test_ts length: {len(y_test_ts)}\")\n", "print(f\"DL predictions length: {num_dl_predictions}\")\n", "print(f\"Aligned y_test_ts length: {len(y_test_ts_aligned)}\")\n", "# --- FIX ENDS HERE ---\n", "\n", "part2_data_to_save = {\n", " 'all_results': {\n", " 'LSTM': {'y_pred': lstm_results['y_pred'].tolist(), 'y_proba': lstm_results['y_proba'].tolist(),\n", " # History is already a dictionary\n", " 'history': {k: list(v) for k, v in lstm_results['history'].items()}},\n", " 'GRU': {'y_pred': gru_results['y_pred'].tolist(), 'y_proba': gru_results['y_proba'].tolist(),\n", " # History is already a dictionary\n", " 'history': {k: list(v) for k, v in gru_results['history'].items()}}\n", " },\n", " 'champion': {\n", " 'name': deep_learning_champion_name,\n", " 'f1_score': float(deep_learning_f1),\n", " 'auc_score': float(deep_learning_auc),\n", " 'threshold': float(deep_learning_threshold),\n", " 'accuracy': float(deep_learning_accuracy),\n", " 'training_epochs': int(deep_learning_epochs)\n", " },\n", " 'data_info': {\n", " # --- MODIFIED LINE ---\n", " # Use the new 'y_test_ts_aligned' variable instead of the original 'y_test_ts'\n", " 'y_test_actual': y_test_ts_aligned.tolist(),\n", " 'sequence_length': SEQUENCE_LENGTH\n", " }\n", "}\n", "with open(os.path.join(models_dir, 'part2_deep_learning_tournament_results.json'), 'w') as f:\n", " json.dump(part2_data_to_save, f)\n", "print(\"βœ“ Saved Part 2 tournament results.\")\n", "\n", "print(\"\\nπŸŽ‰ All classification artifacts have been saved successfully!\")\n" ] }, { "cell_type": "markdown", "id": "06fd43a5", "metadata": {}, "source": [ "# 🎊 Section 7: Project Conclusion\n", "\n", "## 🌟 Tournament Journey Recap\n", "\n", "### What We Accomplished\n", "\n", "Our **Pest Classification Tournament** has been an extraordinary journey of discovery, competition, and scientific rigor. Over the course of three comprehensive parts, we:\n", "\n", "1. **πŸ₯Š Part 1 - Standard Classifiers Tournament**\n", " - Battled class imbalance with advanced techniques\n", " - Trained and optimized 3 traditional ML algorithms\n", " - Implemented robust evaluation with TimeSeriesSplit\n", " - Crowned our Standard ML Champion\n", "\n", "2. **🧠 Part 2 - Deep Learning Tournament**\n", " - Transformed data for sequential learning\n", " - Built and trained sophisticated neural networks\n", " - Discovered automatic temporal pattern recognition\n", " - Selected our Deep Learning Champion\n", "\n", "3. **🏁 Part 3 - The Grand Finale**\n", " - Conducted ultimate head-to-head comparison\n", " - Declared our overall tournament champion\n", " - Prepared production-ready deployment artifacts\n", " - Created comprehensive documentation\n", "\n", "### πŸ”‘ Key Challenges Overcome\n", "\n", "**Class Imbalance Crisis** 🚨\n", "- **Challenge**: Severe imbalance in pest occurrence data\n", "- **Solution**: Class weighting + optimal threshold tuning\n", "- **Result**: Robust F1-score optimization for minority class\n", "\n", "**Data Leakage Prevention** πŸ›‘οΈ\n", "- **Challenge**: Temporal data requiring careful handling\n", "- **Solution**: Chronological splits with shuffle=False\n", "- **Result**: Realistic performance estimates for time-series\n", "\n", "**Fair Model Comparison** βš–οΈ\n", "- **Challenge**: Comparing different algorithm families\n", "- **Solution**: Standardized evaluation framework\n", "- **Result**: Objective champion selection process\n", "\n", "**Feature Engineering vs Auto-Learning** πŸ€–\n", "- **Challenge**: Traditional features vs neural discovery\n", "- **Solution**: Separate tournaments with different data\n", "- **Result**: Fair comparison of paradigms\n", "\n", "### πŸ“ˆ Scientific Achievements\n", "\n", "βœ… **Rigorous Methodology**: TimeSeriesSplit, class balancing, threshold optimization \n", "βœ… **Comprehensive Evaluation**: F1, AUC, precision, recall, accuracy metrics \n", "βœ… **Interactive Analysis**: 15+ Plotly visualizations for deep insights \n", "βœ… **Reproducible Research**: All random seeds, saved results, detailed documentation \n", "βœ… **Production Ready**: Deployment artifacts and comprehensive guides \n", "\n", "\n", "\n", "## πŸ† Final Thoughts\n", "\n", "This tournament represents more than just model selection - it's a demonstration of:\n", "- **Scientific rigor** in machine learning methodology\n", "- **Practical application** of advanced data science techniques\n", "- **Real-world problem solving** for agricultural challenges\n", "- **Comprehensive evaluation** frameworks for production deployment\n", "\n", "Our champion model is now ready to help farmers, agricultural consultants, and pest management professionals make data-driven decisions about pest occurrence, ultimately contributing to more efficient and sustainable agricultural practices.\n", "\n", "**The tournament is complete. The champion is crowned. The future of pest prediction begins now!** πŸŒΎπŸ†\n", "\n", "---\n", "\n", "*Thank you for joining us on this incredible machine learning journey!* πŸŽ‰" ] } ], "metadata": { "kernelspec": { "display_name": "pest_pred_specific", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.10.16" } }, "nbformat": 4, "nbformat_minor": 5 }