Upload main.ipynb

Model Architecture Description

🧠 Dual CNN Architecture for Interpretable Music Genre Classification

Model 1: Batch Normalization CNN (bn_model)
Architecture Overview:
• Base Model: ResNet152V2 pre-trained on ImageNet
• Input Shape: (300, 300, 3) RGB images
• Transfer Learning: Feature extraction + fine-tuning approach

Layer Architecture:
1. Feature Extractor: ResNet152V2 (frozen initially, then fine-tuned from conv5_block3_preact_bn)
2. Global Pooling: GlobalMaxPooling2D for spatial dimension reduction
3. Dense Layer 1: 512 neurons + ReLU + L1 regularization (0.01)
4. Dropout 1: 30% dropout for regularization
5. Dense Layer 2: 256 neurons + ReLU + L1 regularization (0.01)
6. Dropout 2: 30% dropout for regularization
7. Dense Layer 3: 128 neurons + ReLU + L1 regularization (0.01)
8. Dropout 3: 30% dropout for regularization
9. Output Layer: 3 neurons + Softmax for multi-class classification

Model 2: Concept Whitening CNN (cw_model)
Architecture Overview:
• Base Model: ResNet152V2 pre-trained on ImageNet
• Novel Component: Custom Concept Whitening layer for interpretability
• Input Shape: (300, 300, 3) RGB images

Layer Architecture:
1. Feature Extractor: ResNet152V2 (frozen initially, then fine-tuned from conv5_block3_preact_cw)
2. 🔬 Concept Whitening Layer: Custom interpretability layer performing:
• Covariance matrix computation of input activations
• Singular Value Decomposition (SVD) for whitening matrix calculation
• Feature decorrelation and normalization for enhanced interpretability
3. Global Pooling: GlobalMaxPooling2D for spatial dimension reduction
4. Dense Layer 1: 512 neurons + ReLU + L1 regularization (0.01)
5. Dropout 1: 30% dropout for regularization
6. Dense Layer 2: 256 neurons + ReLU + L1 regularization (0.01)
7. Dropout 2: 30% dropout for regularization
8. Dense Layer 3: 128 neurons + ReLU + L1 regularization (0.01)
9. Dropout 3: 30% dropout for regularization
10. Output Layer: 3 neurons + Softmax for multi-class classification

Training Configuration:
Optimizer: RMSprop (learning_rate=1e-4)
Loss Function: Categorical Crossentropy
Callbacks:
• Learning Rate Scheduler (step decay)
• ReduceLROnPlateau (factor=0.6, patience=8)
• ModelCheckpoint (save best model)
• EarlyStopping (patience=5)

Key Innovations:
🎯 Concept Whitening Layer Features:
• Interpretability: Makes CNN decision-making process more transparent
• Mathematical Foundation: SVD-based feature decorrelation
• Real-time Processing: Applied during forward pass without additional inference cost
• Research Contribution: Novel approach to explainable AI in music genre classification

Model Comparison:
• Performance: Both models achieve competitive accuracy on vinyl album cover classification
• Interpretability: Concept Whitening model provides enhanced feature visualization and understanding
• Architecture: Identical except for the custom whitening layer insertion
• Research Value: Empirical comparison of accuracy vs. interpretability trade-offs in deep learning

main.ipynb

@@ -0,0 +1,840 @@
+{
+  "cells": [
+    {
+      "cell_type": "markdown",
+      "source": [
+        "# Interpretable Music Genre Classification Using Whitened Artwork Concepts\n"
+      ],
+      "metadata": {
+        "id": "3tvGZXi_DJUt"
+      }
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "bM1UyWDzs-Nt"
+      },
+      "source": [
+        "## requirements.txt\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "uHQBlPYKibjR"
+      },
+      "outputs": [],
+      "source": [
+        "# Importing libraries\n",
+        "import os\n",
+        "import random\n",
+        "import numpy as np\n",
+        "import tensorflow as tf\n",
+        "import math\n",
+        "import matplotlib.pyplot as plt\n",
+        "import shutil\n",
+        "import pandas as pd\n",
+        "from sklearn.model_selection import train_test_split\n",
+        "from tensorflow.keras import layers\n",
+        "from tensorflow.keras import models\n",
+        "from keras.preprocessing.image import ImageDataGenerator\n",
+        "from tensorflow.keras.applications import ResNet152V2\n",
+        "from tensorflow.keras import regularizers\n",
+        "from tensorflow.keras.callbacks import LearningRateScheduler\n",
+        "from keras.callbacks import ReduceLROnPlateau, ModelCheckpoint, EarlyStopping\n",
+        "import keras\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "C4MUfWGXmJVU"
+      },
+      "outputs": [],
+      "source": [
+        "# Set matplotlib to inline mode\n",
+        "%matplotlib inline\n",
+        "\n",
+        "# Seed value definition\n",
+        "def set_seed(seed=42):\n",
+        "    \"\"\"Set seed for reproducibility.\"\"\"\n",
+        "    os.environ['PYTHONHASHSEED'] = str(seed)\n",
+        "    random.seed(seed)\n",
+        "    np.random.seed(seed)\n",
+        "    tf.random.set_seed(seed)\n",
+        "\n",
+        "# Call the function with the desired seed\n",
+        "set_seed(42)\n"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "source": [
+        "## dataset_preparation.py"
+      ],
+      "metadata": {
+        "id": "yeKy3Wk3Ciyo"
+      }
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "1_Mh5OHKmJVU"
+      },
+      "outputs": [],
+      "source": [
+        "# Specify paths to the datasets\n",
+        "original_dataset_directory = \"/content/drive/MyDrive/documentos/vinyl genre class/data/For thesis\"\n",
+        "original_train_directory = os.path.join(original_dataset_directory, 'Training')\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "WrWwgb3PmJVV"
+      },
+      "outputs": [],
+      "source": [
+        "# 0 for electronic, 1 for rock, 2 for hiphop\n",
+        "# Initialize lists to store genres and image files\n",
+        "genres = []\n",
+        "img_files= []\n",
+        "\n",
+        "# List all files in the training directory\n",
+        "imgs = os.listdir(original_train_directory)\n",
+        "\n",
+        "# Load training labels from CSV\n",
+        "training_lbls = pd.read_csv('/content/drive/MyDrive/documentos/vinyl genre class/data/For thesis/training_labels.csv')\n",
+        "\n",
+        "# Loop through all labels and append corresponding genres and image files to the lists\n",
+        "for k in range(len(training_lbls)):\n",
+        "    for file in imgs:\n",
+        "        if file == training_lbls['name'][k]:\n",
+        "            genres.append(training_lbls['category'][k])\n",
+        "            img_files.append(file)\n",
+        "\n",
+        "# Create a dataframe to map each image with its label\n",
+        "df = pd.DataFrame({'image': img_files,'category':genres})\n",
+        "\n",
+        "# Display the first few rows of the dataframe\n",
+        "df.head(min(3000, len(df)))\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "5MuUMX0QWAMp"
+      },
+      "outputs": [],
+      "source": [
+        "# Define directories for train, test and validation datasets\n",
+        "base_dir = \"./data_split\"\n",
+        "train_dir = os.path.join(base_dir, 'train')\n",
+        "test_dir = os.path.join(base_dir, 'test')\n",
+        "validate_dir = os.path.join(base_dir, 'validate')\n",
+        "\n",
+        "# Create directories for train, test and validation datasets if they don't exist\n",
+        "for directory in [base_dir, train_dir, test_dir, validate_dir]:\n",
+        "    if not os.path.exists(directory):\n",
+        "        os.mkdir(directory)\n",
+        "        print(directory, \"created\")\n",
+        "\n",
+        "# Define directories for each genre in train, test and validation datasets\n",
+        "subdirectories_electronic = []\n",
+        "subdirectories_rock = []\n",
+        "subdirectories_hiphop = []\n",
+        "\n",
+        "# Create directories for each genre in train, test and validation datasets if they don't exist\n",
+        "for subdirectory in [train_dir, test_dir, validate_dir]:\n",
+        "    subdirectories_electronic.append(os.path.join(subdirectory,'electronic'))\n",
+        "    if not os.path.exists(os.path.join(subdirectory,'electronic')):\n",
+        "        os.mkdir(os.path.join(subdirectory,'electronic'))\n",
+        "        print(os.path.join(subdirectory,'electronic'),\"created\")\n",
+        "\n",
+        "    subdirectories_rock.append(os.path.join(subdirectory, 'rock'))\n",
+        "    if not os.path.exists(os.path.join(subdirectory, 'rock')):\n",
+        "        os.mkdir(os.path.join(subdirectory, 'rock'))\n",
+        "        print(os.path.join(subdirectory, 'rock'), \"created\")\n",
+        "\n",
+        "    subdirectories_hiphop.append(os.path.join(subdirectory,'hiphop'))\n",
+        "    if not os.path.exists(os.path.join(subdirectory,'hiphop')):\n",
+        "        os.mkdir(os.path.join(subdirectory,'hiphop'))\n",
+        "        print(os.path.join(subdirectory,'hiphop'),\"created\")\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "fVJUacXowtzf"
+      },
+      "outputs": [],
+      "source": [
+        "# Split the data into train, test and validation sets\n",
+        "x_train, x_test1, y_train, y_test1 = train_test_split(df[\"image\"], df[\"category\"] ,test_size=0.2, random_state=42)\n",
+        "x_test, x_val, y_test, y_val = train_test_split(x_test1, y_test1 ,test_size=0.5, random_state=42)\n",
+        "\n",
+        "# Print the size of each set\n",
+        "print('Training data set size :', y_train.shape[0])\n",
+        "print('Test data set size :', y_test.shape[0])\n",
+        "print('Validation data set size :', y_val.shape[0])\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "X0EHWy1qxID_"
+      },
+      "outputs": [],
+      "source": [
+        "# Copy the files into the corresponding genre directories in train, test and validation datasets\n",
+        "x = [x_train,x_test,x_val]\n",
+        "y = [y_train,y_test,y_val]\n",
+        "k = 0\n",
+        "\n",
+        "for images,genres in zip(x,y) :\n",
+        "    for (file,category) in zip(images,genres):\n",
+        "        if category == 'electronic':\n",
+        "            src = os.path.join(original_train_directory, file)\n",
+        "            dst = os.path.join(subdirectories_electronic[k], file)\n",
+        "            shutil.copyfile(src, dst)\n",
+        "        elif category == 'rock':\n",
+        "            src = os.path.join(original_train_directory, file)\n",
+        "            dst = os.path.join(subdirectories_rock[k], file)\n",
+        "            shutil.copyfile(src, dst)\n",
+        "        else:\n",
+        "            src = os.path.join(original_train_directory, file)\n",
+        "            dst = os.path.join(subdirectories_hiphop[k], file)\n",
+        "            shutil.copyfile(src, dst)\n",
+        "\n",
+        "    print(len(os.listdir(subdirectories_electronic[k])), \" electronic covers copied to:\", subdirectories_electronic[k])\n",
+        "    print(len(os.listdir(subdirectories_rock[k])), \" rock covers copies to:\", subdirectories_rock[k])\n",
+        "    print(len(os.listdir(subdirectories_hiphop[k])), \" hiphop covers copied to:\", subdirectories_hiphop[k])\n",
+        "\n",
+        "    k = k + 1\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "ZFldb7zIxgfj"
+      },
+      "outputs": [],
+      "source": [
+        "# Visualize training electronic images examples\n",
+        "plt.figure(figsize=(25,20))\n",
+        "plt.suptitle(\"Train Electronic Images\", fontsize=20)\n",
+        "\n",
+        "images = os.listdir(subdirectories_electronic[0])\n",
+        "for i in range(len(images)-20, len(images)):\n",
+        "    plt.subplot(5,5,i-len(images)+20+1)\n",
+        "\n",
+        "    full_image = plt.imread(os.path.join(original_train_directory, images[i]))\n",
+        "    plt.xticks([])\n",
+        "    plt.yticks([])\n",
+        "    plt.grid(False)\n",
+        "    plt.imshow(full_image, cmap=plt.cm.binary)\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "XVl8cEzY7lP6"
+      },
+      "outputs": [],
+      "source": [
+        "# Visualize training rock images examples\n",
+        "plt.figure(figsize=(25,20))\n",
+        "plt.suptitle(\" Train Rock Images\", fontsize=20)\n",
+        "\n",
+        "images = os.listdir(subdirectories_rock[0])\n",
+        "for i in range(len(images)-20, len(images)):\n",
+        "    plt.subplot(5,5,i-len(images)+20+1)\n",
+        "\n",
+        "    full_image = plt.imread(os.path.join(original_train_directory, images[i]))\n",
+        "    plt.xticks([])\n",
+        "    plt.yticks([])\n",
+        "    plt.grid(False)\n",
+        "    plt.imshow(full_image, cmap=plt.cm.binary)"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "nNizgXt9cw3U"
+      },
+      "outputs": [],
+      "source": [
+        "# Visualize training hiphop images examples\n",
+        "plt.figure(figsize=(25,20))\n",
+        "plt.suptitle(\"Train HipHop images\", fontsize=20)\n",
+        "\n",
+        "images = os.listdir(subdirectories_hiphop[0])\n",
+        "for i in range(len(images)-20, len(images)):\n",
+        "    plt.subplot(5,5,i-len(images)+20+1)\n",
+        "\n",
+        "    full_image = plt.imread(os.path.join(original_train_directory, images[i]))\n",
+        "    plt.xticks([])\n",
+        "    plt.yticks([])\n",
+        "    plt.grid(False)\n",
+        "    plt.imshow(full_image, cmap=plt.cm.binary)"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "nqYNkhrb9EKS"
+      },
+      "outputs": [],
+      "source": [
+        "# Create ImageDataGenerators for training and validation datasets\n",
+        "# ImageDataGenerators are used to generate batches of tensor image data with real-time data augmentation\n",
+        "train_datagen = ImageDataGenerator(rescale=1./255)\n",
+        "test_datagen = ImageDataGenerator(rescale=1./255)\n",
+        "\n",
+        "# Define target size for resizing images and batch size\n",
+        "# Class mode is set to 'categorical' for multi-class classification\n",
+        "train_generator = train_datagen.flow_from_directory(\n",
+        "        train_dir,\n",
+        "        target_size=(300,300),\n",
+        "        batch_size=20,\n",
+        "        class_mode='categorical')\n",
+        "\n",
+        "validation_generator = test_datagen.flow_from_directory(\n",
+        "        validate_dir,\n",
+        "        target_size=(300,300),\n",
+        "        batch_size=20,\n",
+        "        class_mode='categorical')\n",
+        "\n",
+        "# Print the shapes of data and labels batches to confirm they are as expected\n",
+        "for data_batch, labels_batch in train_generator:\n",
+        "    print('data batch shape:', data_batch.shape)\n",
+        "    print('labels batch shape:', labels_batch.shape)\n",
+        "    break\n"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "jR58jYwFmJVX"
+      },
+      "source": [
+        "## model_without_whitening.py"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "ZnCSZKVYmJVX"
+      },
+      "outputs": [],
+      "source": [
+        "# Import ResNet152v2 pre-trained model with ImageNet weights\n",
+        "from tensorflow.keras.applications import ResNet152V2\n",
+        "\n",
+        "# 1. ResNet152V2: Utilizes a pre-trained architecture to initialize weights, aiding in feature extraction.\n",
+        "pre_trained_model = ResNet152V2(weights='imagenet', include_top=False, input_shape=(300, 300, 3))\n",
+        "pre_trained_model.summary()"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "D0apnmP_Cys9"
+      },
+      "outputs": [],
+      "source": [
+        "# Set all pre-trained model layers to non-trainable\n",
+        "for layer in pre_trained_model.layers:\n",
+        "    layer.trainable = False\n",
+        "\n",
+        "# Define the custom neural network architecture\n",
+        "last_layer = pre_trained_model.get_layer('post_relu')\n",
+        "last_output = last_layer.output\n",
+        "\n",
+        "# 2. GlobalMaxPooling2D: Reduces the spatial dimensions of the output volume.\n",
+        "x = tf.keras.layers.GlobalMaxPooling2D()(last_output)\n",
+        "\n",
+        "# 3. Dense 512 and ReLU Activation: Fully connected layer with 512 neurons and ReLU activation for non-linearity.\n",
+        "x = tf.keras.layers.Dense(512, activation='relu', kernel_regularizer=regularizers.l1(0.01))(x)\n",
+        "\n",
+        "# 4. Dropout 0.3: Prevents overfitting by randomly setting a fraction of input units to 0 during training.\n",
+        "x = tf.keras.layers.Dropout(0.3)(x)\n",
+        "\n",
+        "# 5. Dense 256 and ReLU Activation: Another fully connected layer with 256 neurons and ReLU activation.\n",
+        "x = tf.keras.layers.Dense(256, activation='relu', kernel_regularizer=regularizers.l1(0.01))(x)\n",
+        "\n",
+        "# 6. Dropout 0.3: Another dropout layer for regularization.\n",
+        "x = tf.keras.layers.Dropout(0.3)(x)\n",
+        "\n",
+        "# 7. Dense 128 and ReLU Activation: Further fully connected layer with 128 neurons for feature transformation.\n",
+        "x = tf.keras.layers.Dense(128, activation='relu', kernel_regularizer=regularizers.l1(0.01))(x)\n",
+        "\n",
+        "# 8. Dropout 0.3: Final dropout layer to counteract overfitting.\n",
+        "x = tf.keras.layers.Dropout(0.3)(x)\n",
+        "\n",
+        "# 9. Dense 3 and Softmax Activation: Output layer with 3 neurons corresponding to the classes, with softmax activation for probability distribution.\n",
+        "x = tf.keras.layers.Dense(3, activation='softmax')(x)\n",
+        "\n",
+        "# Combine the pre-trained model with the additional layers to complete the architecture\n",
+        "bn_model = tf.keras.Model(pre_trained_model.input, x)\n",
+        "bn_model.summary()\n",
+        "\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "6EErvH34HK0K"
+      },
+      "outputs": [],
+      "source": [
+        "# Open trainable layers\n",
+        "pre_trained_model.trainable = True\n",
+        "set_trainable = False\n",
+        "for layer in pre_trained_model.layers:\n",
+        "    if layer.name == 'conv5_block3_preact_bn':\n",
+        "        set_trainable = True\n",
+        "    if set_trainable:\n",
+        "        layer.trainable = True\n",
+        "    else:\n",
+        "        layer.trainable = False\n",
+        "\n",
+        "# Define the learning rate scheduler function\n",
+        "def lr_scheduler(epoch):\n",
+        "    lr = 1e-4\n",
+        "    if epoch > 10:\n",
+        "        lr *= 0.1\n",
+        "    if epoch > 20:\n",
+        "        lr *= 0.1\n",
+        "    return lr\n",
+        "\n",
+        "# Set up callback functions\n",
+        "lr_scheduler_callback = LearningRateScheduler(lr_scheduler)\n",
+        "lr_reduce = ReduceLROnPlateau(monitor='val_accuracy', factor=0.6, patience=8, verbose=1, mode='max', min_lr=1e-4)\n",
+        "checkpoint = ModelCheckpoint('bn_finetune.h16', monitor='val_accuracy', mode='max', save_best_only=True, verbose=1)\n",
+        "early_stopping = EarlyStopping(monitor='val_accuracy', min_delta=0, patience=5, verbose=1, mode='max')\n",
+        "\n",
+        "# Compile the model\n",
+        "# This step prepares the model for training.\n",
+        "# It specifies the optimizer to update model weights, the loss function to evaluate performance,\n",
+        "# and the metric to monitor during training.\n",
+        "bn_model.compile(\n",
+        "    optimizer=tf.keras.optimizers.RMSprop(learning_rate=1e-4),  # RMSprop optimizer with a learning rate of 1e-4\n",
+        "    loss='categorical_crossentropy',  # categorical cross-entropy for multi-class classification\n",
+        "    metrics=['accuracy']  # Monitoring accuracy during training\n",
+        ")\n",
+        "\n",
+        "# Train the model\n",
+        "# This step starts the training process by feeding the data into the model.\n",
+        "bn_history = bn_model.fit(\n",
+        "    train_generator,  # Using the training data generator\n",
+        "    epochs=10,  # Training for 10 complete passes through the training dataset\n",
+        "    validation_data=validation_generator,  # Using the validation data generator\n",
+        "    validation_steps=len(validation_generator),  # Number of batches to draw from the validation generator for evaluation\n",
+        "    callbacks=[lr_reduce, checkpoint, early_stopping, lr_scheduler_callback]  # Applying various callback functions\n",
+        ")\n"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "source": [
+        "## model_with_whitening.py\n"
+      ],
+      "metadata": {
+        "id": "AFaWtJySUqej"
+      }
+    },
+    {
+      "cell_type": "code",
+      "source": [
+        "#  Create custome Keras layer for concept whitening\n",
+        "class ConceptWhitening(tf.keras.layers.Layer):\n",
+        "    def __init__(self, epsilon=1e-5, **kwargs):\n",
+        "        super(ConceptWhitening, self).__init__(**kwargs)\n",
+        "        self.epsilon = epsilon\n",
+        "\n",
+        "    def build(self, input_shape):\n",
+        "        super(ConceptWhitening, self).build(input_shape)\n",
+        "\n",
+        "    def call(self, inputs):\n",
+        "        # Compute the covariance matrix of the input activations\n",
+        "        mean = tf.reduce_mean(inputs, axis=[0, 1, 2], keepdims=True)\n",
+        "        centered_inputs = inputs - mean\n",
+        "        cov_matrix = tf.reduce_mean(tf.einsum('bijc,bijd->bcd', centered_inputs, centered_inputs), axis=0)\n",
+        "\n",
+        "        # Compute the whitening matrix\n",
+        "        s, u, v = tf.linalg.svd(cov_matrix)\n",
+        "        whitening_matrix = tf.einsum('bi,bj->bij', tf.math.rsqrt(tf.reshape(s, [-1, 1]) + self.epsilon), u)\n",
+        "\n",
+        "        # Squeeze out the singleton dimension\n",
+        "        whitening_matrix = tf.squeeze(whitening_matrix, axis=1)\n",
+        "\n",
+        "        # Apply the whitening transformation\n",
+        "        whitened_inputs = tf.einsum('bijc,bd,de->bije', centered_inputs, u, whitening_matrix)\n",
+        "\n",
+        "        # Debugging: Print the shape of the centered inputs for verification\n",
+        "        print(\"Shape of centered_inputs:\", tf.shape(centered_inputs))\n",
+        "\n",
+        "        # Debugging: Print the shape of 'u' to ensure it's as expected\n",
+        "        print(\"Shape of u:\", tf.shape(u))\n",
+        "\n",
+        "        # Debugging: Print the shape of the whitening matrix for validation\n",
+        "        print(\"Shape of whitening_matrix:\", tf.shape(whitening_matrix))\n",
+        "\n",
+        "        return whitened_inputs\n",
+        "\n",
+        "    def get_config(self):\n",
+        "        config = super(ConceptWhitening, self).get_config()\n",
+        "        config.update({\"epsilon\": self.epsilon})\n",
+        "        return config"
+      ],
+      "metadata": {
+        "id": "yatkMSAqUtRt"
+      },
+      "execution_count": null,
+      "outputs": []
+    },
+    {
+      "cell_type": "code",
+      "source": [
+        "# Import ResNet152v2 pre trained model with imagenet weights\n",
+        "from tensorflow.keras.applications import ResNet152V2\n",
+        "from tensorflow.keras import regularizers\n",
+        "\n",
+        "# Import your pre-trained model\n",
+        "pre_trained_model = ResNet152V2(weights='imagenet', include_top=False, input_shape=(300, 300, 3))\n",
+        "\n",
+        "# Make all layers non-trainable\n",
+        "for layer in pre_trained_model.layers:\n",
+        "    layer.trainable = False\n",
+        "\n",
+        "# Get the last layer's output\n",
+        "last_layer = pre_trained_model.get_layer('post_relu')\n",
+        "last_output = last_layer.output\n",
+        "\n",
+        "# Insert the Concept Whitening layer\n",
+        "x = ConceptWhitening()(last_output)\n",
+        "\n",
+        "# Add your additional layers\n",
+        "x = tf.keras.layers.GlobalMaxPooling2D()(x)\n",
+        "x = tf.keras.layers.Dense(512, activation='relu', kernel_regularizer=regularizers.l1(0.01))(x)\n",
+        "x = tf.keras.layers.Dropout(0.3)(x)\n",
+        "x = tf.keras.layers.Dense(256, activation='relu', kernel_regularizer=regularizers.l1(0.01))(x)\n",
+        "x = tf.keras.layers.Dropout(0.3)(x)\n",
+        "x = tf.keras.layers.Dense(128, activation='relu', kernel_regularizer=regularizers.l1(0.01))(x)\n",
+        "x = tf.keras.layers.Dropout(0.3)(x)\n",
+        "x = tf.keras.layers.Dense(3, activation='softmax')(x)\n",
+        "\n",
+        "# Create the final model\n",
+        "cw_model = tf.keras.Model(pre_trained_model.input, x)\n",
+        "cw_model.summary()"
+      ],
+      "metadata": {
+        "id": "O_F93p9QVFnJ"
+      },
+      "execution_count": null,
+      "outputs": []
+    },
+    {
+      "cell_type": "code",
+      "source": [
+        "# Open trainable layers\n",
+        "pre_trained_model.trainable = True\n",
+        "set_trainable = False\n",
+        "for layer in pre_trained_model.layers:\n",
+        "    if layer.name == 'conv5_block3_preact_cw':\n",
+        "        set_trainable = True\n",
+        "    if set_trainable:\n",
+        "        layer.trainable = True\n",
+        "    else:\n",
+        "        layer.trainable = False\n",
+        "\n",
+        "# Define the learning rate scheduler function\n",
+        "def lr_scheduler(epoch):\n",
+        "    lr = 1e-4\n",
+        "    if epoch > 10:\n",
+        "        lr *= 0.1\n",
+        "    if epoch > 20:\n",
+        "        lr *= 0.1\n",
+        "    return lr\n",
+        "\n",
+        "# Set up callback functions\n",
+        "lr_scheduler_callback = LearningRateScheduler(lr_scheduler)\n",
+        "lr_reduce = ReduceLROnPlateau(monitor='val_accuracy', factor=0.6, patience=8, verbose=1, mode='max', min_lr=1e-4)\n",
+        "checkpoint = ModelCheckpoint('cw_finetune.h16', monitor='val_accuracy', mode='max', save_best_only=True, verbose=1)\n",
+        "early_stopping = EarlyStopping(monitor='val_accuracy', min_delta=0, patience=5, verbose=1, mode='max')\n",
+        "\n",
+        "# Compile the model\n",
+        "cw_model.compile(optimizer=tf.keras.optimizers.RMSprop(learning_rate=1e-4),\n",
+        "              loss='categorical_crossentropy',\n",
+        "              metrics=['accuracy'])\n",
+        "\n",
+        "# Train the model\n",
+        "cw_history = cw_model.fit(\n",
+        "      train_generator,\n",
+        "      epochs=10,\n",
+        "      validation_data=validation_generator,\n",
+        "      validation_steps=len(validation_generator),\n",
+        "      callbacks=[lr_reduce, checkpoint, early_stopping, lr_scheduler_callback])"
+      ],
+      "metadata": {
+        "id": "YB4TP7vWV4sB"
+      },
+      "execution_count": null,
+      "outputs": []
+    },
+    {
+      "cell_type": "markdown",
+      "source": [
+        "## comparison_analysis.py"
+      ],
+      "metadata": {
+        "id": "JoWM_4yMVNl5"
+      }
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "5Cv68ofGIXfa"
+      },
+      "outputs": [],
+      "source": [
+        "# Retrieve accuracy and loss from the bn_model's history\n",
+        "acc = bn_history.history['accuracy']\n",
+        "val_acc = bn_history.history['val_accuracy']\n",
+        "loss = bn_history.history['loss']\n",
+        "val_loss = bn_history.history['val_loss']\n",
+        "\n",
+        "# Create a range of epochs to use in the plot\n",
+        "epochs = range(1, len(acc) + 1)\n",
+        "\n",
+        "# Plot training and validation accuracy\n",
+        "plt.plot(epochs, acc, 'bo', label='Training acc')\n",
+        "plt.plot(epochs, val_acc, 'b', label='Validation acc')\n",
+        "plt.title('Training and validation accuracy')\n",
+        "plt.legend()\n",
+        "\n",
+        "plt.figure()\n",
+        "\n",
+        "# Plot training and validation loss\n",
+        "plt.plot(epochs, loss, 'bo', label='Training loss')\n",
+        "plt.plot(epochs, val_loss, 'b', label='Validation loss')\n",
+        "plt.title('Training and validation loss')\n",
+        "plt.legend()\n",
+        "\n",
+        "plt.show()\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "source": [
+        "# Retrieve accuracy and loss from the cw_model's history\n",
+        "acc = cw_history.history['accuracy']\n",
+        "val_acc = cw_history.history['val_accuracy']\n",
+        "loss = cw_history.history['loss']\n",
+        "val_loss = cw_history.history['val_loss']\n",
+        "\n",
+        "# Create a range of epochs to use in the plot\n",
+        "epochs = range(1, len(acc) + 1)\n",
+        "\n",
+        "# Plot training and validation accuracy\n",
+        "plt.plot(epochs, acc, 'bo', label='Training acc')\n",
+        "plt.plot(epochs, val_acc, 'b', label='Validation acc')\n",
+        "plt.title('Training and validation accuracy')\n",
+        "plt.legend()\n",
+        "\n",
+        "plt.figure()\n",
+        "\n",
+        "# Plot training and validation loss\n",
+        "plt.plot(epochs, loss, 'bo', label='Training loss')\n",
+        "plt.plot(epochs, val_loss, 'b', label='Validation loss')\n",
+        "plt.title('Training and validation loss')\n",
+        "plt.legend()\n",
+        "\n",
+        "plt.show()"
+      ],
+      "metadata": {
+        "id": "MCibCLsrxFuO"
+      },
+      "execution_count": null,
+      "outputs": []
+    },
+    {
+      "cell_type": "code",
+      "source": [
+        "# Extract the metrics from the history dictionaries\n",
+        "bn_acc = bn_history['accuracy']\n",
+        "bn_val_acc = bn_history['val_accuracy']\n",
+        "bn_loss = bn_history['loss']\n",
+        "bn_val_loss = bn_history['val_loss']\n",
+        "\n",
+        "cw_acc = cw_history['accuracy']\n",
+        "cw_val_acc = cw_history['val_accuracy']\n",
+        "cw_loss = cw_history['loss']\n",
+        "cw_val_loss = cw_history['val_loss']\n",
+        "\n",
+        "epochs = range(1, len(bn_acc) + 1)\n",
+        "\n",
+        "# Create the plots\n",
+        "plt.figure(figsize=(10, 5))\n",
+        "\n",
+        "# Plot for accuracy\n",
+        "plt.subplot(1, 2, 1)\n",
+        "plt.plot(epochs, bn_acc, 'bo-', label='bn_model Training acc')\n",
+        "plt.plot(epochs, bn_val_acc, 'ro-', label='bn_model Validation acc')\n",
+        "plt.plot(epochs, cw_acc, 'go-', label='cw_model Training acc')\n",
+        "plt.plot(epochs, cw_val_acc, 'mo-', label='cw_model Validation acc')\n",
+        "plt.title('Training and Validation Accuracy')\n",
+        "plt.xlabel('Epochs')\n",
+        "plt.ylabel('Accuracy')\n",
+        "plt.legend()\n",
+        "\n",
+        "# Plot for loss\n",
+        "plt.subplot(1, 2, 2)\n",
+        "plt.plot(epochs, bn_loss, 'bo-', label='bn_model Training loss')\n",
+        "plt.plot(epochs, bn_val_loss, 'ro-', label='bn_model Validation loss')\n",
+        "plt.plot(epochs, cw_loss, 'go-', label='cw_model Training loss')\n",
+        "plt.plot(epochs, cw_val_loss, 'mo-', label='cw_model Validation loss')\n",
+        "plt.title('Training and Validation Loss')\n",
+        "plt.xlabel('Epochs')\n",
+        "plt.ylabel('Loss')\n",
+        "plt.legend()\n",
+        "\n",
+        "plt.tight_layout()\n",
+        "plt.show()"
+      ],
+      "metadata": {
+        "id": "aUjYDrhsYyrs"
+      },
+      "execution_count": null,
+      "outputs": []
+    },
+    {
+      "cell_type": "markdown",
+      "source": [
+        "## deploy"
+      ],
+      "metadata": {
+        "id": "UuE29qx6Z4DC"
+      }
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "S5fqPCg-Izf0"
+      },
+      "outputs": [],
+      "source": [
+        "# Save the model (bn_model) without concept whitening\n",
+        "bn_model.save('bn_model.h6')\n",
+        "\n",
+        "# Load test images\n",
+        "images_test = os.listdir(\"/content/drive/MyDrive/documentos/vinyl genre class/data/For thesis/data/For thesis/Testing\")\n",
+        "\n",
+        "# Initialize a dataframe to store test results\n",
+        "df_test = pd.DataFrame(columns=['category', 'name'])\n",
+        "categories = ['electronic','rock','hiphop']\n",
+        "\n",
+        "# Loop through all test images, predict their category and append results to the dataframe\n",
+        "for image in images_test:\n",
+        "     if image.endswith(\".jpg\") is True or image.endswith(\".jpeg\") is True:\n",
+        "            img = tf.keras.utils.load_img (\n",
+        "                f\"/content/drive/MyDrive/documentos/vinyl genre class/data/For thesis/data/For thesis/Testing/{image}\", target_size=(300, 300)\n",
+        "            )\n",
+        "            img_array = tf.keras.utils.img_to_array(img)\n",
+        "            img_array = tf.expand_dims(img_array, 0)\n",
+        "            img_array /= 255.\n",
+        "\n",
+        "            preds = bn_model.predict(img_array)\n",
+        "            score = preds[0]\n",
+        "            max_indexes = np.argmax(score)\n",
+        "\n",
+        "            df_test = pd.concat([df_test, pd.DataFrame({'category': [categories[max_indexes]], 'name': [image]})], ignore_index=True)\n",
+        "            print(f\"{image} is {categories[max_indexes]}\")\n",
+        "\n",
+        "# Save test results to a CSV file\n",
+        "df_test.to_csv('results_bn_new.csv', header=True)"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "source": [
+        "# Save the model (cw_model) with concept whitening\n",
+        "cw_model.save('cw_model.h6')\n",
+        "\n",
+        "# Load test images\n",
+        "images_test = os.listdir(\"/content/drive/MyDrive/documentos/vinyl genre class/data/For thesis/Testing\")\n",
+        "\n",
+        "# Initialize a dataframe to store test results\n",
+        "df_test = pd.DataFrame(columns=['category', 'name'])\n",
+        "categories = ['electronic','rock','hiphop']\n",
+        "\n",
+        "# Loop through all test images, predict their category and append results to the dataframe\n",
+        "for image in images_test:\n",
+        "     if image.endswith(\".jpg\") is True or image.endswith(\".jpeg\") is True:\n",
+        "            img = tf.keras.utils.load_img (\n",
+        "                f\"/content/drive/MyDrive/documentos/vinyl genre class/data/For thesis/data/For thesis/Testing/{image}\", target_size=(300, 300)\n",
+        "            )\n",
+        "            img_array = tf.keras.utils.img_to_array(img)\n",
+        "            img_array = tf.expand_dims(img_array, 0)\n",
+        "            img_array /= 255.\n",
+        "\n",
+        "            preds = cw_model.predict(img_array)\n",
+        "            score = preds[0]\n",
+        "            max_indexes = np.argmax(score)\n",
+        "\n",
+        "            df_test = pd.concat([df_test, pd.DataFrame({'category': [categories[max_indexes]], 'name': [image]})], ignore_index=True)\n",
+        "            print(f\"{image} is {categories[max_indexes]}\")\n",
+        "\n",
+        "# Save test results to a CSV file\n",
+        "df_test.to_csv('results_cw_new.csv', header=True)"
+      ],
+      "metadata": {
+        "id": "dfahimjUaM4G"
+      },
+      "execution_count": null,
+      "outputs": []
+    }
+  ],
+  "metadata": {
+    "colab": {
+      "provenance": []
+    },
+    "gpuClass": "standard",
+    "kernelspec": {
+      "display_name": "Python 3 (ipykernel)",
+      "language": "python",
+      "name": "python3"
+    },
+    "language_info": {
+      "codemirror_mode": {
+        "name": "ipython",
+        "version": 3
+      },
+      "file_extension": ".py",
+      "mimetype": "text/x-python",
+      "name": "python",
+      "nbconvert_exporter": "python",
+      "pygments_lexer": "ipython3",
+      "version": "3.10.10"
+    }
+  },
+  "nbformat": 4,
+  "nbformat_minor": 0
+}