pages omitted

app.py

@@ -15,7 +15,7 @@ st.markdown(
     Streamlit is an open-source app framework built specifically for
     Machine Learning and Data Science projects.
 
-    **👈 Select a demo from the sidebar** to see some examples
-    of what Streamlit can do!
+    - Go to `docs/notebooks` for access of python notebooks
+    - For more information, go [here](https://wyn-education.streamlit.app/)
 """
 )

pages/1_Main_Deck.py

@@ -1,38 +0,0 @@
-import streamlit as st
-import base64
-
-def get_pdf_file_content_as_base64(file_path: str):
-    """
-    Converts a PDF file to base64 for embedding.
-    Args:
-    file_path (str): The file path to the PDF file.
-
-    Returns:
-    str: The base64 encoded string of the PDF file.
-    """
-    with open(file_path, "rb") as pdf_file:
-        base64_encoded_data = base64.b64encode(pdf_file.read()).decode('utf-8')
-    return base64_encoded_data
-
-def main():
-    st.title('PDF Display in Streamlit')
-    
-    # Path to the PDF file
-    pdf_path = "AI_Decoded__Yin_2024.pdf"
-    
-    # Display the PDF by downloading it
-    with open(pdf_path, "rb") as file:
-        btn = st.download_button(
-            label="Download PDF",
-            data=file,
-            file_name="xyz.pdf",
-            mime="application/octet-stream"
-        )
-    
-    # Display the PDF embedded in the page
-    base64_pdf = get_pdf_file_content_as_base64(pdf_path)
-    pdf_display = f'<iframe src="data:application/pdf;base64,{base64_pdf}" width="700" height="1000" type="application/pdf"></iframe>'
-    st.markdown(pdf_display, unsafe_allow_html=True)
-
-if __name__ == "__main__":
-    main()

pages/2_Artificial_Neural_Networks.py

@@ -1,441 +0,0 @@
-def page_for_ann():
-    import streamlit as st
-
-    st.write("# Welcome to Artificial Neural Network! 👋")
-    st.sidebar.success("Select a session from the menu of content.")
-
-    st.markdown(
-        r"""
-        # Artificial Neural Networks Overview
-
-        Artificial Neural Networks (ANNs) are a cornerstone of modern machine learning, offering powerful tools for learning representations and making predictions. This document covers key aspects of ANNs, including representation learning, forward propagation, loss functions, optimization techniques, and a practical application example using TensorFlow.
-
-        ## What is Representation Learning and the Importance of Neural Networks?
-
-        Representation learning is a set of methods that allows a machine to automatically discover the representations needed for feature detection or classification from raw data. Neural networks are crucial in representation learning because they can learn complex patterns and relationships in the data. By adjusting weights and biases, neural networks transform input data into more useful representations for a specific task.
-        """
-    )
-
-    video_file_ann = open("./videos/4_ann.mp4", "rb")
-    video_bytes_ann = video_file_ann.read()
-    st.video(video_bytes_ann)
-
-    st.markdown(
-        r"""
-        ## Forward Propagation in Neural Networks
-
-        Forward propagation is the process of calculating the output of a neural network by applying a set of weights to the input data and using activation functions in the network layers.
-        """
-    )
-
-    st.image(
-        "https://miro.medium.com/v2/resize:fit:640/1*sPg-0hha7o3iNPjY4n-vow.jpeg",
-        caption="Visualization of One Neuron (a.k.a. 'unit' or 'node' in the community) in an Artificial Neural Network (ANN)",
-        use_column_width=True,
-    )
-
-    st.markdown(
-        r"""
-        ### Fully Connected Dense Layers
-
-        In a fully connected dense layer, each neuron is connected to every neuron in the previous layer. The output of each neuron is a weighted sum of its inputs, passed through an activation function.
-
-        ### Activation Functions
-
-        - **ReLU (Rectified Linear Unit)**: ReLU is defined as $ f(x) = \max(0, x) $. It introduces non-linearity in the model, allowing the network to learn complex patterns. In the context of linear regression, ReLU can be seen as an extension that allows for more complex, non-linear relationships.
-        
-        - **Sigmoid**: The sigmoid function outputs values between 0 and 1, making it suitable for binary classification. It is defined as $ f(x) = \frac{1}{1 + e^{-x}} $.
-
-        - **Softmax**: Used in multi-class classification, softmax converts a vector of values into a probability distribution. For a vector $ \mathbf{z} $, softmax is defined as $ \sigma(\mathbf{z})_i = \frac{e^{z_i}}{\sum_{j=1}^{K} e^{z_j}} $ for $ i = 1, ..., K $ and $ K $ being the number of classes.
-        """
-    )
-
-    video_file_ann_forwardpropagation = open(
-        "./videos/4_ann_forwardpropagation.mp4", "rb"
-    )
-    video_bytes_ann_forwardpropagation = video_file_ann_forwardpropagation.read()
-    st.video(video_bytes_ann_forwardpropagation)
-
-    st.markdown(
-        r"""
-        ## Loss Functions
-
-        Loss functions measure how well the model performs on the given data.
-
-        - **Mean Square Error (MSE)**: Used for regression tasks, it's defined as $ MSE = \frac{1}{n} \sum_{i=1}^{n} (y_i - \hat{y_i})^2 $, where $ y_i $ is the true value, $ \hat{y_i} $ is the predicted value, and $ n $ is the number of samples.
-        
-        - **Binary Cross-Entropy**: Used for binary classification, it's defined as $ -\frac{1}{n} \sum_{i=1}^{n} [y_i \log(\hat{y_i}) + (1 - y_i) \log(1 - \hat{y_i})] $, where $ y_i $ is the true label, and $ \hat{y_i} $ is the predicted probability.
-
-        ## Optimization and Gradient Descent
-
-        Optimization in neural networks is about finding the best set of weights and biases to minimize the loss function.
-
-        - **Gradient Descent**: This is an optimization algorithm used to minimize the loss function by iteratively moving in the direction of steepest descent. The update rule is $ \theta = \theta - \alpha \nabla_\theta J(\theta) $, where $ \theta $ represents parameters, $ \alpha $ is the learning rate, and $ \nabla_\theta J(\theta) $ is the gradient of the loss function.
-        """
-    )
-
-    video_file_ann_loss = open("./videos/4_ann_loss.mp4", "rb")
-    video_bytes_ann_loss = video_file_ann_loss.read()
-    st.video(video_bytes_ann_loss)
-
-    st.markdown(
-        r"""
-        ## Practical Application: 3-Layer Neural Network in TensorFlow
-
-        Here's an example of implementing a 3-layer neural network using TensorFlow:
-
-        ```python
-        import tensorflow as tf
-
-        # Define the model
-        model = tf.keras.Sequential([
-            tf.keras.layers.Dense(128, activation='relu', input_shape=(input_shape,)),
-            tf.keras.layers.Dense(64, activation='relu'),
-            tf.keras.layers.Dense(num_classes, activation='softmax')
-        ])
-
-        # Compile the model
-        model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])
-
-        # Train the model
-        model.fit(X_train, y_train, epochs=10, validation_data=(X_val, y_val))
-        ```
-
-        - The model is defined with three layers: two hidden layers with ReLU activation and an output layer with softmax activation.
-        - The model is compiled with the Adam optimizer and categorical cross-entropy loss function.
-        - `model.fit` trains the model with the training data, specifying epochs and validation data.
-
-        This TensorFlow example demonstrates the construction, compilation, and training of a basic ANN for classification tasks.
-
-        """
-    )
-
-    st.markdown(
-        r"""
-        # Tensorflow Implementation on Colab
-
-        # MNIST Data
-
-        According to Yann LeCun on this [site](http://yann.lecun.com/exdb/mnist/#:~:text=It%20is%20a%20good%20database,efforts%20on%20preprocessing%20and%20formatting.), the MNIST database of handwritten digits, available from this page, has a training set of 60,000 examples, and a test set of 10,000 examples. It is a subset of a larger set available from NIST. The digits have been size-normalized and centered in a fixed-size image.
-
-        It is a good database for people who want to try learning techniques and pattern recognition methods on real-world data while spending minimal efforts on preprocessing and formatting.
-
-        ## Research Question
-
-        Can AI recognize pictures?
-
-        ## Agenda
-
-        This is the standrad image data for developing neural network models and convolutional neural network models.
-
-        - Neural Networks (1st meeting)
-        - Convolutional Neural Networks (2nd meeting)
-
-        One you have the basic knowledge, there two tasks:
-        - First, you build a model to make good predictions on MNIST dataset
-        - Next, you will get a new data set that is similar and you will build advanced models to make good predictions on it. Plan: We use the MNIST Fashion dataset.
-
-        Goal: To develop state-of-the-art CNN models.
-
-        ## Notes
-
-        Most of the notes are taken using markdown area. To learn more diverse editting techniques for you to take quick notes using markdown, please click [here](https://colab.research.google.com/notebooks/markdown_guide.ipynb).
-
-        An interesting visualization is [here](https://media.licdn.com/dms/image/C4D12AQFKRQOp_aXz0g/article-cover_image-shrink_600_2000/0/1577211633664?e=2147483647&v=beta&t=pZ5TdXalXICIqPXTDBX6NP-CYlGS3wE2Kn6y1XiYjHI)
-        """
-    )
-
-    st.image(
-        "https://media.licdn.com/dms/image/C4D12AQFKRQOp_aXz0g/article-cover_image-shrink_600_2000/0/1577211633664?e=2147483647&v=beta&t=pZ5TdXalXICIqPXTDBX6NP-CYlGS3wE2Kn6y1XiYjHI",
-        caption="Neural Network Architecture",
-        use_column_width=True,
-    )
-
-    st.markdown(
-        r"""
-
-        Given a picture, we can look at the pixel values. These pixel values provide information of the content/object in the image. These values are propagated into a network of neurons that are connected with lines (weights or also known as parameters) to be concluded to a final labeled category. The network architecture including width and length is completely dependent on the scientists and the data.
-        ```python
-        # import library
-        import tensorflow as tf
-
-        # use tf API to get data
-        mnist = tf.keras.datasets.mnist
-        (training_images, training_labels), (test_images, test_labels) = mnist.load_data()
-        print(training_images.shape)
-        print(training_images[0].shape)
-
-        # import
-        import os
-        import pickle
-
-        # dump: this means save the desired file into a pickle file
-        # note: a pickle file is a file ends with .pkl format
-        # b = (0, 1)
-        a = (training_images, training_labels), (test_images, test_labels)
-        with open('mnist_data.pkl', 'wb') as handle:
-            pickle.dump(a, handle, protocol=pickle.HIGHEST_PROTOCOL)
-
-        # load: this means start loading the desired file from the pickle file
-        # note: this means that you do a with-open clause, and grab whatever
-        #       inside of the .pkl file and defined it as a new object
-        #       please make sure the name and the directory are correct
-        with open('mnist_data.pkl', 'rb') as handle:
-            b = pickle.load(handle)
-
-        # preview
-        type(b), len(b)
-
-        # library
-        import matplotlib.pyplot as plt
-
-        # plot one sample
-        plt.imshow(training_images[0])
-        plt.title('title: this is digit ' + str(training_labels[0]))
-        plt.colorbar()
-
-        # library
-        import numpy as np
-
-        # unique digits
-        np.unique(training_labels)
-        ```
-
-        ### One hot encode
-
-        We need to use *to_categorical* function to do one-hot encode. The operation of one-hot encode will check each unique level of the data and create a new binary vector for that level.
-
-        For example, the vector below has 3 levels: 1, 3, 4. Hence, for each level (for each of the value 1, 3, 4), there will be a new vector created that only takes values 1 or 0.
-
-        $$
-        \begin{bmatrix}
-        1 \\
-        3 \\
-        1 \\
-        4 \\
-        \end{bmatrix}
-        \rightarrow_\text{one hot}
-        \begin{bmatrix}
-        1 & 0 & 0\\
-        0 & 1 & 0\\
-        1 & 0 & 0\\
-        0 & 0 & 1\\
-        \end{bmatrix}
-        $$
-
-        Source is [here](https://www.tensorflow.org/api_docs/python/tf/keras/utils/to_categorical).
-
-        ```python
-        # one hot encode
-        training_labels_one_hot_encoded = tf.keras.utils.to_categorical(training_labels, num_classes=10)
-        test_labels_one_hot_encoded = tf.keras.utils.to_categorical(test_labels, num_classes=10)
-
-        # rescale
-        training_images = training_images / 255
-        test_images = test_images / 255
-
-        ```
-
-        ## Activation Functions
-
-        An Activation Function decides whether a neuron should be activated or not. This means that it will decide whether the neuron's input to the network is important or not in the process of prediction using simpler mathematical operations. See [here](https://miro.medium.com/max/1400/1*p_hyqAtyI8pbt2kEl6siOQ.png).
-        """
-    )
-
-    st.image(
-        "https://miro.medium.com/max/1400/1*p_hyqAtyI8pbt2kEl6siOQ.png",
-        caption="Activation Functions",
-        use_column_width=True,
-    )
-
-    st.markdown(
-        r"""
-        Why use activation function? It is because we need to have non-linear component in each neuron.
-
-        ## Softmax Function
-
-        The softmax function is used as the activation function in the output layer of neural network models that predict a multinomial probability distribution. That is, softmax is used as the activation function for multi-class classification problems where class membership is required on more than two class labels. See [here](https://miro.medium.com/max/805/1*ReYpdIZ3ZSAPb2W8cJpkBg.jpeg).
-        """
-    )
-
-    st.image(
-        "https://miro.medium.com/max/805/1*ReYpdIZ3ZSAPb2W8cJpkBg.jpeg",
-        caption="Softmax Functions",
-        use_column_width=True,
-    )
-
-    st.markdown(
-        r"""
-        ## Regularization
-
-        Regularization refers to techniques that are used to calibrate machine learning models in order to minimize the adjusted loss function and prevent overfitting or underfitting. Using Regularization, we can fit our machine learning model appropriately on a given test set and hence reduce the errors in it. See [here](https://miro.medium.com/proxy/1*mh-QVwSKbe2kYCBh9mwA0Q.png).
-
-        If you are using L1 (this is the square one on the left), you are more likely to have sparse results (this means some of your weights are zeros).
-        """
-    )
-
-    st.image(
-        "https://miro.medium.com/proxy/1*mh-QVwSKbe2kYCBh9mwA0Q.png",
-        caption="Regularizations",
-        use_column_width=True,
-    )
-
-    st.markdown(
-        r"""
-        ### Question: Is this good or bad?
-
-        It depends. If you want to avoid overfitting way more than you do, use L1. If you don't really care about it, you can use L2.
-
-        ## Build a Neural Network
-
-        ```python
-        # use sequential api to build model
-        model = tf.keras.models.Sequential([
-            tf.keras.layers.Flatten(), # neural network requires the input layer to be a vector instead of 2D array
-            ## Your Changes Start Here ##
-            # Things to change:
-            # number of hidden layers
-            # number of neurons per hidden layer
-            # the activation functions: Please refer this website: https://keras.io/api/layers/activations/
-            # the kernel_regularizer: Please refer this website: https://www.tensorflow.org/api_docs/python/tf/keras/regularizers/Regularizer
-            tf.keras.layers.Dense(1024, activation='relu', use_bias=True), # input units (usually starts with 128) and activation (it's a choice, usually relu)
-            tf.keras.layers.Dense(512, activation='relu', use_bias=True), # create more hidden layer if desired
-            tf.keras.layers.Dense(64, activation='relu', use_bias=True), # create more hidden layer if desired
-            # ... you can have however many you want
-            ## Your Changes Ends Here ##
-            tf.keras.layers.Dense(10, activation='softmax') # output layer or end layer | you have to match the number of classes
-        ])
-        ```
-
-        ### Loss Function: MSE
-
-        What is a loss function? A loss function helps you to identify how many mistakes the model is making. The most common one (originated from linear regression) is L2 loss or also known as the square loss. This loss function is written below
-
-        $$\mathcal{L}(Y, \hat{Y}) = \frac{1}{n} \sum_{i=1}^n (y_i - \hat{y}_i)^2$$
-
-        where the $Y$ is ground truth at a vector level and the $\hat{Y}$ is the educated guess or estimate from the model at a vector level. The left hand side of the formula is defined by the right hand side which means that the vectors can be expanded using sub-index $i$. This sub-index $i$ is a running index indicating which sample the function is referring to. For example, let us take a look at the baby version below.
-
-        For example, suppose we have a few data points below. Denote $y_i - \hat{y}_i$ as "diff" for short hand nickname. Then the table below walks you through how to execute the above formula mathematically.
-
-        | $i$ | $y_i$ | $\hat{y}_i$ | $\text{diff}$ | $\text{diff}^2$ |
-        | --- | --- | --- | --- | --- |
-        | 1  |  10  |  20  | -10  | 100  |
-        | 2  |  10  |  5  | 5  | 25  |
-
-        If you watch carefully, the formula is essentially $\text{something}^2$ (we read "something square" or "square of something"), this means that it is a [convex function](https://en.wikipedia.org/wiki/Convex_function). Hence, there always exists an optimal point!!!
-
-        ### Loss Function: BCE
-
-        If you are training a binary classifier, chances are you are using binary cross-entropy / log loss as your loss function. Loss Function: Binary Cross-Entropy / Log Loss takes the following form
-
-        $$\mathcal{L}(y, \hat{y}) = -\frac{1}{n} \sum_{i=1}^n y_i \log(\hat{y}_i) + (1 - y_i) \log (1-\hat{y}_i)$$
-
-        where $y$ is the label and $\hat{y}$ is the predicted probability of the data point for all $n$ points.
-
-        The plot below gives us a clear picture —as the predicted probability of the true class gets closer to zero, the loss increases exponentially. See [here](https://miro.medium.com/max/836/1*T8KWtAn8FkAcsg8RsjiZ6Q.png).
-        """
-    )
-
-    st.image(
-        "https://miro.medium.com/max/836/1*T8KWtAn8FkAcsg8RsjiZ6Q.png",
-        caption="Visualization of Cross-entropy",
-        use_column_width=True,
-    )
-
-    st.markdown(
-        r"""
-        Consider the following scenarios:
-        - If predicted probabilities $\hat{y}_i$ is small (close to 0) and truth $y_i = 0$, what happens?
-        - If predicted probabilities $\hat{y}_i$ is small (close to 0) and truth $y_i = 1$, what happens?
-        - If predicted probabilities $\hat{y}_i$ is large (close to 1) and truth $y_i = 0$, what happens?
-        - If predicted probabilities $\hat{y}_i$ is large (close to 1) and truth $y_i = 1$, what happens?
-
-        Answers:
-        - If prediction $\hat{y}_i = 0.001$ and $y_i = 0$, then the loss is $-(1-0)\log(1-0.001)=-\log(0.999) \approx 0$, i.e. error is small so the model is good
-        - If prediction $\hat{y}_i = 0.001$ and $y_i = 1$, then the loss is $-\log(0.001)=-\log(0.001) = 3$, i.e. the error is large so the model is bad
-        - If prediction $\hat{y}_i = 0.999$ and $y_i = 0$, then the loss is $-(1-0)\log(1-0.999)=-\log(0.001)=3$, i.e. the error is large so the model is bad
-        - If prediction $\hat{y}_i = 0.999$ and $y_i = 1$, then the loss is $-\log(0.999) \approx 0$, i.e. error is small so the model is good
-
-        A good paper to refer to is [here](https://www.semanticscholar.org/paper/On-Loss-Functions-for-Deep-Neural-Networks-in-Janocha-Czarnecki/d589392ab606a3d2861988ebcba95176517939ec). See [here](https://d3i71xaburhd42.cloudfront.net/d589392ab606a3d2861988ebcba95176517939ec/2-Table1-1.png).
-
-        ### Optimization
-
-        An optimizer is an algorithm to search for the most optimal (some minimum value) on a loss surface. Recall the loss surface is how many mistakes the model is making, so it is intuitive that the desired point (the optimal point) is where the loss is small. The location where the loss is small is the "best" parameters we need in our model. See [here](https://miro.medium.com/max/1400/1*47skUygd3tWf3yB9A10QHg.gif).
-        """
-    )
-
-    st.image(
-        "https://towardsdatascience.com/a-visual-explanation-of-gradient-descent-methods-momentum-adagrad-rmsprop-adam-f898b102325c",
-        caption="Visualization of Gradient Descent",
-        use_column_width=True,
-    )
-
-    st.markdown(
-        r"""
-        - Animation of gradient descent is [here](https://towardsdatascience.com/a-visual-explanation-of-gradient-descent-methods-momentum-adagrad-rmsprop-adam-f898b102325c)
-        - Other animation can be found [here](https://towardsai.net/p/machine-learning/analysis-of-learning-rate-in-gradient-descent-algorithm-using-python)
-        Today, there are many different upgrades of optimization algorithm. Please see [here](https://user-images.githubusercontent.com/11681225/49325458-fc785480-f585-11e8-8d2a-9012d6024c6e.gif).
-
-        Source of this animation is [here](https://github.com/ilguyi/optimizers.numpy).
-        """
-    )
-
-    with st.expander("Code💻"):
-        st.markdown(
-            r"""
-
-            ```python
-            # compile
-            model.compile(
-                ## Your Changes Start Here ##
-                # optimizer: https://keras.io/api/optimizers/
-                # loss: https://keras.io/api/losses/
-                optimizer="adam",
-                loss="categorical_crossentropy",
-                ## Your Changes Ends Here ##
-                metrics=['accuracy']
-            )
-
-            # check gpu
-            %tensorflow_version 2.x
-            import tensorflow as tf
-            device_name = tf.test.gpu_device_name()
-            if device_name != '/device:GPU:0':
-            raise SystemError('GPU device not found')
-            print('Found GPU at: {}'.format(device_name))
-
-            # callbacks
-            callbacks = [ tf.keras.callbacks.ModelCheckpoint("toy_mnist_model.h5", save_best_only=True) ]
-
-            %%time
-            # fit | we train the model here
-            # use GPU
-            with tf.device('/device:GPU:0'):
-                history = model.fit(
-                    training_images, training_labels_one_hot_encoded,
-                    validation_split=0.2,
-                    ## Your Changes Start Here ##
-                    # there is a sweet spot, you don't want to overfit, but you also want to make sure your model learned something
-                    epochs=10
-                    ## Your Changes Ends Here ##
-                )
-
-            # use CPU
-            history = model.fit(
-                    training_images, # features | explanatory variables
-                    training_labels_one_hot_encoded, # label <= create yourself (it's just a vector of 1's and 0's)
-                    validation_split=0.2,
-                    ## Your Changes Start Here ##
-                    # there is a sweet spot, you don't want to overfit, but you also want to make sure your model learned something
-                    epochs=10,
-                    ## Your Changes Ends Here ##
-                    callbacks=callbacks
-                )
-
-            # save model: it must be .h5 format
-            model.save('your_neuralnet_model.h5')
-            ```
-            """
-        )
-
-page_for_ann()

pages/3_Convolutional_Neural_Networks.py

@@ -1,197 +0,0 @@
-def page_for_cnn():
-    import streamlit as st
-
-    st.write("# Welcome to Convolutional Neural Network! 👋")
-    st.sidebar.success("Select a session from the menu of content.")
-
-    st.markdown(
-        r"""
-        # Convolutional Neural Network (CNN) Overview
-
-        This document provides a comprehensive explanation of Convolutional Neural Networks (CNNs), a class of deep neural networks, widely used for analyzing visual imagery.
-
-        ## Motivation for CNNs
-
-        ### Difference from Tabular Data
-        Convolutional Neural Networks are specifically designed for processing data that comes in the form of multiple arrays, such as images. Unlike tabular data where each feature column can be independent, in image data, neighboring pixels are closely related, suggesting the presence of spatial hierarchy in features - a trait CNNs capitalize on.
-
-        ### Advantage Over Traditional Neural Networks
-        CNNs preserve the spatial relationship between pixels by learning internal feature representations using small squares of input data. They require fewer parameters compared to fully connected networks with the same depth, making them more efficient and reducing the risk of overfitting.
-       """
-    )
-
-    video_file_cnn = open("./videos/6_cnn.mp4", "rb")
-    video_bytes_cnn = video_file_cnn.read()
-    st.video(video_bytes_cnn)
-
-    st.markdown(
-        r"""
-        ## Convolution Operation in CNNs
-
-        ### Convolution with an Image and Filter
-        A convolution operation involves a filter (also known as a kernel) which is a small matrix used to apply effects like blurring, sharpening, edge detection, etc. The convolution operation is an element-wise multiplication of the filter matrix and the part of the image it is covering, followed by summing up these multiplication results into a single output pixel.
-        """
-    )
-
-    st.image(
-        "https://miro.medium.com/v2/resize:fit:1052/0*jLoqqFsO-52KHTn9.gif",
-        caption="Visualization of Convolutional Operation",
-        use_column_width=True,
-    )
-
-    st.markdown(
-        r"""
-        To illustrate a convolution operation using element-wise matrix multiplication, let's define two matrices and perform the operation. The convolution operation is typically denoted by $ \otimes $.
-
-        Let's consider a 3x3 matrix $ A $ and a 2x2 filter $ B $, and we'll perform the convolution operation on them. For simplicity, we'll use a stride of 1 and no padding.
-
-        Matrix $ A $ (the image):
-        $$ A = \begin{bmatrix} 1 & 2 & 3 \\ 4 & 5 & 6 \\ 7 & 8 & 9 \end{bmatrix} $$
-
-        Filter $ B $ (the kernel):
-        $$ B = \begin{bmatrix} -1 & 0 \\ 0 & 1 \end{bmatrix} $$
-
-        The convolution operation $ A \otimes B $ is calculated as follows:
-
-        1. Place the filter on the top-left corner of the matrix $ A $, and perform element-wise multiplication followed by a sum:
-
-        $$ \begin{bmatrix} 1 & 2 \\ 4 & 5 \end{bmatrix} \otimes \begin{bmatrix} -1 & 0 \\ 0 & 1 \end{bmatrix} = (1 \times -1) + (2 \times 0) + (4 \times 0) + (5 \times 1) = -1 + 0 + 0 + 5 = 4 $$
-
-        2. Move the filter one stride to the right and repeat:
-
-        $$ \begin{bmatrix} 2 & 3 \\ 5 & 6 \end{bmatrix} \otimes \begin{bmatrix} -1 & 0 \\ 0 & 1 \end{bmatrix} = (2 \times -1) + (3 \times 0) + (5 \times 0) + (6 \times 1) = -2 + 0 + 0 + 6 = 4 $$
-
-        3. Move the filter down one stride and repeat for the left column:
-
-        $$ \begin{bmatrix} 4 & 5 \\ 7 & 8 \end{bmatrix} \otimes \begin{bmatrix} -1 & 0 \\ 0 & 1 \end{bmatrix} = (4 \times -1) + (5 \times 0) + (7 \times 0) + (8 \times 1) = -4 + 0 + 0 + 8 = 4 $$
-
-        4. Finally, move the filter one stride to the right:
-
-        $$ \begin{bmatrix} 5 & 6 \\ 8 & 9 \end{bmatrix} \otimes \begin{bmatrix} -1 & 0 \\ 0 & 1 \end{bmatrix} = (5 \times -1) + (6 \times 0) + (8 \times 0) + (9 \times 1) = -5 + 0 + 0 + 9 = 4 $$
-
-        So, the resulting matrix after the convolution operation is:
-        $$ \begin{bmatrix} 4 & 4 \\ 4 & 4 \end{bmatrix} $$
-
-        This numerical example demonstrates a basic convolution operation with a stride of 1, no padding, and element-wise multiplication followed by a sum.
-        """
-    )
-
-    st.image(
-        "https://indiantechwarrior.com/wp-content/uploads/2022/05/conv3.gif",
-        caption="Visualization of Convolutional Operation (multiple filters)",
-        use_column_width=True,
-    )
-
-    st.markdown(
-        r"""
-        ### Filter Size and Stride
-        - **Filter Size**: The size of the filter (e.g., 3x3, 5x5) determines the number of pixels in the image to consider at a time for applying the convolution operation.
-        - **Stride**: Stride defines the number of pixels by which we slide the filter across the image. A larger stride results in a smaller output size.
-
-        ## Pooling Layers in CNNs
-
-        ### Max Pooling and Average Pooling
-        - **Max Pooling**: This operation reduces the spatial size (width, height) of the input volume for the next convolutional layer. It takes the maximum value over a window defined by the filter size.
-        
-        - **Average Pooling**: Similar to max pooling, but it takes the average of the elements in the filter window.
-        """
-    )
-
-    st.image(
-        "https://i.stack.imgur.com/pLlwx.png",
-        caption="Visualization of Impact of Filter on Image",
-        use_column_width=True,
-    )
-
-    st.markdown(
-        r"""
-        ## What are filters in image processing?
-
-        In the context of image processing (and, in general, signal processing), the kernels (also known as filters) are used to perform some specific operation on the image. For example, you can use a [Gaussian filter](https://homepages.inf.ed.ac.uk/rbf/HIPR2/gsmooth.htm) to smooth the image (including its edges).
-
-        ## What are filters in CNNs?
-
-        In the context of convolutional neural networks (CNNs), the filters (or kernels) are the learnable parameters of the model.
-
-        Before training, the kernels are usually randomly initialised (so they are not usually hardcoded). During training, depending on the loss or error of the network (according to the loss function), the kernels (or filters) are updated, so that to minimise the loss (or error). After training, they are typically fixed. Incredibly, the filters learned by CNNs can be similar to the [Gabor filter](https://en.wikipedia.org/wiki/Gabor_filter) (which is thought to be related to our visual system [this paper](http://www.cns.nyu.edu/%7Etony/vns/readings/olshausen-field-1996.pdf)). See [figure 9.19 of chapter 9 (p. 365) of the Deep Learning book by Goodfellow et al.](https://www.deeplearningbook.org/contents/convnets.html).
-
-        ## What are the differences and similarities?
-
-        In both contexts, the words "kernel" and "filter" are roughly synonymous, so they are often used interchangeably. Furthermore, in both cases, the kernels are related to the [convolution (or cross-correlation) operation](https://ai.stackexchange.com/a/22000/2444). More specifically, the application of a filter, which is a function $h$ to an input, which is another function $f$, is equivalent to the convolution of $f$ and $h$. In mathematics, this is often denoted by $f \otimes h = g$, where $\otimes$ is the convolution operator and $g$ is the result of the convolution operation and is often called the convolution (of $f$ and $h$) as we discussed above. In the case of image processing, $g$ is the filtered image. In the case of CNNs $g$ is often called an [activation map](https://stats.stackexchange.com/a/292064/82135).
-
-        ## Convolution Operation on Color Images
-
-        Color images have three channels (RGB). The convolution operation is applied to each channel separately with the same filter, and the results are summed up to produce the final output.
-
-        ## Convolution in 3D Data (e.g., MRI Scans)
-
-        In the case of 3D data, like MRI scans, the convolution operation is extended to three dimensions. The filter used for convolution is also a 3D matrix, moving along the depth of the input volume in addition to the height and width.
-        """
-    )
-
-    st.image(
-        "https://d3i71xaburhd42.cloudfront.net/ce3e49a4abd5b3ce6b615b60b4ffe6d09cb76a90/2-Figure1-1.png",
-        caption="MRI Classification using 3D Convolutional Neural Networks (credits to https://arxiv.org/abs/2003.08818)",
-        use_column_width=True,
-    )
-
-    st.markdown(
-        r"""
-        ## Example: LeNet Style CNN in TensorFlow
-
-        Below is an example of a LeNet-style CNN model implemented using TensorFlow:
-
-        ```python
-        import tensorflow as tf
-
-        model = tf.keras.Sequential([
-            # C1 Convolutional Layer
-            tf.keras.layers.Conv2D(filters=6, kernel_size=(3, 3), activation='relu', input_shape=(32,32,1)),
-            
-            # S2 Pooling Layer
-            tf.keras.layers.MaxPooling2D(pool_size=(2, 2), strides=2),
-            
-            # C3 Convolutional Layer
-            tf.keras.layers.Conv2D(filters=16, kernel_size=(3, 3), activation='relu'),
-            
-            # S4 Pooling Layer
-            tf.keras.layers.MaxPooling2D(pool_size=(2, 2), strides=2),
-            
-            # Flatten the CNN output to feed it with fully connected layers
-            tf.keras.layers.Flatten(),
-            
-            # C5 Fully Connected Layer
-            tf.keras.layers.Dense(units=120, activation='relu'),
-            
-            # F6 Fully Connected Layer
-            tf.keras.layers.Dense(units=84, activation='relu'),
-            
-            # Output Layer with softmax activation
-            tf.keras.layers.Dense(units=10, activation='softmax')
-        ])
-
-        # Compile the model
-        model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])
-
-        # Model summary
-        model.summary()
-        ```
-
-        ### Explanation of the Code
-        - The model starts with a `Sequential` container as it is a linear stack of layers.
-        - The first layer, `Conv2D`, performs the convolution operation with 6 filters each of size 3x3 and ReLU activation. The input shape (32,32,1) corresponds to 32x32 pixels with 1 channel (grayscale).
-        - The `MaxPooling2D` layer performs max pooling with a 2x2 window and stride of 2, reducing the spatial dimensions.
-        - Another convolutional and max pooling layer follow, with more filters but the same approach.
-        - `Flatten` is used to flatten the output from the convolutional layers to feed into fully connected layers.
-        - Two `Dense` layers follow, representing fully connected layers with ReLU activation.
-        - The final `Dense` layer has a softmax activation function, used for multi-class classification.
-
-        This model demonstrates the basic architecture of a CNN, closely resembling
-
-        the LeNet architecture, one of the earliest and most influential CNN architectures.
-
-        """
-    )
-
-
-page_for_cnn()

pages/4_Convolutional_Operation.py

@@ -1,148 +0,0 @@
-def page_for_convop():
-    import streamlit as st
-
-    st.write("# Welcome to Convolutional Operation! 👋")
-    st.sidebar.success("Select a session from the menu of content.")
-
-    st.markdown(
-        r"""
-        # Convolution Operation in Statistics
-
-        This document provides an in-depth explanation of the convolution operation in statistics, its application in image data, and the mathematical principles behind it.
-       """
-    )
-
-    st.image(
-        "https://quincyaflint.weebly.com/uploads/2/6/5/0/26500868/convolution-1-fast_orig.gif",
-        caption="Convolutional Operation (the convolution is the product of the overlapping areas as the signal is shifted)",
-        use_column_width=True,
-    )
-
-    st.markdown(
-        r"""
-        ## Convolution Operation: Integration and Summation
-
-        ### Definition
-        In statistics and signal processing, convolution is a mathematical operation that combines two functions to produce a third function. It represents the amount of overlap between one function as it is shifted over another.
-
-        ### In Continuous and Discrete Forms
-        - **Continuous Convolution**: In continuous domains, it is defined as the integral of the product of the two functions after one is reversed and shifted. For functions $ f $ and $ g $, the convolution is written as:
-
-        $$ (f * g)(t) = \int_{-\infty}^{\infty} f(\tau) g(t - \tau) d\tau $$
-
-        - **Discrete Convolution**: In discrete domains, such as in signal processing or image analysis, convolution is typically expressed as a summation:
-
-        $$ (f * g)[n] = \sum_{m=-\infty}^{\infty} f[m] g[n - m] $$
-       """
-    )
-
-    video_file_convop = open("./videos/5_convop.mp4", "rb")
-    video_bytes_convop = video_file_convop.read()
-    st.video(video_bytes_convop)
-
-    st.markdown(
-        r"""
-        ## Image Data and Its Dimensions
-
-        ### Image Data Structure
-        Image data is fundamentally different from tabular data. An image can be represented as a matrix of pixel values. In terms of dimensions:
-
-        - **Width and Height**: These are the dimensions of the image in pixels.
-        - **3 in (Width, Height, 3)**: This represents the three color channels, typically Red, Green, and Blue (RGB). Each pixel has three values corresponding to the intensity of each color channel.
-
-        ### Contrast with Tabular Data
-        Tabular data is structured in rows and columns, each column potentially being a different feature type (numerical, categorical, etc.), whereas image data is a consistent array of pixel intensity values across its dimensions.
-
-        ## Motivation for Convolution Operation in Image Processing
-
-        Convolution in image processing is used for various purposes such as:
-
-        - **Feature Detection**: Detecting features like edges, corners, or textures.
-        - **Image Blurring and Sharpening**: Smoothing or emphasizing details in an image.
-        - **Noise Reduction**: Eliminating random variations in the image's intensity.
-
-        These operations are achieved by applying different convolution kernels (filters) over the image.
-       """
-    )
-
-    video_file_convop_imagedata = open("./videos/5_convop_imagedata.mp4", "rb")
-    video_bytes_convop_imagedata = video_file_convop_imagedata.read()
-    st.video(video_bytes_convop_imagedata)
-
-    st.markdown(
-        r"""
-        ## Applying Convolution Operation to 2D Data
-
-        ### Convolution with Matrices
-        Convolution can be applied to 2D data like images using matrices. If $ A $ is an image matrix and $ B $ is a kernel matrix, the convolution $ C $ is computed as follows:
-
-        - For each element $ C[i, j] $ of the convolution matrix:
-        $$ C[i, j] = \sum_{u} \sum_{v} A[i + u, j + v] \cdot B[u, v] $$
-
-        - Here, $ A[i + u, j + v] $ is a pixel value from the image, and $ B[u, v] $ is a value from the kernel. The kernel is slid over the image, and at each position, the product of the overlapped values of $ A $ and $ B $ is computed and summed to produce an entry in the output matrix $ C $.
-
-        This operation effectively applies a filter to the image, transforming it in a way that depends on the properties of the kernel.
-       """
-    )
-
-    video_file_convop_matrices = open("./videos/5_convop_matrices.mp4", "rb")
-    video_bytes_convop_matrices = video_file_convop_matrices.read()
-    st.video(video_bytes_convop_matrices)
-
-    st.markdown(
-        r"""
-        ## TensorFlow Conv1D and Conv2D API Explanation
-
-        This document provides a brief explanation of TensorFlow's Conv1D and Conv2D APIs, with example code snippets to illustrate their usage.
-
-        ### Conv1D in TensorFlow
-
-        `Conv1D` is used for convolutional operations on 1-dimensional data, commonly seen in time-series analysis or natural language processing.
-
-        #### Example Code:
-
-        ```python
-        import tensorflow as tf
-
-        # Example of a Conv1D layer
-        model = tf.keras.Sequential([
-            tf.keras.layers.Conv1D(filters=32, kernel_size=3, activation='relu', input_shape=(10, 64))
-        ])
-        ```
-
-        - `filters=32`: Specifies the number of output filters in the convolution.
-        - `kernel_size=3`: The length of the 1D convolution window.
-        - `activation='relu'`: The activation function used after the convolution.
-        - `input_shape=(10, 64)`: The shape of the input data (10 time steps, 64 features per step).
-
-        `Conv1D` is ideal for data where the relationship between consecutive data points is significant.
-
-        ### Conv2D in TensorFlow
-
-        `Conv2D` is used for convolutional operations on 2-dimensional data, like images.
-
-        #### Example Code:
-
-        ```python
-        # Example of a Conv2D layer
-        model = tf.keras.Sequential([
-            tf.keras.layers.Conv2D(filters=64, kernel_size=(3, 3), activation='relu', input_shape=(28, 28, 1))
-        ])
-        ```
-
-        - `filters=64`: Number of output filters in the convolution.
-        - `kernel_size=(3, 3)`: The height and width of the 2D convolution window.
-        - `activation='relu'`: Activation function after the convolution.
-        - `input_shape=(28, 28, 1)`: Shape of input data (28x28 pixels, 1 color channel).
-
-        `Conv2D` is widely used in image processing tasks as it helps the model learn spatial hierarchies in images.
-
-        ---
-
-        Both `Conv1D` and `Conv2D` are crucial layers in building neural networks for processing sequential data and images, respectively. They help in extracting important features from the data, which is essential for tasks like classification, recognition, and prediction.
-
-        """
-    )
-
-
-page_for_convop()

pages/5_Convolutional_Neural_Networks_Advanced.py

@@ -1,651 +0,0 @@
-def page_for_cnnadvanced():
-    import streamlit as st
-
-    st.write("# Welcome to Advanced Convolutional Neural Networks! 👋")
-    st.sidebar.success("Select a session from the menu of content.")
-
-    st.markdown(
-        r"""
-        """
-    )
-
-    st.markdown(
-        r"""
-        ## VGG16
-
-        VGG16 is a deep convolutional neural network (CNN) architecture developed by the Visual Geometry Group at the University of Oxford. Here's a simple implementation of the VGG16 model using TensorFlow 2.x and Keras Sequential API:
-        """
-    )
-
-    st.image(
-        "https://datagen.tech/wp-content/uploads/2022/11/image2-1.png",
-        caption="VGG16",
-        use_column_width=True,
-    )
-
-    with st.expander("Code💻"):
-        st.markdown(
-            r"""
-            ```py
-            import tensorflow as tf
-            from tensorflow.keras import layers, models
-
-            def build_vgg16_model(input_shape=(224, 224, 3), num_classes=1000):
-                model = models.Sequential()
-
-                # Block 1: 2x Convolutional Layers + Max Pooling
-                model.add(layers.Conv2D(64, kernel_size=(3, 3), padding='same', activation='relu', input_shape=input_shape))
-                model.add(layers.Conv2D(64, kernel_size=(3, 3), padding='same', activation='relu'))
-                model.add(layers.MaxPooling2D(pool_size=(2, 2), strides=(2, 2)))
-
-                # Block 2: 2x Convolutional Layers + Max Pooling
-                model.add(layers.Conv2D(128, kernel_size=(3, 3), padding='same', activation='relu'))
-                model.add(layers.Conv2D(128, kernel_size=(3, 3), padding='same', activation='relu'))
-                model.add(layers.MaxPooling2D(pool_size=(2, 2), strides=(2, 2)))
-
-                # Block 3: 3x Convolutional Layers + Max Pooling
-                model.add(layers.Conv2D(256, kernel_size=(3, 3), padding='same', activation='relu'))
-                model.add(layers.Conv2D(256, kernel_size=(3, 3), padding='same', activation='relu'))
-                model.add(layers.Conv2D(256, kernel_size=(3, 3), padding='same', activation='relu'))
-                model.add(layers.MaxPooling2D(pool_size=(2, 2), strides=(2, 2)))
-
-                # Block 4: 3x Convolutional Layers + Max Pooling
-                model.add(layers.Conv2D(512, kernel_size=(3, 3), padding='same', activation='relu'))
-                model.add(layers.Conv2D(512, kernel_size=(3, 3), padding='same', activation='relu'))
-                model.add(layers.Conv2D(512, kernel_size=(3, 3), padding='same', activation='relu'))
-                model.add(layers.MaxPooling2D(pool_size=(2, 2), strides=(2, 2)))
-
-                # Block 5: 3x Convolutional Layers + Max Pooling
-                model.add(layers.Conv2D(512, kernel_size=(3, 3), padding='same', activation='relu'))
-                model.add(layers.Conv2D(512, kernel_size=(3, 3), padding='same', activation='relu'))
-                model.add(layers.Conv2D(512, kernel_size=(3, 3), padding='same', activation='relu'))
-                model.add(layers.MaxPooling2D(pool_size=(2, 2), strides=(2, 2)))
-
-                # Flatten the output
-                model.add(layers.Flatten())
-
-                # Fully Connected Layers
-                model.add(layers.Dense(4096, activation='relu'))
-                model.add(layers.Dropout(0.5))
-                model.add(layers.Dense(4096, activation='relu'))
-                model.add(layers.Dropout(0.5))
-
-                # Output Layer
-                model.add(layers.Dense(num_classes, activation='softmax'))
-
-                return model
-
-            # Build the VGG16 model
-            vgg16_model = build_vgg16_model(input_shape=(224, 224, 3), num_classes=1000)
-            vgg16_model.summary
-            ```
-            """
-        )
-
-    st.markdown(
-        r"""
-        VGG16 is introduced in a paper titled "Very Deep Convolutional Networks for Large-Scale Image Recognition" by K. Simonyan and A. Zisserman.
-
-        Here is the link to the VGG16 paper: 
-
-        [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://arxiv.org/abs/1409.1556)
-        """
-    )
-
-    st.markdown(
-        r"""
-        ## ResNet
-
-        ResNet (Residual Network) is a deep convolutional neural network (CNN) architecture that introduces residual connections to improve learning and avoid the vanishing gradient problem in deep networks. Here's a simple implementation of the ResNet-50 model using TensorFlow 2.x and Keras functional API:
-        """
-    )
-
-    st.image(
-        "https://production-media.paperswithcode.com/methods/resnet-e1548261477164_2_mD02h5A.png",
-        caption="ResBlock",
-        use_column_width=True,
-    )
-
-    with st.expander("Code💻"):
-        st.markdown(
-            r"""
-            ```py
-            import tensorflow as tf
-            from tensorflow.keras import layers, models, Input
-
-            def identity_block(input_tensor, kernel_size, filters):
-                filters1, filters2, filters3 = filters
-
-                x = layers.Conv2D(filters1, (1, 1))(input_tensor)
-                x = layers.BatchNormalization()(x)
-                x = layers.Activation('relu')(x)
-
-                x = layers.Conv2D(filters2, kernel_size, padding='same')(x)
-                x = layers.BatchNormalization()(x)
-                x = layers.Activation('relu')(x)
-
-                x = layers.Conv2D(filters3, (1, 1))(x)
-                x = layers.BatchNormalization()(x)
-
-                x = layers.add([x, input_tensor])
-                x = layers.Activation('relu')(x)
-                return x
-
-            def conv_block(input_tensor, kernel_size, filters, strides=(2, 2)):
-                filters1, filters2, filters3 = filters
-
-                x = layers.Conv2D(filters1, (1, 1), strides=strides)(input_tensor)
-                x = layers.BatchNormalization()(x)
-                x = layers.Activation('relu')(x)
-
-                x = layers.Conv2D(filters2, kernel_size, padding='same')(x)
-                x = layers.BatchNormalization()(x)
-                x = layers.Activation('relu')(x)
-
-                x = layers.Conv2D(filters3, (1, 1))(x)
-                x = layers.BatchNormalization()(x)
-
-                shortcut = layers.Conv2D(filters3, (1, 1), strides=strides)(input_tensor)
-                shortcut = layers.BatchNormalization()(shortcut)
-
-                x = layers.add([x, shortcut])
-                x = layers.Activation('relu')(x)
-                return x
-
-            def build_resnet50(input_shape=(224, 224, 3), num_classes=1000):
-                input_tensor = Input(shape=input_shape)
-
-                # Initial convolution layer
-                x = layers.ZeroPadding2D(padding=(3, 3))(input_tensor)
-                x = layers.Conv2D(64, (7, 7), strides=(2, 2))(x)
-                x = layers.BatchNormalization()(x)
-                x = layers.Activation('relu')(x)
-                x = layers.ZeroPadding2D(padding=(1, 1))(x)
-                x = layers.MaxPooling2D((3, 3), strides=(2, 2))(x)
-
-                # Residual blocks
-                x = conv_block(x, 3, [64, 64, 256], strides=(1, 1))
-                x = identity_block(x, 3, [64, 64, 256])
-                x = identity_block(x, 3, [64, 64, 256])
-
-                x = conv_block(x, 3, [128, 128, 512])
-                x = identity_block(x, 3, [128, 128, 512])
-                x = identity_block(x, 3, [128, 128, 512])
-                x = identity_block(x, 3, [128, 128, 512])
-
-                x = conv_block(x, 3, [256, 256, 1024])
-                x = identity_block(x, 3, [256, 256, 1024])
-                x = identity_block(x, 3, [256, 256, 1024])
-                x = identity_block(x, 3, [256, 256, 1024])
-                x = identity_block(x, 3, [256, 256, 1024])
-
-                x = conv_block(x, 3, [512, 512, 2048])
-                x = identity_block(x, 3, [512, 512, 2048])
-                x = identity_block(x, 3, [512, 512, 2048])
-
-                # Final part
-                x = layers.GlobalAveragePooling2D()(x)
-                x = layers.Dense(num_classes, activation='softmax')(x)
-
-                # Create model
-                model = Model(input_tensor, x)
-
-                return model
-            ```
-            """
-        )
-
-    st.markdown(
-        r"""
-        ResNet was introduced in the paper "Deep Residual Learning for Image Recognition" by Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Here's the link to the ResNet paper: [Deep Residual Learning for Image Recognition](https://arxiv.org/pdf/1512.03385.pdf)
-
-        ## DenseNet
-
-        DenseNet (Densely Connected Convolutional Networks) is a deep convolutional neural network (CNN) architecture that introduces dense connections between layers to improve learning and parameter efficiency. Here's a simple implementation of the DenseNet-121 model using TensorFlow 2.x and Keras functional API:
-        """
-    )
-
-    st.image(
-        "https://pytorch.org/assets/images/densenet1.png",
-        caption="DenseNet",
-        use_column_width=True,
-    )
-
-    with st.expander("Code💻"):
-        st.markdown(
-            r"""
-            ```py
-            import tensorflow as tf
-            from tensorflow.keras import layers, models, Input
-
-            def dense_block(x, num_layers, growth_rate):
-                for _ in range(num_layers):
-                    output = layers.BatchNormalization()(x)
-                    output = layers.Activation('relu')(output)
-                    output = layers.Conv2D(4 * growth_rate, (1, 1), padding='same', kernel_initializer='he_normal')(output)
-                    output = layers.BatchNormalization()(output)
-                    output = layers.Activation('relu')(output)
-                    output = layers.Conv2D(growth_rate, (3, 3), padding='same', kernel_initializer='he_normal')(output)
-                    x = layers.Concatenate()([x, output])
-                return x
-
-            def transition_layer(x, compression_factor):
-                num_filters = int(x.shape[-1] * compression_factor)
-                x = layers.BatchNormalization()(x)
-                x = layers.Activation('relu')(x)
-                x = layers.Conv2D(num_filters, (1, 1), padding='same', kernel_initializer='he_normal')(x)
-                x = layers.AveragePooling2D((2, 2), strides=(2, 2))(x)
-                return x
-
-            def build_densenet121(input_shape=(224, 224, 3), num_classes=1000, growth_rate=32, compression_factor=0.5):
-                input_tensor = Input(shape=input_shape)
-
-                # Initial convolution layer
-                x = layers.Conv2D(2 * growth_rate, (7, 7), strides=(2, 2), padding='same', kernel_initializer='he_normal')(input_tensor)
-                x = layers.BatchNormalization()(x)
-                x = layers.Activation('relu')(x)
-                x = layers.MaxPooling2D((3, 3), strides=(2, 2), padding='same')(x)
-
-                # Dense blocks and transition layers
-                x = dense_block(x, 6, growth_rate)
-                x = transition_layer(x, compression_factor)
-
-                x = dense_block(x, 12, growth_rate)
-                x = transition_layer(x, compression_factor)
-
-                x = dense_block(x, 24, growth_rate)
-                x = transition_layer(x, compression_factor)
-
-                x = dense_block(x, 16, growth_rate)
-
-                # Global average pooling and output layer
-                x = layers.GlobalAveragePooling2D()(x)
-                x = layers.Dense(num_classes, activation='softmax')(x)
-
-                model = models.Model(input_tensor, x, name='densenet121')
-                return model
-
-            # Build the DenseNet-121 model
-            densenet121_model = build_densenet121(input_shape=(224, 224, 3), num_classes=1000)
-            densenet121_model.summary()
-            ```
-            """
-        )
-
-    st.markdown(
-        r"""
-        DenseNet was introduced in the paper "Densely Connected Convolutional Networks" by Gao Huang, Zhuang Liu, Laurens van der Maaten, and Kilian Q. Weinberger. Here is the link to the original paper for Densenet: [Densely Connected Convolutional Networks](https://arxiv.org/abs/1608.06993)
-        """
-    )
-
-    st.markdown(
-        r"""
-        ## Inception
-
-        Inception is a deep convolutional neural network (CNN) architecture that was introduced in the GoogLeNet model. It uses inception modules to efficiently learn different features at multiple scales. Here's a simple implementation of the Inception V1 (GoogLeNet) model using TensorFlow 2.x and Keras functional API:
-        """
-    )
-
-    st.image(
-        "https://media5.datahacker.rs/2018/11/Featured-Image-017-CNN-Inception-Network-1.jpg",
-        caption="Inception",
-        use_column_width=True,
-    )
-
-    with st.expander("Code💻"):
-        st.markdown(
-            r"""
-            ```py
-            import tensorflow as tf
-            from tensorflow.keras import layers, models, Input
-
-            def inception_module(x, filters_1x1, filters_3x3_reduce, filters_3x3, filters_5x5_reduce, filters_5x5, filters_pool_proj):
-                conv_1x1 = layers.Conv2D(filters_1x1, (1, 1), padding='same', activation='relu', kernel_initializer='he_normal')(x)
-                
-                conv_3x3_reduce = layers.Conv2D(filters_3x3_reduce, (1, 1), padding='same', activation='relu', kernel_initializer='he_normal')(x)
-                conv_3x3 = layers.Conv2D(filters_3x3, (3, 3), padding='same', activation='relu', kernel_initializer='he_normal')(conv_3x3_reduce)
-                
-                conv_5x5_reduce = layers.Conv2D(filters_5x5_reduce, (1, 1), padding='same', activation='relu', kernel_initializer='he_normal')(x)
-                conv_5x5 = layers.Conv2D(filters_5x5, (5, 5), padding='same', activation='relu', kernel_initializer='he_normal')(conv_5x5_reduce)
-                
-                max_pool = layers.MaxPooling2D((3, 3), strides=(1, 1), padding='same')(x)
-                pool_proj = layers.Conv2D(filters_pool_proj, (1, 1), padding='same', activation='relu', kernel_initializer='he_normal')(max_pool)
-                
-                output = layers.Concatenate(axis=-1)([conv_1x1, conv_3x3, conv_5x5, pool_proj])
-                return output
-
-            def build_inception_v1(input_shape=(224, 224, 3), num_classes=1000):
-                input_tensor = Input(shape=input_shape)
-
-                # Initial convolution and max pooling layers
-                x = layers.Conv2D(64, (7, 7), strides=(2, 2), padding='same', activation='relu', kernel_initializer='he_normal')(input_tensor)
-                x = layers.MaxPooling2D((3, 3), strides=(2, 2), padding='same')(x)
-
-                # Local response normalization
-                x = layers.BatchNormalization()(x)
-
-                # Convolution and max pooling layers
-                x = layers.Conv2D(64, (1, 1), strides=(1, 1), padding='same', activation='relu', kernel_initializer='he_normal')(x)
-                x = layers.Conv2D(192, (3, 3), strides=(1, 1), padding='same', activation='relu', kernel_initializer='he_normal')(x)
-
-                # Local response normalization
-                x = layers.BatchNormalization()(x)
-
-                x = layers.MaxPooling2D((3, 3), strides=(2, 2), padding='same')(x)
-
-                # Inception modules
-                x = inception_module(x, filters_1x1=64, filters_3x3_reduce=96, filters_3x3=128, filters_5x5_reduce=16, filters_5x5=32, filters_pool_proj=32)
-                x = inception_module(x, filters_1x1=128, filters_
-                x = inception_module(x, filters_1x1=128, filters_3x3_reduce=128, filters_3x3=192, filters_5x5_reduce=32, filters_5x5=96, filters_pool_proj=64)
-                x = layers.MaxPooling2D((3, 3), strides=(2, 2), padding='same')(x)
-                
-                x = inception_module(x, filters_1x1=192, filters_3x3_reduce=96, filters_3x3=208, filters_5x5_reduce=16, filters_5x5=48, filters_pool_proj=64)
-                x = inception_module(x, filters_1x1=160, filters_3x3_reduce=112, filters_3x3=224, filters_5x5_reduce=24, filters_5x5=64, filters_pool_proj=64)
-                x = inception_module(x, filters_1x1=128, filters_3x3_reduce=128, filters_3x3=256, filters_5x5_reduce=24, filters_5x5=64, filters_pool_proj=64)
-                x = inception_module(x, filters_1x1=112, filters_3x3_reduce=144, filters_3x3=288, filters_5x5_reduce=32, filters_5x5=64, filters_pool_proj=64)
-                x = inception_module(x, filters_1x1=256, filters_3x3_reduce=160, filters_3x3=320, filters_5x5_reduce=32, filters_5x5=128, filters_pool_proj=128)
-                x = layers.MaxPooling2D((3, 3), strides=(2, 2), padding='same')(x)
-                
-                x = inception_module(x, filters_1x1=256, filters_3x3_reduce=160, filters_3x3=320, filters_5x5_reduce=32, filters_5x5=128, filters_pool_proj=128)
-                x = inception_module(x, filters_1x1=384, filters_3x3_reduce=192, filters_3x3=384, filters_5x5_reduce=48, filters_5x5=128, filters_pool_proj=128)
-
-                # Global average pooling and output layer
-                x = layers.GlobalAveragePooling2D()(x)
-                x = layers.Dropout(0.4)(x)
-                x = layers.Dense(num_classes, activation='softmax')(x)
-
-                model = models.Model(input_tensor, x, name='inception_v1')
-                return model
-
-            # Build the Inception V1 (GoogLeNet) model
-            inception_v1_model = build_inception_v1(input_shape=(224, 224, 3), num_classes=1000)
-            inception_v1_model.summary()
-            ```
-            """
-        )
-
-    st.markdown(
-        r"""
-        The paper that introduced the Inception model is titled "Going deeper with convolutions" by Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. The paper for Inception model can be found at this link: [Going deeper with convolutions](https://arxiv.org/abs/1409.4842)
-
-        """
-    )
-
-    st.markdown(
-        r"""
-        ## Vision Transformer (ViT)
-
-        The [Vision Transformer (ViT)](https://arxiv.org/pdf/2010.11929.pdf) model represents a significant leap in the field of computer vision and was first introduced in a groundbreaking research paper at the International Conference on Learning Representations (ICLR) 2021. The paper, titled "An Image is Worth 16*16 Words: Transformers for Image Recognition at Scale," not only presented this innovative model but also laid the foundation for its widespread application in various image recognition tasks. The ViT model is a testament to the evolving landscape of neural network architectures, particularly in how they handle image data.
-
-        Unlike traditional approaches that primarily rely on convolutional neural networks (CNNs), the ViT model adopts the transformer architecture, which has been immensely successful in natural language processing. This approach treats an image as a sequence of patches, much like words in a sentence, allowing the model to interpret and analyze images in a novel way.
-
-        Key steps in the Vision Transformer architecture include:
-        1. Splitting an image into fixed-size patches.
-        2. Flattening these patches.
-        3. Transforming the flattened patches into lower-dimensional linear embeddings.
-        4. Adding positional embeddings to retain the order of the patches.
-        5. Feeding these sequences into a standard transformer encoder, similar to those used in language models.
-
-        The initial training of ViT models involves leveraging large datasets like ImageNet and ImageNet-21k. This process, known as pretraining, is fully supervised and utilizes image labels to teach the model the fundamentals of image recognition. Subsequently, the model undergoes finetuning, where it is further trained on specific downstream datasets for tasks such as image classification.
-
-        The flexibility and efficiency of Vision Transformers have led to their adoption in a variety of applications beyond mere image classification. These include object detection, image segmentation, action recognition, and even in more complex tasks like generative modeling, visual grounding, visual-question answering, and visual reasoning.
-
-        Overall, the introduction of the Vision Transformer model marks a significant advancement in the way neural networks approach visual data, offering a fresh perspective and new possibilities in the field of computer vision.
-        """
-    )
-
-    st.image(
-        "https://production-media.paperswithcode.com/methods/Screen_Shot_2021-01-26_at_9.43.31_PM_uI4jjMq.png",
-        caption="Vision Transformer (ViT)",
-        use_column_width=True,
-    )
-
-    with st.expander("Code💻"):
-        st.markdown(
-            r"""
-            ```python
-            from typing import Any, Dict
-
-            import matplotlib.pyplot as plt
-            import numpy as np
-            import tensorflow as tf
-            import tensorflow_addons as tfa
-            from tensorflow import keras
-            from tensorflow.keras import layers
-
-
-            # learning_rate = 0.001
-            # weight_decay = 0.0001
-            # batch_size = 256
-            # num_epochs = 800
-            # image_size = 72  # We'll resize input images to this size
-            # patch_size = 6  # Size of the patches to be extract from the input images
-            # num_patches = (image_size // patch_size) ** 2
-            # projection_dim = 64
-            # num_heads = 4
-            # transformer_units = [
-            #     projection_dim * 2,
-            #     projection_dim,
-            # ]  # Size of the transformer layers
-            # transformer_layers = 10
-            # mlp_head_units = [2048, 1024]  # Size of the dense layers of the final classifier
-
-
-            def mlp(x: tf.Tensor, hidden_units: list[int], dropout_rate: float) -> tf.Tensor:
-                # Constructs a multi-layer perceptron with Gelu activation and dropout layers.
-
-                # Args:
-                #     x (tf.Tensor): Input tensor to the MLP.
-                #     hidden_units (list[int]): A list of integers for the number of units in each hidden layer.
-                #     dropout_rate (float): The rate of dropout to apply after each hidden layer.
-
-                # Returns:
-                #     A tensor representing the output of the MLP.
-                # For each specified number of hidden units,
-                # add a dense layer with gelu activation followed by dropout.
-                for units in hidden_units:
-                    x = layers.Dense(units, activation=tf.nn.gelu)(x)
-                    x = layers.Dropout(dropout_rate)(x)
-
-                # Return the final output after passing through all the hidden layers
-                return x
-
-
-            class Patches(layers.Layer):
-                def __init__(self, patch_size: int):
-                    # A layer that extracts patches from an image tensor.
-
-                    # Args:
-                    #     patch_size (int): The size of each patch to extract.
-                    super(Patches, self).__init__()
-                    self.patch_size = patch_size
-
-                def call(self, images: tf.Tensor) -> tf.Tensor:
-                    # Extracts patches from the input image tensor.
-
-                    # Args:
-                    #     images (tf.Tensor): A tensor representing a batch of images.
-
-                    # Returns:
-                    #     A tensor representing the extracted patches.
-                    batch_size = tf.shape(images)[0]
-                    patches = tf.image.extract_patches(
-                        images=images,
-                        sizes=[1, self.patch_size, self.patch_size, 1],
-                        strides=[1, self.patch_size, self.patch_size, 1],
-                        rates=[1, 1, 1, 1],
-                        padding="VALID",
-                    )
-                    patch_dims = patches.shape[-1]
-                    patches = tf.reshape(patches, [batch_size, -1, patch_dims])
-                    return patches
-
-
-            def plot_patches(sample_index: int) -> None:
-                # Plots the original image and patches extracted from it.
-
-                # Args:
-                #     sample_index (int): The index of the image to plot from the training set.
-                plt.figure(figsize=(4, 4))
-                # Select a random image from the training set
-                image = x_train[np.random.choice(range(x_train.shape[sample_index]))]
-                plt.imshow(image.astype("uint8"))
-                plt.axis("off")
-
-                resized_image = tf.image.resize(
-                    tf.convert_to_tensor([image]), size=(image_size, image_size)
-                )
-                # Extract patches from the resized image
-                patches = Patches(patch_size)(resized_image)
-                print(f"Image size: {image_size} X {image_size}")
-                print(f"Patch size: {patch_size} X {patch_size}")
-                print(f"Patches per image: {patches.shape[1]}")
-                print(f"Elements per patch: {patches.shape[-1]}")
-
-                n = int(np.sqrt(patches.shape[1]))
-                plt.figure(figsize=(4, 4))
-                # Plot all patches as subplots on a grid
-                for i, patch in enumerate(patches[sample_index]):
-                    ax = plt.subplot(n, n, i + 1)
-                    patch_img = tf.reshape(patch, (patch_size, patch_size, 3))
-                    plt.imshow(patch_img.numpy().astype("uint8"))
-                    plt.axis("off")
-
-
-            class PatchEncoder(layers.Layer):
-                def __init__(self, num_patches: int, projection_dim: int):
-                    # Initializes the PatchEncoder layer.
-
-                    # Args:
-                    #     num_patches (int): The number of patches to extract from an image.
-                    #     projection_dim (int): The dimensionality of the encoding space.
-                    super(PatchEncoder, self).__init__()
-                    self.num_patches = num_patches
-                    # A Dense layer to project the patches into the encoding space
-                    self.projection = layers.Dense(units=projection_dim)
-                    # An Embedding layer to provide position embeddings for each patch
-                    self.position_embedding = layers.Embedding(
-                        input_dim=num_patches, output_dim=projection_dim
-                    )
-
-                def call(self, patch):
-                    # Encodes a patch by projecting it into the encoding space and adding a
-                    # position embedding.
-
-                    # Args:
-                    #     patch (tf.Tensor): A patch extracted from an image.
-
-                    # Returns:
-                    #     tf.Tensor: The encoded patch, with shape (batch_size, projection_dim).
-                    positions = tf.range(start=0, limit=self.num_patches, delta=1)
-                    encoded = self.projection(patch) + self.position_embedding(positions)
-                    return encoded
-
-
-            def create_vit_classifier() -> keras.Model:
-                # Creates a Vision Transformer (ViT) classifier model.
-
-                # Returns:
-                #     keras.Model: A ViT classifier model.
-                inputs = layers.Input(shape=input_shape)
-                # Augment data.
-                augmented = data_augmentation(inputs)
-                # Create patches.
-                patches = Patches(patch_size)(augmented)
-                # Encode patches.
-                encoded_patches = PatchEncoder(num_patches, projection_dim)(patches)
-
-                # Create multiple layers of the Transformer block.
-                for _ in range(transformer_layers):
-                    # Layer normalization 1.
-                    x1 = layers.LayerNormalization(epsilon=1e-6)(encoded_patches)
-                    # Create a multi-head attention layer.
-                    attention_output = layers.MultiHeadAttention(
-                        num_heads=num_heads, key_dim=projection_dim, dropout=0.1
-                    )(x1, x1)
-                    # Skip connection 1.
-                    x2 = layers.Add()([attention_output, encoded_patches])
-                    # Layer normalization 2.
-                    x3 = layers.LayerNormalization(epsilon=1e-6)(x2)
-                    # MLP.
-                    x3 = mlp(x3, hidden_units=transformer_units, dropout_rate=0.1)
-                    # Skip connection 2.
-                    encoded_patches = layers.Add()([x3, x2])
-
-                # Create a [batch_size, projection_dim] tensor.
-                representation = layers.LayerNormalization(epsilon=1e-6)(encoded_patches)
-                representation = layers.Flatten()(representation)
-                representation = layers.Dropout(0.5)(representation)
-                # Add MLP.
-                features = mlp(representation, hidden_units=mlp_head_units, dropout_rate=0.5)
-                # Classify outputs.
-                logits = layers.Dense(num_classes)(features)
-                # Create the Keras model.
-                model = keras.Model(inputs=inputs, outputs=logits)
-                return model
-
-
-            def run_experiment(model: keras.Model) -> Dict[str, Any]:
-                # Trains a given Keras model on training data, evaluates it on test data,
-                # and returns the training history.
-
-                # Args:
-                #     model (keras.Model): A Keras model to train and evaluate.
-
-                # Returns:
-                #     dict: A dictionary containing the training history.
-                optimizer = tfa.optimizers.AdamW(
-                    learning_rate=learning_rate, weight_decay=weight_decay
-                )
-
-                model.compile(
-                    optimizer=optimizer,
-                    loss=keras.losses.SparseCategoricalCrossentropy(from_logits=True),
-                    metrics=[
-                        keras.metrics.SparseCategoricalAccuracy(name="accuracy"),
-                        keras.metrics.SparseTopKCategoricalAccuracy(5, name="top-5-accuracy"),
-                    ],
-                )
-
-                checkpoint_filepath = "/tmp/checkpoint"
-                checkpoint_callback = keras.callbacks.ModelCheckpoint(
-                    checkpoint_filepath,
-                    monitor="val_accuracy",
-                    save_best_only=True,
-                    save_weights_only=True,
-                )
-
-                history = model.fit(
-                    x=x_train,
-                    y=y_train,
-                    batch_size=batch_size,
-                    epochs=num_epochs,
-                    validation_split=0.1,
-                    callbacks=[checkpoint_callback],
-                )
-
-                model.load_weights(checkpoint_filepath)
-                _, accuracy, top_5_accuracy = model.evaluate(x_test, y_test)
-                print(f"Test accuracy: {round(accuracy * 100, 2)}%")
-                print(f"Test top 5 accuracy: {round(top_5_accuracy * 100, 2)}%")
-
-                return history
-            ```
-            """
-        )
-
-    st.markdown(
-        r"""
-        Related work driving the momentum of ViT:
-        - [Attention Is All You Need](https://arxiv.org/abs/1706.03762)
-        - [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://arxiv.org/abs/2010.11929)
-        - [Do Vision Transformers See Like Convolutional Neural Networks?](https://arxiv.org/abs/2108.08810)
-        - [Vision Transformers for Dense Prediction](https://arxiv.org/abs/2103.13413)
-        - [VT-ADL: A Vision Transformer Network for Image Anomaly Detection and Localization](https://arxiv.org/abs/2104.10036)
-        - [ViViT: A Video Vision Transformer](https://openaccess.thecvf.com/content/ICCV2021/papers/Arnab_ViViT_A_Video_Vision_Transformer_ICCV_2021_paper.pdf)
-        - [Autonomous Driving: Tesla FSD - image-to-BEV transform + multi-camera fusion](https://www.youtube.com/live/ODSJsviD_SU?si=14Q3If7cBME69hkc)
-        """
-    )
-
-
-page_for_cnnadvanced()

pages/6_Autoencoders.py

@@ -1,154 +0,0 @@
-def page_for_ae():
-    import streamlit as st
-
-    st.write("# Welcome to Autoencoder and Variational Autoencoder! 👋")
-    st.sidebar.success("Select a session from the menu of content.")
-
-    st.markdown(
-        r"""
-        # Understanding Autoencoders and Variational Autoencoders
-
-        ## Why Use Autoencoders?
-
-        Autoencoders are a class of neural network models aimed at learning efficient representations (encodings) of the input data, typically for the purpose of dimensionality reduction or feature extraction. By learning to ignore "noise" in the input data, autoencoders focus on reconstructing the most important aspects, effectively learning the most salient features of the training data. They are particularly useful in unsupervised learning tasks where you're dealing with unlabeled data. Autoencoders have wide applications ranging from anomaly detection to image denoising, and they serve as a building block for more complex models such as variational autoencoders.
-
-        ## What is an Autoencoder?
-
-        An autoencoder is a type of artificial neural network used to learn efficient codings of input data. It consists of two main parts: the encoder and the decoder. The encoder compresses the input into a latent-space representation, and the decoder reconstructs the input from this representation. 
-        """
-    )
-
-    st.image(
-        "https://blog.keras.io/img/ae/autoencoder_schema.jpg",
-        caption="Visualization of an Autoencoder (intuition)",
-        use_column_width=True,
-    )
-
-    st.markdown(
-        r"""
-        ### Mathematical Form
-
-        The mathematical formulation of a simple autoencoder can be represented as follows:
-
-        1. **Encoder**: This maps the input $ x $ to a latent representation $ h $, denoted as $ h = f(x) $. Here, $ f $ is the encoding function.
-        2. **Latent Layer**: This layer represents the compressed knowledge of the input data.
-        3. **Decoder**: The decoder maps the latent representation $ h $ back to a reconstruction $ r $, which is as close as possible to the original input $ x $. This is denoted as $ r = g(h) $, where $ g $ is the decoding function.
-
-        For instance, in an autoencoder trained on the MNIST dataset, the encoder would compress the 28x28 pixel input images into a smaller representation, and the decoder would attempt to reconstruct the original images from this compressed form.
-
-        """
-    )
-
-    st.image(
-        "https://miro.medium.com/max/1400/0*_K4yRm8ILDZ02vPq.png",
-        caption="Visualization of an Autoencoder (mathematical)",
-        use_column_width=True,
-    )
-
-    video_file_manytomany_ae = open("./videos/7_manytomany_ae.mp4", "rb")
-    video_bytes_manytomany_ae = video_file_manytomany_ae.read()
-    st.video(video_bytes_manytomany_ae)
-
-    st.markdown(
-        r"""
-        ## Autoencoder Implementation in TensorFlow
-
-        ```python
-        import tensorflow as tf
-
-        # Define the encoder part
-        encoder = tf.keras.layers.Dense(units=64, activation='relu', input_shape=(784,))
-
-        # Define the latent space
-        latent_space = tf.keras.layers.Dense(units=32, activation='relu')
-
-        # Define the decoder part
-        decoder = tf.keras.layers.Dense(units=784, activation='sigmoid')
-
-        # Build the autoencoder model
-        autoencoder_model = tf.keras.Sequential([encoder, latent_space, decoder])
-
-        # Compile the model
-        autoencoder_model.compile(optimizer='adam', loss='binary_crossentropy')
-
-        # Summary of the model
-        autoencoder_model.summary()
-        ```
-
-        - This code sets up a basic autoencoder in TensorFlow for the MNIST dataset.
-        - The input data is first flattened (since MNIST images are 28x28 pixels, the input shape is 784).
-        - The encoder compresses the input to a 64-unit layer, then to a smaller 32-unit latent layer.
-        - The decoder then reconstructs the input from the latent space representation back to the original 784 units.
-        - The model is compiled using the Adam optimizer and binary cross-entropy loss, suitable for binary image reconstruction.
-
-        ## Motivation for Variational Autoencoders
-
-        Variational Autoencoders (VAEs) go a step further than traditional autoencoders by not only learning to encode and decode data but also by learning the distribution of the data. They are generative models, which means they can generate new instances that are similar to the input data. This capability makes VAEs incredibly powerful for tasks like image generation, style transfer, and more advanced forms of anomaly detection. They introduce a probabilistic approach to the encoding process, making them robust and versatile.
-
-        ## Architecture of Variational Autoencoders
-
-        The architecture of a Variational Autoencoder differs from a standard autoencoder in that it produces two outputs for the encoder: a mean and a variance. These parameters define a distribution for each latent attribute. 
-
-        - The encoder outputs parameters to a statistical distribution, rather than outputting a fixed value for each attribute in the latent space.
-        - During training, a sample is drawn from this distribution and passed to the decoder, which then reconstructs the input data.
-        - This approach forces the model to learn efficient representations of the data in a structured latent space, often leading to better and more controlled generation of new data.
-       """
-    )
-
-    video_file_manytomany_vae = open("./videos/7_manytomany_vae.mp4", "rb")
-    video_bytes_manytomany_vae = video_file_manytomany_vae.read()
-    st.video(video_bytes_manytomany_vae)
-
-    st.markdown(
-        r"""
-        ## Variational Autoencoder Implementation in TensorFlow
-
-        ```python
-        class Sampling(tf.keras.layers.Layer):
-            # Sampling layer to sample from the distribution
-            def call(self, inputs):
-                mean, log_var = inputs
-                batch = tf.shape(mean)[0]
-                dim = tf.shape(mean)[1]
-                epsilon = tf.keras.backend.random_normal(shape=(batch, dim))
-                return mean + tf.exp(0.5 * log_var) * epsilon
-
-        # Encoder
-        vae_encoder_input = tf.keras.Input(shape=(784,))
-        x = tf.keras.layers.Dense(64, activation='relu')(vae_encoder_input)
-        mean = tf.keras.layers.Dense(32)(x)
-        log_var = tf.keras.layers.Dense(
-
-        32)(x)
-        z = Sampling()([mean, log_var])
-
-        # Decoder
-        vae_decoder_input = tf.keras.Input(shape=(32,))
-        x = tf.keras.layers.Dense(64, activation='relu')(vae_decoder_input)
-        vae_decoder_output = tf.keras.layers.Dense(784, activation='sigmoid')(x)
-
-        # VAE model
-        vae_encoder = tf.keras.Model(vae_encoder_input, [mean, log_var, z])
-        vae_decoder = tf.keras.Model(vae_decoder_input, vae_decoder_output)
-        vae_output = vae_decoder(vae_encoder(vae_encoder_input)[2])
-        vae = tf.keras.Model(vae_encoder_input, vae_output)
-
-        # Compile VAE
-        vae.compile(optimizer='adam', loss='binary_crossentropy')
-
-        # VAE Summary
-        vae.summary()
-        ```
-
-        - This TensorFlow code implements a Variational Autoencoder for the MNIST dataset.
-        - The `Sampling` layer samples from the distribution defined by the encoder output.
-        - The encoder outputs the mean and log variance, which are used to sample a point in the latent space.
-        - The decoder reconstructs the input from the sampled latent point.
-        - The model is compiled using the Adam optimizer and binary cross-entropy loss.
-
-        This Variational Autoencoder model can generate new data similar to the input data, offering a powerful tool for generative modeling tasks.
-        """
-    )
-
-
-page_for_ae()