init commit

LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2023 Sibendra Timalsina
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

README.md

@@ -1,3 +1,62 @@
----
-license: openrail
----
+# Nepali Sentiment Analysis
+
+This repository contains code for a Nepali Sentiment Analysis model. The model is trained to predict the sentiment (positive, negative, or neutral) of Nepali text.
+
+## Dataset
+
+The dataset used for training this model is sourced from [Shushant/NepaliSentiment](https://huggingface.co/datasets/Shushant/NepaliSentiment) on the Hugging Face Datasets Hub. The dataset consists of labeled Nepali text samples with corresponding sentiment labels.
+
+## Model Architecture
+
+The model architecture used in this repository is based on the BERT (Bidirectional Encoder Representations from Transformers) model, specifically the `bert-base-multilingual-cased` variant. BERT is a powerful pre-trained language model that can be fine-tuned for various natural language processing tasks, including sentiment analysis.
+
+The model takes Nepali text as input and produces sentiment predictions as output. It uses tokenization techniques to convert the text into numerical representations and leverages the transformer-based architecture to capture contextual relationships between words in the text.
+
+## Usage
+
+To use the Nepali Sentiment Analysis model:
+
+1. Install the required dependencies listed in `requirements.txt`.
+
+2. Load the trained model using `model = BertForSequenceClassification.from_pretrained('bert-base-multilingual-cased', num_labels=3)`.
+
+3. Tokenize your Nepali text data using the same tokenization techniques used during training.
+
+4. Feed the tokenized input to the model and obtain the predicted sentiment labels.
+
+5. Evaluate the model's performance using suitable metrics such as accuracy, precision, recall, or F1-score.
+
+## Repository Structure
+
+- `model_training.ipynb`: Jupyter Notebook containing the code for training the Nepali Sentiment Analysis model.
+- `inference.ipynb`: Jupyter Notebook demonstrating how to perform inference using the trained model.
+- `requirements.txt`: List of required dependencies for running the code.
+- `README.md`: This readme file providing an overview of the repository.
+
+## Acknowledgments
+
+The dataset used in this project is sourced from [Shushant/NepaliSentiment](https://huggingface.co/datasets/Shushant/NepaliSentiment) on the Hugging Face Datasets Hub. I would like to thank the creators and contributors for providing the dataset and making it publicly available.
+
+## Performance and Suggestions
+
+During the training process, the model achieved the following accuracies for different configurations:
+
+- For batch_size=16, learning_rate=2e-5, and epochs=10, the accuracy achieved was 62.17%.
+- For batch_size=16, learning_rate=2e-5, and epochs=20, the accuracy achieved was 61.92%.
+- For batch_size=16, learning_rate=2e-5, and epochs=5, the accuracy achieved was 61.62%.
+
+To improve the accuracy of the model, you can consider making the following changes:
+
+- Adjust the batch size: Try experimenting with different batch sizes to see if it has any impact on the model's performance. Sometimes, a smaller or larger batch size can lead to better results.
+- Fine-tune the learning rate: The learning rate determines the step size at which the model learns from the data. You can try different learning rates to find the optimal value that improves the model's accuracy.
+- Increase the number of epochs: Training the model for more epochs might help it converge to a better solution. You can try increasing the number of epochs and monitor the performance on a validation set to determine the optimal number of epochs.
+
+It's important to note that the accuracy of the model can be influenced by various factors, including the quality and size of the training data, the complexity of the task, and the model architecture
+
+ itself. Additionally, the lack of a neutral label in the dataset might affect the model's ability to accurately classify neutral sentiment.
+
+We encourage open-source contributors to explore ways to improve the accuracy of this model and provide additional datasets that are similar in nature to [Shushant/NepaliSentiment](https://huggingface.co/datasets/Shushant/NepaliSentiment). By collaborating and sharing insights, we can collectively enhance the performance of Nepali sentiment analysis models and contribute to the field of natural language processing.
+
+## License
+
+This project is licensed under the [MIT License](LICENSE).

inference.ipynb

@@ -0,0 +1,31 @@
+{
+ "cells": [
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Inference Notebook\n",
+    "\n",
+    "This Jupyter Notebook demonstrates how to perform inference using the trained Nepali Sentiment Analysis model.\n",
+    "\n",
+    "1. Load the trained model.\n",
+    "2. Preprocess the input text.\n",
+    "3. Tokenize the preprocessed text.\n",
+    "4. Perform inference using the tokenized input.\n",
+    "5. Interpret the output predictions.\n",
+    "6. Display the predicted sentiment labels.\n",
+    "\n",
+    "Please refer to the code cells in the notebook for detailed implementation and instructions."
+   ]
+  }
+ ],
+ "metadata": {
+  "language_info": {
+   "name": "python"
+  },
+  "orig_nbformat": 4
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}

model-training.ipynb

@@ -0,0 +1 @@
+{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.10.10","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"code","source":"from transformers import TFAutoModel , AutoTokenizer\n\nimport matplotlib.pyplot as plt\nimport os \nimport pandas as pd\nimport torch\n\nfrom sklearn.preprocessing import LabelEncoder\nimport numpy as np\nfrom torch.utils.data import TensorDataset\n\nfrom transformers import BertForSequenceClassification, AdamW\nfrom torch.utils.data import DataLoader, RandomSampler","metadata":{"execution":{"iopub.status.busy":"2023-06-23T02:45:23.458569Z","iopub.execute_input":"2023-06-23T02:45:23.458927Z","iopub.status.idle":"2023-06-23T02:45:46.629141Z","shell.execute_reply.started":"2023-06-23T02:45:23.458899Z","shell.execute_reply":"2023-06-23T02:45:46.628208Z"},"trusted":true},"execution_count":1,"outputs":[{"name":"stderr","text":"/opt/conda/lib/python3.10/site-packages/tensorflow_io/python/ops/__init__.py:98: UserWarning: unable to load libtensorflow_io_plugins.so: unable to open file: libtensorflow_io_plugins.so, from paths: ['/opt/conda/lib/python3.10/site-packages/tensorflow_io/python/ops/libtensorflow_io_plugins.so']\ncaused by: ['/opt/conda/lib/python3.10/site-packages/tensorflow_io/python/ops/libtensorflow_io_plugins.so: undefined symbol: _ZN3tsl6StatusC1EN10tensorflow5error4CodeESt17basic_string_viewIcSt11char_traitsIcEENS_14SourceLocationE']\n  warnings.warn(f\"unable to load libtensorflow_io_plugins.so: {e}\")\n/opt/conda/lib/python3.10/site-packages/tensorflow_io/python/ops/__init__.py:104: UserWarning: file system plugins are not loaded: unable to open file: libtensorflow_io.so, from paths: ['/opt/conda/lib/python3.10/site-packages/tensorflow_io/python/ops/libtensorflow_io.so']\ncaused by: ['/opt/conda/lib/python3.10/site-packages/tensorflow_io/python/ops/libtensorflow_io.so: undefined symbol: _ZTVN10tensorflow13GcsFileSystemE']\n  warnings.warn(f\"file system plugins are not loaded: {e}\")\n","output_type":"stream"}]},{"cell_type":"code","source":"# Print the list of files in the input directory\nprint(os.listdir('/kaggle/input/'))\n\n# Load the train and test datasets\ndf_train = pd.read_csv('/kaggle/input/sentiment-analysis/train.csv')\ndf_test = pd.read_csv('/kaggle/input/sentiment-analysis/test.csv')","metadata":{"execution":{"iopub.status.busy":"2023-06-23T02:45:46.631154Z","iopub.execute_input":"2023-06-23T02:45:46.631488Z","iopub.status.idle":"2023-06-23T02:45:46.725453Z","shell.execute_reply.started":"2023-06-23T02:45:46.631457Z","shell.execute_reply":"2023-06-23T02:45:46.724483Z"},"trusted":true},"execution_count":2,"outputs":[{"name":"stdout","text":"['sentiment-analysis']\n","output_type":"stream"}]},{"cell_type":"code","source":"# Bar graph for train dataset\ntrain_labels = df_train['label'].value_counts()\n\n# Bar graph for test dataset\ntest_labels = df_test['label'].value_counts()\n\n# Create subplots\nfig, axs = plt.subplots(1, 2, figsize=(12, 6))\n\n# Train dataset subplot\naxs[0].bar(train_labels.index, train_labels.values)\naxs[0].set_xlabel('Labels')\naxs[0].set_ylabel('Count')\naxs[0].set_title('Train Dataset - Label Distribution')\n\n# Test dataset subplot\naxs[1].bar(test_labels.index, test_labels.values)\naxs[1].set_xlabel('Labels')\naxs[1].set_ylabel('Count')\naxs[1].set_title('Test Dataset - Label Distribution')\n\n# Adjust spacing between subplots\nplt.subplots_adjust(wspace=0.4)\n\n# Display the plot\nplt.show()\n","metadata":{"execution":{"iopub.status.busy":"2023-06-23T02:45:46.726756Z","iopub.execute_input":"2023-06-23T02:45:46.727166Z","iopub.status.idle":"2023-06-23T02:45:47.158235Z","shell.execute_reply.started":"2023-06-23T02:45:46.727134Z","shell.execute_reply":"2023-06-23T02:45:47.157376Z"},"trusted":true},"execution_count":3,"outputs":[{"output_type":"display_data","data":{"text/plain":"<Figure size 1200x600 with 2 Axes>","image/png":"iVBORw0KGgoAAAANSUhEUgAAA/YAAAIhCAYAAADkVCF3AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAABV9ElEQVR4nO3dfVxUZf7/8fco9wgIIndKgGXehFmpmViLpuIdWplr5eZaa2WZumSum+uW6LZQVuaGaTdraqFptWqZrYa35aqllKbm161db1PETQRvCASv3x/F+TkCCggOR17Px2MeD+aca875nDMzzPWec841DmOMEQAAAAAAsKV6ri4AAAAAAABUHcEeAAAAAAAbI9gDAAAAAGBjBHsAAAAAAGyMYA8AAAAAgI0R7AEAAAAAsDGCPQAAAAAANkawBwAAAADAxgj2AAAAAADYGMEeFofDUaHb2rVrL2k9ycnJcjgc1VP0L86tr379+goMDFTbtm01fPhwbdq06ZKWnZKSoiVLllRPoZfo9OnTSk5OvuTnoDxdunRRbGxstSxrzpw5cjgc2rJlS7Us79xl7t27t0LtSm5eXl4KCwtT165dlZqaquzs7FKPqcrrsqrPR1nrio6OVmJiYqWWczHz58/XtGnTypzncDiUnJxcresDgEtxufohUuX/f+/du9epBnd3dzVq1EgdOnTQE088oZ07d162WmraoUOHlJycrK1bt9bI8qvz867k8/R///tftSzv3GVWtF3JzcfHR02bNlXPnj2VlpamEydOlHrMAw88oOjo6ErVU9Xno6x1ORwOjRw5slLLuZgZM2Zozpw5paaXvGfKmoea4ebqAlB7bNy40en+X/7yF61Zs0arV692mt66detLWs9DDz2kXr16XdIyyjJw4EA9+eSTMsYoLy9PO3bs0Ntvv6033nhDo0eP1t/+9rcqLTclJUUDBw7UnXfeWb0FV8Hp06c1adIkST+HcFzY7Nmz1bJlS505c0bZ2dlav369nn/+eb344otauHChunfvbrWtyuuyqs9HTb0Hzjd//nzt2LFDSUlJpeZt3LhRTZs2rfEaAKCiLlc/RKr6/+9Ro0Zp8ODBOnv2rI4fP66vv/5ab731ltLS0pSamqo//OEPl62WmnLo0CFNmjRJ0dHRuuGGG1xdTq23fPlyBQQEqLCwUIcOHdKqVas0btw4vfDCC1q6dKnatm1rtX366af1+9//vlLLr+rzUZV1VcWMGTMUHBysBx54wGl6eHi4Nm7cqKuvvrrGa8DPCPaw3HLLLU73GzdurHr16pWafr7Tp0/Lx8enwutp2rRpjQSK0NBQp1p79uyppKQkPfLII3rllVfUsmVLPfbYY9W+XtResbGxat++vXX/7rvv1hNPPKFbb71VAwYM0HfffafQ0FBJNfe6PFfJe+VyrOtiLva+BoDLrar9kMvpqquucqqnT58+GjNmjAYMGKBx48YpNjZWvXv3dmGFuNzatWun4OBg6/69996rkSNHKj4+Xv3799e///1veXp6StJlCbklfQ1XB2pPT89a9d6tCzgVH5VScqr2Z599pri4OPn4+Oh3v/udJGnhwoVKSEhQeHi4vL291apVKz311FM6deqU0zIudBry8uXLddNNN8nb21stW7bUW2+9dUn11q9fX9OnT1dwcLBeeOEFa/pPP/2kJ598UjfccIMCAgIUFBSkTp066cMPP3R6vMPh0KlTpzR37lzrVKuSb9OPHj2qESNGqHXr1mrQoIFCQkJ0++236/PPPy9Vx8yZM9W2bVs1aNBAfn5+atmypf70pz85tcnKytLw4cPVtGlTeXh4KCYmRpMmTVJRUZGkn09paty4sSRp0qRJVj3nf0Na07Zs2aJ7771X0dHR8vb2VnR0tO677z7t27evzPY5OTl68MEHFRQUJF9fX/Xr10///e9/S7VbuXKlunXrJn9/f/n4+Khz585atWpVtdd/1VVX6aWXXtKJEyf0+uuvW9PLel2uXr1aXbp0UaNGjeTt7a2rrrpKd999t06fPn3R56NkeV999ZUGDhyowMBA60P2Qqf5LV68WNdff728vLzUrFkzvfLKK07zy7scYe3atU6nqHbp0kXLli3Tvn37nE4VLFHWqfg7duzQHXfcocDAQHl5eemGG27Q3Llzy1zPu+++qwkTJigiIkL+/v7q3r27du/eXf6OB4BqUFhYqGeffVYtW7aUp6enGjdurAcffFBHjx51ancp/78ry9vbW7NmzZK7u7tTX6Mi/YSL1fL999/rwQcfVPPmzeXj46MmTZqoX79+2r59u1MNZ8+e1bPPPqsWLVrI29tbDRs21PXXX1/qbMXvvvtOgwcPVkhIiDw9PdWqVSu9+uqr1vy1a9eqQ4cOkqQHH3zQqudyX7qVkZGhO+64Q02bNpWXl5euueYaDR8+vNxT7g8cOKABAwbI399fAQEBuv/++0u9JqSf+6qdOnWSr6+vGjRooJ49e+rrr7+u9vrbtm2rCRMmaP/+/Vq4cKE1vazT499//3117NhRAQEB8vHxUbNmzay+9cWejwceeEANGjTQ9u3blZCQID8/P3Xr1q3cdZV4/fXXde2118rT01OtW7fWggULnOaX1085vw8SHR2tnTt3at26dVZtJess71T89evXq1u3bvLz85OPj4/i4uK0bNmyMtezZs0aPfbYYwoODlajRo00YMAAHTp0qMxtAsEeVXD48GHdf//9Gjx4sD755BONGDFC0s8fFn369NGsWbO0fPlyJSUl6b333lO/fv0qtNxt27bpySef1BNPPKEPP/xQ119/vYYNG6bPPvvskur19vZW9+7dtWfPHh08eFCSVFBQoGPHjmns2LFasmSJ3n33Xeso7ttvv209duPGjfL29lafPn20ceNGbdy4UTNmzJAkHTt2TJI0ceJELVu2TLNnz1azZs3UpUsXp+vkFixYoBEjRig+Pl6LFy/WkiVL9MQTTzh94ZGVlaWbb75ZK1as0DPPPKN//vOfGjZsmFJTU/Xwww9L+vmUpuXLl0uShg0bZtXz9NNPX9L+qay9e/eqRYsWmjZtmlasWKHnn39ehw8fVocOHcr8wB02bJjq1atnXe/95ZdfqkuXLjp+/LjVJj09XQkJCfL399fcuXP13nvvKSgoSD179qyRcN+nTx/Vr1//gq+tvXv3qm/fvvLw8NBbb72l5cuX67nnnpOvr68KCwsr/HwMGDBA11xzjd5//3299tprF6xr69atSkpK0hNPPKHFixcrLi5Ov//97/Xiiy9WehtnzJihzp07KywszKrt/NNcz7V7927FxcVp586deuWVV7Ro0SK1bt1aDzzwgKZMmVKq/Z/+9Cft27dPf//73/XGG2/ou+++U79+/VRcXFzpWgGgIs6ePas77rhDzz33nAYPHqxly5bpueeeU0ZGhrp06aL8/HxJ1ff/uzIiIiLUrl07bdiwwfpCviL9hIvVcujQITVq1EjPPfecli9frldffVVubm7q2LGj05epU6ZMUXJysu677z4tW7ZMCxcu1LBhw5w+a7/99lt16NBBO3bs0EsvvaSPP/5Yffv21ejRo61LAW666SbNnj1bkvTnP//Zquehhx6q8r6piv/85z/q1KmTZs6cqU8//VTPPPOMvvjiC9166606c+ZMqfZ33XWXrrnmGn3wwQdKTk7WkiVL1LNnT6e2KSkpuu+++9S6dWu99957euedd3TixAnddttt+vbbb6t9G/r37y9JF+xrbNy4Uffcc4+aNWumBQsWaNmyZXrmmWes11BFno/CwkL1799ft99+uz788EPruSzPRx99pFdeeUWTJ0/WBx98oKioKN1333364IMPKr2NixcvVrNmzXTjjTdatS1evLjc9uvWrdPtt9+u3NxczZo1S++++678/PzUr18/py9ASjz00ENyd3fX/PnzNWXKFK1du1b3339/peusMwxQjqFDhxpfX1+nafHx8UaSWbVq1QUfe/bsWXPmzBmzbt06I8ls27bNmjdx4kRz/ksvKirKeHl5mX379lnT8vPzTVBQkBk+fPhFa5VkHn/88XLn//GPfzSSzBdffFHm/KKiInPmzBkzbNgwc+ONNzrN8/X1NUOHDr1oDSXL6Natm7nrrrus6SNHjjQNGza84GOHDx9uGjRo4LT9xhjz4osvGklm586dxhhjjh49aiSZiRMnXrSeqoiPjzfXXXddpR5TVFRkTp48aXx9fc3f/vY3a/rs2bONJKd9YYwx//rXv4wk8+yzzxpjjDl16pQJCgoy/fr1c2pXXFxs2rZta26++eZSy9yzZ88Fayppt3nz5nLbhIaGmlatWln3z39dfvDBB0aS2bp1a7nLuNDzUbK8Z555ptx554qKijIOh6PU+nr06GH8/f3NqVOnnLbt/H2wZs0aI8msWbPGmta3b18TFRVVZu3n133vvfcaT09Ps3//fqd2vXv3Nj4+Pub48eNO6+nTp49Tu/fee89IMhs3bixzfQBQWef3Q959910jyfzjH/9ward582YjycyYMcMYc+n/v8uyZ88eI8m88MIL5ba55557jCRz5MiRMueX10+oTC1FRUWmsLDQNG/e3DzxxBPW9MTERHPDDTdc8LE9e/Y0TZs2Nbm5uU7TR44caby8vMyxY8eMMf9/f86ePfui9VRFVFSU6du3b4Xbl/Qp9+3bZySZDz/80JpX8nl67r4wxph58+YZSSY9Pd0YY8z+/fuNm5ubGTVqlFO7EydOmLCwMDNo0KBSy7yYknZHjx4tc35+fr6RZHr37m1NGzp0qNPnckk/r+QztiwXej6GDh1qJJm33nqrzHnn9wEkGW9vb5OVlWVNKyoqMi1btjTXXHNNqW07X1l9kOuuu87Ex8eXalvynjm37ltuucWEhISYEydOOK0/NjbWNG3a1Jw9e9ZpPSNGjHBa5pQpU4wkc/jw4VLrgzEcsUelBQYG6vbbby81/b///a8GDx6ssLAw1a9fX+7u7oqPj5ck7dq166LLveGGG3TVVVdZ9728vHTttdeWe4p3ZRhjSk17//331blzZzVo0EBubm5yd3fXrFmzKlRriddee0033XSTvLy8rGWsWrXKaRk333yzjh8/rvvuu08ffvhhmUe1P/74Y3Xt2lUREREqKiqybiXX6a1bt64KWy0VFxc7Le/s2bNVWs65Tp48qT/+8Y+65ppr5ObmJjc3NzVo0ECnTp0qc9/95je/cbofFxenqKgorVmzRpK0YcMGHTt2TEOHDi1Va69evbR58+ZSl3NUh7JeE+e64YYb5OHhoUceeURz584t8/KBirj77rsr3Pa6665zGmRHkgYPHqy8vDx99dVXVVp/Ra1evVrdunVTZGSk0/QHHnhAp0+fLnW0v+RIRInrr79ekqrl/QoAZfn444/VsGFD9evXz+nz4oYbblBYWJh1FLy6/n9XVlmfKxXpJ1xIUVGRUlJS1Lp1a3l4eMjNzU0eHh767rvvSvU1tm3bphEjRmjFihXKy8tzWs5PP/2kVatW6a677pKPj4/T/uvTp49++umnKv+K0LnLKioquujna0VkZ2fr0UcfVWRkpLXfoqKiJJXdpzy/rzFo0CC5ublZfY0VK1aoqKhIv/3tb51q9fLyUnx8fI38IkFF9kPJafaDBg3Se++9px9++KFK66pMX6Nbt27W+ELSz5et3nPPPfr++++tM1trwqlTp/TFF19o4MCBatCggdP6hwwZooMHD5a6pI++RuUQ7FFp4eHhpaadPHlSt912m7744gs9++yzWrt2rTZv3qxFixZJknV63IU0atSo1DRPT88KPfZiSv4BRERESJIWLVqkQYMGqUmTJkpPT9fGjRu1efNm/e53v9NPP/1UoWVOnTpVjz32mDp27Kh//OMf2rRpkzZv3qxevXo51TxkyBC99dZb2rdvn+6++26FhISoY8eOysjIsNocOXJES5culbu7u9Ptuuuuk6Qq/4zL1Vdf7bS8yZMnV2k55xo8eLCmT5+uhx56SCtWrNCXX36pzZs3q3HjxmU+V2FhYWVO+/HHHyX9vO3Sz79qcP72P//88zLGWKczVpdTp07pxx9/tF4PZbn66qu1cuVKhYSE6PHHH9fVV1+tq6++utK/rlDW+6U85e0rSdb+qik//vhjmbWW7KPz13/++7VkYKDqeL8CQFmOHDmi48ePy8PDo9TnRVZWlvVZWV3/vytr37598vT0VFBQkKSK9xMuZMyYMXr66ad15513aunSpfriiy+0efNmtW3b1mkZ48eP14svvqhNmzapd+/eatSokbp162b95OyPP/6ooqIipaWlldp3ffr0kVT1vsb5yzt/bJbKOnv2rBISErRo0SKNGzdOq1at0pdffml98VCRvoabm5saNWpUqq/RoUOHUvUuXLiwWn8ur8T5fc+y/OpXv9KSJUusLx2aNm2q2NhYvfvuuxVej4+Pj/z9/Svc3lV9jZycHBlj6GvUIEbFR6WVNZjG6tWrdejQIa1du9Y6Si/J6douV8nPz9fKlSt19dVXWyORp6enKyYmRgsXLnTanoKCggovNz09XV26dNHMmTOdppf1u6UPPvigHnzwQZ06dUqfffaZJk6cqMTERP373/9WVFSUgoODdf311+uvf/1rmeu60IfChSxdutRpm6q6nBK5ubn6+OOPNXHiRD311FPW9JIxC8qSlZVV5rRrrrlGkqyRZNPS0sodPfXcb5arw7Jly1RcXHzRnxW67bbbdNttt6m4uFhbtmxRWlqakpKSFBoaqnvvvbdC66rIb+GWKG9fSf//w83Ly0tS6dfqpXZKGjVqpMOHD5eaXjJIzbkj/gKAK5QMoFVyTfr5/Pz8rL+r4/93Zfzwww/KzMxUfHy83Nx+7l5Xpp9QnvT0dP32t79VSkqK0/T//e9/atiwoXXfzc1NY8aM0ZgxY3T8+HGtXLlSf/rTn9SzZ08dOHBAgYGB1pHRxx9/vMx1xcTEVLiuc23evLlallNix44d2rZtm+bMmaOhQ4da07///vtyH5OVlaUmTZpY94uKivTjjz9an50ln2El15RfDh999JGki/+E4R133KE77rhDBQUF2rRpk1JTUzV48GBFR0erU6dOF11PZfoZUuX7GiVhWrq0vkZgYKDq1atHX6MGEexRLUr+qZz75pfkNOq4KxQXF2vkyJH68ccflZqaak13OBzy8PBw+meYlZVValR8qfyzBhwOR6nt/eabb7Rx48ZSpzOX8PX1Ve/evVVYWKg777xTO3fuVFRUlBITE/XJJ5/o6quvVmBgYLnbU9lvKtu0aVOhdhXlcDhkjCm13X//+9/LHTRt3rx5TqeIbdiwQfv27bMGfuncubMaNmyob7/9ViNHjqzWesuyf/9+jR07VgEBARo+fHiFHlO/fn117NhRLVu21Lx58/TVV1/p3nvvrfZvjnfu3Klt27Y5nY4/f/58+fn56aabbpIka7TZb775Ri1atLDalXQgzlWZM166deumxYsX69ChQ05fAL399tvy8fHhJ2sAuFxiYqIWLFig4uJidezYsUKPuRz/v/Pz8/XQQw+pqKhI48aNs6ZXtJ9woVrKWsayZcv0ww8/WF+Qn69hw4YaOHCgfvjhByUlJWnv3r1q3bq1unbtqq+//lrXX3+9PDw8yt2eyu6bc39WtjpUpU85b948tWvXzrr/3nvvqaioyArVPXv2lJubm/7zn/9U6rT1qtq2bZtSUlIUHR2tQYMGVegxnp6eio+PV8OGDbVixQp9/fXX6tSpU7X3NVatWqUjR45YB02Ki4u1cOFCpwNg5/Y1Si4XkH4+YFRW3RWpzdfXVx07dtSiRYv04osvytvbW9LPZ2ikp6eradOmuvbaay918+o0gj2qRVxcnAIDA/Xoo49q4sSJcnd317x587Rt27bLVsORI0e0adMmGWN04sQJ7dixQ2+//ba2bdumJ554whpdXvq5c7Bo0SKNGDFCAwcO1IEDB/SXv/xF4eHh+u6775yW26ZNG61du1ZLly5VeHi4/Pz81KJFCyUmJuovf/mLJk6cqPj4eO3evVuTJ09WTEyMNZqpJD388MPy9vZW586dFR4erqysLKWmpiogIMD6Zzl58mRlZGQoLi5Oo0ePVosWLfTTTz9p7969+uSTT/Taa6+padOm8vPzU1RUlD788EN169ZNQUFBCg4OLvfnTKoiLy+vzJFRGzdurPj4eP3qV7/SCy+8YK133bp1mjVrltORg3Nt2bJFDz30kH7961/rwIEDmjBhgpo0aWL9mkKDBg2UlpamoUOH6tixYxo4cKBCQkJ09OhRbdu2TUePHi11tKOiduzYYV1Hl52drc8//1yzZ89W/fr1tXjxYusnhsry2muvafXq1erbt6+uuuoq/fTTT9bPL3bv3l2Sqv35iIiIUP/+/ZWcnKzw8HClp6crIyNDzz//vHx8fCT9fBphixYtNHbsWBUVFSkwMFCLFy/W+vXrSy2vTZs2WrRokWbOnKl27dqpXr165XbAJk6caI318MwzzygoKEjz5s3TsmXLNGXKFAUEBFRpmwCgutx7772aN2+e+vTpo9///ve6+eab5e7uroMHD2rNmjW64447dNddd9Xo/+/9+/dr06ZNOnv2rHJzc/X1119bl9u99NJLSkhIsNpWtJ9woVoSExM1Z84ctWzZUtdff70yMzP1wgsvWAGsRL9+/RQbG6v27durcePG2rdvn6ZNm6aoqCg1b95ckvS3v/1Nt956q2677TY99thjio6O1okTJ/T9999r6dKlWr16taSfL2Xw9vbWvHnz1KpVKzVo0EARERGXfNbfubKyssrsa0RHR6tt27a6+uqr9dRTT8kYo6CgIC1dutTpEsbzLVq0SG5uburRo4d27typp59+Wm3btrVCdXR0tCZPnqwJEybov//9r3r16qXAwEAdOXJEX375pXx9fS86mnx5MjMzFRAQoDNnzujQoUNatWqV3nnnHYWEhGjp0qUX/BLlmWee0cGDB9WtWzc1bdpUx48f19/+9jencaqq+/kIDg7W7bffrqefflq+vr6aMWOG/u///s/pJ+/69OmjoKAgDRs2TJMnT5abm5vmzJmjAwcOlFpemzZttGDBAi1cuFDNmjWTl5dXuQeWUlNT1aNHD3Xt2lVjx46Vh4eHZsyYoR07dujdd9+t9NkHOI+rRu1D7VfeqPjljZq+YcMG06lTJ+Pj42MaN25sHnroIfPVV1+VGhGzvBHByxohNT4+vsyRNs8nybrVq1fP+Pv7mzZt2phHHnmk3FG6n3vuORMdHW08PT1Nq1atzJtvvllmbVu3bjWdO3c2Pj4+RpJVT0FBgRk7dqxp0qSJ8fLyMjfddJNZsmRJqVFI586da7p27WpCQ0ONh4eHiYiIMIMGDTLffPON03qOHj1qRo8ebWJiYoy7u7sJCgoy7dq1MxMmTDAnT5602q1cudLceOONxtPT00iq0Ij9FVXyqwdl3Uq2++DBg+buu+82gYGBxs/Pz/Tq1cvs2LHDREVFOdVSMqLpp59+aoYMGWIaNmxovL29TZ8+fcx3331Xat3r1q0zffv2NUFBQcbd3d00adLE9O3b17z//vulllnRUfFLbh4eHiYkJMTEx8eblJQUk52dXeox5z/3GzduNHfddZeJiooynp6eplGjRiY+Pt589NFHTo8r7/m40Gi5F3oPfPDBB+a6664zHh4eJjo62kydOrXU4//973+bhIQE4+/vbxo3bmxGjRplli1bVmpU/GPHjpmBAweahg0bGofD4bROlTEC8/bt202/fv1MQECA8fDwMG3bti01Cm/JqPjnPi/GlD36LQBcirL6IWfOnDEvvviiadu2rfHy8jINGjQwLVu2NMOHD7c+Wy71/3dZSv7Hldzq169vAgMDTbt27UxSUpL16zXnqmg/4UK15OTkmGHDhpmQkBDj4+Njbr31VvP555+X6h+99NJLJi4uzgQHBxsPDw9z1VVXmWHDhpm9e/eW2o7f/e53pkmTJsbd3d00btzYxMXFWb9UU+Ldd981LVu2NO7u7tX+azxRUVHl9jVKtvvbb781PXr0MH5+fiYwMND8+te/Nvv37y9VS8nnaWZmpunXr59p0KCB8fPzM/fdd1+Zv06wZMkS07VrV+Pv7288PT1NVFSUGThwoFm5cmWpZV5MSbuSm6enpwkPDzcJCQnmb3/7m8nLyyv1mPOf+48//tj07t3bNGnSxOqr9OnTx3z++edOjyvv+SjrPVLeuoz5/78iNWPGDHP11Vcbd3d307JlSzNv3rxSj//yyy9NXFyc8fX1NU2aNDETJ040f//730v1w/bu3WsSEhKMn5+fkWSts7x+weeff25uv/124+vra7y9vc0tt9xili5d6tSmvF83KusXgPD/OYyphqErAQAAAACASzAqPgAAAAAANkawBwAAAADAxgj2AAAAAADYGMEeAAAAAAAbI9gDAAAAAGBjBHsAAAAAAGzMzdUF2MXZs2d16NAh+fn5yeFwuLocAIALGGN04sQJRUREqF49vhtHzaHfAQCQKt73INhX0KFDhxQZGenqMgAAtcCBAwfUtGlTV5eBKxj9DgDAuS7W9yDYV5Cfn5+kn3eov7+/i6sBALhCXl6eIiMjrc8EoKbQ7wAASBXvexDsK6jkNDh/f38+YAGgjuPUaNQ0+h0AgHNdrO/BBYIAAAAAANgYwR4AAAAAABsj2AMAAAAAYGMEewAAAAAAbIxgDwAAAACAjRHsAQAAAACwMYI9AAAAAAA2RrAHAAAAAMDGCPYAAAAAANgYwR4AAAAAABsj2AMAAAAAYGMEewAAAAAAbIxgDwAAAACAjRHsAQAAAACwMYI9AAAAAAA2RrAHAAAAAMDGCPYAAAAAANgYwR4AAAAAABsj2AMAAAAAYGNuri6grol+apmrSyhl73N9L9qmttVdkZoBAEDt+wyvSfQPANRVHLEHAAAAAMDGCPYAAAAAANgYwR4AAAAAABsj2AMAAAAAYGMEewAAAAAAbIxgDwAAAACAjRHsAQAAAACwMYI9AAAAAAA2RrAHAAAAAMDGCPYAAAAAANgYwR4AAAAAABsj2AMAAAAAYGMEewAAAAAAbIxgDwAAAACAjRHsAQAAAACwMYI9AAAAAAA2RrAHAAAAAMDGCPYAAAAAANgYwR4AAAAAABsj2AMAAAAAYGMEewAAAAAAbIxgDwAAAACAjRHsAQAAAACwMYI9AAAAAAA2RrAHAAAAAMDGCPYAAAAAANgYwR4AAAAAABsj2AMAAAAAYGMEewAAAAAAbIxgDwAAAACAjRHsAQAAAACwMYI9AAAAAAA2RrAHAAAAAMDGCPYAAAAAANgYwR4AAAAAABsj2AMAAAAAYGMEewAAAAAAbIxgDwAAAACAjRHsAQAAflFUVKQ///nPiomJkbe3t5o1a6bJkyfr7NmzVhtjjJKTkxURESFvb2916dJFO3fudFpOQUGBRo0apeDgYPn6+qp///46ePDg5d4cAEAdQbAHAAD4xfPPP6/XXntN06dP165duzRlyhS98MILSktLs9pMmTJFU6dO1fTp07V582aFhYWpR48eOnHihNUmKSlJixcv1oIFC7R+/XqdPHlSiYmJKi4udsVmAQCucG6uLgAAAKC22Lhxo+644w717dtXkhQdHa13331XW7ZskfTz0fpp06ZpwoQJGjBggCRp7ty5Cg0N1fz58zV8+HDl5uZq1qxZeuedd9S9e3dJUnp6uiIjI7Vy5Ur17NnTNRsHALhiccQeAADgF7feeqtWrVqlf//735Kkbdu2af369erTp48kac+ePcrKylJCQoL1GE9PT8XHx2vDhg2SpMzMTJ05c8apTUREhGJjY6025ysoKFBeXp7TDQCAiuKIPQAAwC/++Mc/Kjc3Vy1btlT9+vVVXFysv/71r7rvvvskSVlZWZKk0NBQp8eFhoZq3759VhsPDw8FBgaWalPy+POlpqZq0qRJ1b05AIA6giP2AAAAv1i4cKHS09M1f/58ffXVV5o7d65efPFFzZ0716mdw+Fwum+MKTXtfBdqM378eOXm5lq3AwcOXNqGAADqFI7YAwAA/OIPf/iDnnrqKd17772SpDZt2mjfvn1KTU3V0KFDFRYWJunno/Lh4eHW47Kzs62j+GFhYSosLFROTo7TUfvs7GzFxcWVuV5PT095enrW1GYBAK5wHLEHAAD4xenTp1WvnnP3qH79+tbP3cXExCgsLEwZGRnW/MLCQq1bt84K7e3atZO7u7tTm8OHD2vHjh3lBnsAAC4FR+wBAAB+0a9fP/31r3/VVVddpeuuu05ff/21pk6dqt/97neSfj4FPykpSSkpKWrevLmaN2+ulJQU+fj4aPDgwZKkgIAADRs2TE8++aQaNWqkoKAgjR07Vm3atLFGyQcAoDoR7AEAAH6Rlpamp59+WiNGjFB2drYiIiI0fPhwPfPMM1abcePGKT8/XyNGjFBOTo46duyoTz/9VH5+flabl19+WW5ubho0aJDy8/PVrVs3zZkzR/Xr13fFZgEArnAOY4xxdRF2kJeXp4CAAOXm5srf37/Ky4l+alk1VlU99j7X96JtalvdFakZAKpbdX0WABdTna+12vYZXpPoHwC40lT084Br7AEAAAAAsDGCPQAAAAAANkawBwAAAADAxgj2AAAAAADYGMEeAAAAAAAbI9gDAAAAAGBjBHsAAAAAAGyMYA8AAAAAgI0R7AEAAAAAsDGCPQAAAAAANkawBwAAAADAxgj2AAAAAADYGMEeAAAAAAAbI9gDAAAAAGBjBHsAAAAAAGyMYA8AAAAAgI25NNinpqaqQ4cO8vPzU0hIiO68807t3r3bqY0xRsnJyYqIiJC3t7e6dOminTt3OrUpKCjQqFGjFBwcLF9fX/Xv318HDx50apOTk6MhQ4YoICBAAQEBGjJkiI4fP17TmwgAAAAAQI1yabBft26dHn/8cW3atEkZGRkqKipSQkKCTp06ZbWZMmWKpk6dqunTp2vz5s0KCwtTjx49dOLECatNUlKSFi9erAULFmj9+vU6efKkEhMTVVxcbLUZPHiwtm7dquXLl2v58uXaunWrhgwZclm3FwAAAACA6ubmypUvX77c6f7s2bMVEhKizMxM/epXv5IxRtOmTdOECRM0YMAASdLcuXMVGhqq+fPna/jw4crNzdWsWbP0zjvvqHv37pKk9PR0RUZGauXKlerZs6d27dql5cuXa9OmTerYsaMk6c0331SnTp20e/dutWjR4vJuOAAAAAAA1aRWXWOfm5srSQoKCpIk7dmzR1lZWUpISLDaeHp6Kj4+Xhs2bJAkZWZm6syZM05tIiIiFBsba7XZuHGjAgICrFAvSbfccosCAgKsNucrKChQXl6e0w0AAAAAgNqm1gR7Y4zGjBmjW2+9VbGxsZKkrKwsSVJoaKhT29DQUGteVlaWPDw8FBgYeME2ISEhpdYZEhJitTlfamqqdT1+QECAIiMjL20DAQAAAACoAbUm2I8cOVLffPON3n333VLzHA6H031jTKlp5zu/TVntL7Sc8ePHKzc317odOHCgIpsBAAAAAMBlVSuC/ahRo/TRRx9pzZo1atq0qTU9LCxMkkodVc/OzraO4oeFhamwsFA5OTkXbHPkyJFS6z169GipswFKeHp6yt/f3+kGAAAAAEBt49Jgb4zRyJEjtWjRIq1evVoxMTFO82NiYhQWFqaMjAxrWmFhodatW6e4uDhJUrt27eTu7u7U5vDhw9qxY4fVplOnTsrNzdWXX35ptfniiy+Um5trtQEAAAAAwI5cOir+448/rvnz5+vDDz+Un5+fdWQ+ICBA3t7ecjgcSkpKUkpKipo3b67mzZsrJSVFPj4+Gjx4sNV22LBhevLJJ9WoUSMFBQVp7NixatOmjTVKfqtWrdSrVy89/PDDev311yVJjzzyiBITExkRHwAAAABgay4N9jNnzpQkdenSxWn67Nmz9cADD0iSxo0bp/z8fI0YMUI5OTnq2LGjPv30U/n5+VntX375Zbm5uWnQoEHKz89Xt27dNGfOHNWvX99qM2/ePI0ePdoaPb9///6aPn16zW4gAAAAAAA1zKXB3hhz0TYOh0PJyclKTk4ut42Xl5fS0tKUlpZWbpugoCClp6dXpUwAAAAAAGqtWjF4HgAAAAAAqBqCPQAAAAAANkawBwAAAADAxgj2AAAAAADYGMEeAAAAAAAbI9gDAAAAAGBjBHsAAAAAAGyMYA8AAAAAgI0R7AEAAAAAsDGCPQAAAAAANkawBwAAAADAxgj2AAAAAADYGMEeAAAAAAAbI9gDAAAAAGBjBHsAAAAAAGyMYA8AAAAAgI0R7AEAAAAAsDGCPQAAAAAANkawBwAAAADAxgj2AAAAAADYGMEeAAAAAAAbI9gDAAAAAGBjBHsAAAAAAGyMYA8AAAAAgI0R7AEAAAAAsDGCPQAAAAAANkawBwAAAADAxgj2AAAAAADYGMEeAAAAAAAbI9gDAAAAAGBjBHsAAAAAAGyMYA8AAAAAgI0R7AEAAAAAsDGCPQAAAAAANkawBwAAAADAxgj2AAAAAADYGMEeAAAAAAAbI9gDAAAAAGBjBHsAAAAAAGyMYA8AAAAAgI0R7AEAAAAAsDGCPQAAAAAANkawBwAAAADAxgj2AAAAAADYGMEeAAAAAAAbI9gDAAAAAGBjBHsAAAAAAGyMYA8AAAAAgI0R7AEAAAAAsDGCPQAAAAAANkawBwAAAADAxgj2AAAAAADYGMEeAAAAAAAbI9gDAAAAAGBjBHsAAAAAAGyMYA8AAAAAgI0R7AEAAAAAsDGCPQAAAAAANkawBwAAAADAxgj2AAAAAADYmJurCwAAAABcKfqpZa4u4bLZ+1xfV5cAoAZwxB4AAAAAABsj2AMAAJzjhx9+0P33369GjRrJx8dHN9xwgzIzM635xhglJycrIiJC3t7e6tKli3bu3Om0jIKCAo0aNUrBwcHy9fVV//79dfDgwcu9KQCAOoJgDwAA8IucnBx17txZ7u7u+uc//6lvv/1WL730kho2bGi1mTJliqZOnarp06dr8+bNCgsLU48ePXTixAmrTVJSkhYvXqwFCxZo/fr1OnnypBITE1VcXOyCrQIAXOm4xh4AAOAXzz//vCIjIzV79mxrWnR0tPW3MUbTpk3ThAkTNGDAAEnS3LlzFRoaqvnz52v48OHKzc3VrFmz9M4776h79+6SpPT0dEVGRmrlypXq2bNnqfUWFBSooKDAup+Xl1dDWwgAuBJxxB4AAOAXH330kdq3b69f//rXCgkJ0Y033qg333zTmr9nzx5lZWUpISHBmubp6an4+Hht2LBBkpSZmakzZ844tYmIiFBsbKzV5nypqakKCAiwbpGRkTW0hQCAKxHBHgAA4Bf//e9/NXPmTDVv3lwrVqzQo48+qtGjR+vtt9+WJGVlZUmSQkNDnR4XGhpqzcvKypKHh4cCAwPLbXO+8ePHKzc317odOHCgujcNAHAF41R8AACAX5w9e1bt27dXSkqKJOnGG2/Uzp07NXPmTP32t7+12jkcDqfHGWNKTTvfhdp4enrK09PzEqsHANRVHLEHAAD4RXh4uFq3bu00rVWrVtq/f78kKSwsTJJKHXnPzs62juKHhYWpsLBQOTk55bYBAKA6EewBAAB+0blzZ+3evdtp2r///W9FRUVJkmJiYhQWFqaMjAxrfmFhodatW6e4uDhJUrt27eTu7u7U5vDhw9qxY4fVBgCA6sSp+AAAAL944oknFBcXp5SUFA0aNEhffvml3njjDb3xxhuSfj4FPykpSSkpKWrevLmaN2+ulJQU+fj4aPDgwZKkgIAADRs2TE8++aQaNWqkoKAgjR07Vm3atLFGyQcAoDoR7AEAAH7RoUMHLV68WOPHj9fkyZMVExOjadOm6Te/+Y3VZty4ccrPz9eIESOUk5Ojjh076tNPP5Wfn5/V5uWXX5abm5sGDRqk/Px8devWTXPmzFH9+vVdsVkAgCscwR4AAOAciYmJSkxMLHe+w+FQcnKykpOTy23j5eWltLQ0paWl1UCFAAA44xp7AAAAAABsjGAPAAAAAICNEewBAAAAALAxgj0AAAAAADZGsAcAAAAAwMYI9gAAAAAA2BjBHgAAAAAAGyPYAwAAAABgYy4N9p999pn69euniIgIORwOLVmyxGn+Aw88IIfD4XS75ZZbnNoUFBRo1KhRCg4Olq+vr/r376+DBw86tcnJydGQIUMUEBCggIAADRkyRMePH6/hrQMAAAAAoOa5NNifOnVKbdu21fTp08tt06tXLx0+fNi6ffLJJ07zk5KStHjxYi1YsEDr16/XyZMnlZiYqOLiYqvN4MGDtXXrVi1fvlzLly/X1q1bNWTIkBrbLgAAAAAALhc3V668d+/e6t279wXbeHp6KiwsrMx5ubm5mjVrlt555x11795dkpSenq7IyEitXLlSPXv21K5du7R8+XJt2rRJHTt2lCS9+eab6tSpk3bv3q0WLVpU70YBAAAAAHAZ1fpr7NeuXauQkBBde+21evjhh5WdnW3Ny8zM1JkzZ5SQkGBNi4iIUGxsrDZs2CBJ2rhxowICAqxQL0m33HKLAgICrDZlKSgoUF5entMNAAAAAIDaplYH+969e2vevHlavXq1XnrpJW3evFm33367CgoKJElZWVny8PBQYGCg0+NCQ0OVlZVltQkJCSm17JCQEKtNWVJTU61r8gMCAhQZGVmNWwYAAAAAQPVw6an4F3PPPfdYf8fGxqp9+/aKiorSsmXLNGDAgHIfZ4yRw+Gw7p/7d3ltzjd+/HiNGTPGup+Xl0e4BwAAAADUOrX6iP35wsPDFRUVpe+++06SFBYWpsLCQuXk5Di1y87OVmhoqNXmyJEjpZZ19OhRq01ZPD095e/v73QDAAAAAKC2sVWw//HHH3XgwAGFh4dLktq1ayd3d3dlZGRYbQ4fPqwdO3YoLi5OktSpUyfl5ubqyy+/tNp88cUXys3NtdoAAAAAAGBXLj0V/+TJk/r++++t+3v27NHWrVsVFBSkoKAgJScn6+6771Z4eLj27t2rP/3pTwoODtZdd90lSQoICNCwYcP05JNPqlGjRgoKCtLYsWPVpk0ba5T8Vq1aqVevXnr44Yf1+uuvS5IeeeQRJSYmMiI+AAAAAMD2XBrst2zZoq5du1r3S65pHzp0qGbOnKnt27fr7bff1vHjxxUeHq6uXbtq4cKF8vPzsx7z8ssvy83NTYMGDVJ+fr66deumOXPmqH79+labefPmafTo0dbo+f3799f06dMv01YCAAAAAFBzXBrsu3TpImNMufNXrFhx0WV4eXkpLS1NaWlp5bYJCgpSenp6lWoEAAAAAKA2s9U19gAAAAAAwBnBHgAAAAAAGyPYAwAAAABgYwR7AAAAAABsjGAPAAAAAICNEewBAAAAALAxgj0AAAAAADZGsAcAAAAAwMYI9gAAAAAA2BjBHgAAAAAAGyPYAwAAAABgYwR7AAAAAABsjGAPAAAAAICNEewBAAAAALAxgj0AAAAAADZGsAcAAAAAwMYI9gAAAAAA2BjBHgAAAAAAGyPYAwAAAABgYwR7AAAAAABsjGAPAAAAAICNEewBAAAAALCxKgX7Zs2a6ccffyw1/fjx42rWrNklFwUAAFAZ9E0AAHVZlYL93r17VVxcXGp6QUGBfvjhh0suCgAAoDLomwAA6jK3yjT+6KOPrL9XrFihgIAA635xcbFWrVql6OjoaisOAADgQuibAABQyWB/5513SpIcDoeGDh3qNM/d3V3R0dF66aWXqq04AACAC6FvAlw+0U8tc3UJl8Xe5/q6ugSg0ioV7M+ePStJiomJ0ebNmxUcHFwjRQEAAFQEfRMAACoZ7Evs2bOnuusAAACoMvomAIC6rErBXpJWrVqlVatWKTs72/q2vMRbb711yYUBAABUBn0TAEBdVaVgP2nSJE2ePFnt27dXeHi4HA5HddcFAABQYfRNAAB1WZWC/WuvvaY5c+ZoyJAh1V0PAABApdE3AQDUZVX6HfvCwkLFxcVVdy0AAABVQt8EAFCXVSnYP/TQQ5o/f3511wIAAFAl9E0AAHVZlU7F/+mnn/TGG29o5cqVuv766+Xu7u40f+rUqdVSHAAAQEXQNwEA1GVVCvbffPONbrjhBknSjh07nOYxWA0AALjc6JsAAOqyKgX7NWvWVHcdAAAAVUbfBABQl1XpGnsAAAAAAFA7VOmIfdeuXS94Wtvq1aurXBAAAEBl0TcBANRlVQr2JdewlThz5oy2bt2qHTt2aOjQodVRFwAAQIXRNwEA1GVVCvYvv/xymdOTk5N18uTJSyoIAACgsuibAADqsmq9xv7+++/XW2+9VZ2LBAAAqDL6JgCAuqBag/3GjRvl5eVVnYsEAACoMvomAIC6oEqn4g8YMMDpvjFGhw8f1pYtW/T0009XS2HApYp+apmrS3Cy97m+ri4BAK5Y9E0AAHVZlYJ9QECA0/169eqpRYsWmjx5shISEqqlMAAAgIqibwIAqMuqFOxnz55d3XUAAABUGX0TAEBdVqVgXyIzM1O7du2Sw+FQ69atdeONN1ZXXQAAAJVG3wQAUBdVKdhnZ2fr3nvv1dq1a9WwYUMZY5Sbm6uuXbtqwYIFaty4cXXXCQAAUC76JgCAuqxKo+KPGjVKeXl52rlzp44dO6acnBzt2LFDeXl5Gj16dHXXCAAAcEH0TQAAdVmVjtgvX75cK1euVKtWraxprVu31quvvsoANQAA4LKjbwIAqMuqdMT+7Nmzcnd3LzXd3d1dZ8+eveSiAAAAKoO+CQCgLqtSsL/99tv1+9//XocOHbKm/fDDD3riiSfUrVu3aisOAACgIuibAADqsioF++nTp+vEiROKjo7W1VdfrWuuuUYxMTE6ceKE0tLSqrtGAACAC6JvAgCoy6p0jX1kZKS++uorZWRk6P/+7/9kjFHr1q3VvXv36q4PAADgouibAADqskodsV+9erVat26tvLw8SVKPHj00atQojR49Wh06dNB1112nzz//vEYKBQAAOB99EwAAKhnsp02bpocfflj+/v6l5gUEBGj48OGaOnVqtRUHAABwIfRNAACoZLDftm2bevXqVe78hIQEZWZmXnJRAAAAFUHfBACASgb7I0eOlPlTMiXc3Nx09OjRSy4KAACgIuibAABQyWDfpEkTbd++vdz533zzjcLDwy+5KAAAgIqgbwIAQCWDfZ8+ffTMM8/op59+KjUvPz9fEydOVGJiYrUVBwAAcCH0TQAAqOTP3f35z3/WokWLdO2112rkyJFq0aKFHA6Hdu3apVdffVXFxcWaMGFCTdUKAADghL4JAACVDPahoaHasGGDHnvsMY0fP17GGEmSw+FQz549NWPGDIWGhtZIoQAAAOejbwIAQCWDvSRFRUXpk08+UU5Ojr7//nsZY9S8eXMFBgbWRH0AAAAXRN8EAFDXVTrYlwgMDFSHDh2qsxYAAIAqo28CAKirKjV4HgAAAAAAqF0I9gAAAAAA2BjBHgAAAAAAGyPYAwAAAABgYwR7AAAAAABsjGAPAAAAAICNEewBAADKkZqaKofDoaSkJGuaMUbJycmKiIiQt7e3unTpop07dzo9rqCgQKNGjVJwcLB8fX3Vv39/HTx48DJXDwCoKwj2AAAAZdi8ebPeeOMNXX/99U7Tp0yZoqlTp2r69OnavHmzwsLC1KNHD504ccJqk5SUpMWLF2vBggVav369Tp48qcTERBUXF1/uzQAA1AEEewAAgPOcPHlSv/nNb/Tmm28qMDDQmm6M0bRp0zRhwgQNGDBAsbGxmjt3rk6fPq358+dLknJzczVr1iy99NJL6t69u2688Ualp6dr+/btWrlypas2CQBwBSPYAwAAnOfxxx9X37591b17d6fpe/bsUVZWlhISEqxpnp6eio+P14YNGyRJmZmZOnPmjFObiIgIxcbGWm3OV1BQoLy8PKcbAAAV5ebqAgAAAGqTBQsW6KuvvtLmzZtLzcvKypIkhYaGOk0PDQ3Vvn37rDYeHh5OR/pL2pQ8/nypqamaNGlSdZQPAKiDOGIPAADwiwMHDuj3v/+90tPT5eXlVW47h8PhdN8YU2ra+S7UZvz48crNzbVuBw4cqHzxAIA6i2APAADwi8zMTGVnZ6tdu3Zyc3OTm5ub1q1bp1deeUVubm7Wkfrzj7xnZ2db88LCwlRYWKicnJxy25zP09NT/v7+TjcAACqKYA8AAPCLbt26afv27dq6dat1a9++vX7zm99o69atatasmcLCwpSRkWE9prCwUOvWrVNcXJwkqV27dnJ3d3dqc/jwYe3YscNqAwBAdeIaewAAgF/4+fkpNjbWaZqvr68aNWpkTU9KSlJKSoqaN2+u5s2bKyUlRT4+Pho8eLAkKSAgQMOGDdOTTz6pRo0aKSgoSGPHjlWbNm1KDcYHAEB1INgDAABUwrhx45Sfn68RI0YoJydHHTt21Keffio/Pz+rzcsvvyw3NzcNGjRI+fn56tatm+bMmaP69eu7sHIAwJWKYA8AAHABa9eudbrvcDiUnJys5OTkch/j5eWltLQ0paWl1WxxAACIa+wBAAAAALA1gj0AAAAAADbm0mD/2WefqV+/foqIiJDD4dCSJUuc5htjlJycrIiICHl7e6tLly7auXOnU5uCggKNGjVKwcHB8vX1Vf/+/XXw4EGnNjk5ORoyZIgCAgIUEBCgIUOG6Pjx4zW8dQAAAAAA1DyXBvtTp06pbdu2mj59epnzp0yZoqlTp2r69OnavHmzwsLC1KNHD504ccJqk5SUpMWLF2vBggVav369Tp48qcTERBUXF1ttBg8erK1bt2r58uVavny5tm7dqiFDhtT49gEAAAAAUNNcOnhe79691bt37zLnGWM0bdo0TZgwQQMGDJAkzZ07V6GhoZo/f76GDx+u3NxczZo1S++884718zHp6emKjIzUypUr1bNnT+3atUvLly/Xpk2b1LFjR0nSm2++qU6dOmn37t1q0aLF5dlYAAAAAABqQK29xn7Pnj3KyspSQkKCNc3T01Px8fHasGGDJCkzM1NnzpxxahMREaHY2FirzcaNGxUQEGCFekm65ZZbFBAQYLUpS0FBgfLy8pxuAAAAAADUNrU22GdlZUmSQkNDnaaHhoZa87KysuTh4aHAwMALtgkJCSm1/JCQEKtNWVJTU61r8gMCAhQZGXlJ2wMAAAAAQE2otcG+hMPhcLpvjCk17Xzntymr/cWWM378eOXm5lq3AwcOVLJyAAAAAABqXq0N9mFhYZJU6qh6dna2dRQ/LCxMhYWFysnJuWCbI0eOlFr+0aNHS50NcC5PT0/5+/s73QAAAAAAqG1qbbCPiYlRWFiYMjIyrGmFhYVat26d4uLiJEnt2rWTu7u7U5vDhw9rx44dVptOnTopNzdXX375pdXmiy++UG5urtUGAAAAAAC7cumo+CdPntT3339v3d+zZ4+2bt2qoKAgXXXVVUpKSlJKSoqaN2+u5s2bKyUlRT4+Pho8eLAkKSAgQMOGDdOTTz6pRo0aKSgoSGPHjlWbNm2sUfJbtWqlXr166eGHH9brr78uSXrkkUeUmJjIiPgAAAAAANtzabDfsmWLunbtat0fM2aMJGno0KGaM2eOxo0bp/z8fI0YMUI5OTnq2LGjPv30U/n5+VmPefnll+Xm5qZBgwYpPz9f3bp105w5c1S/fn2rzbx58zR69Ghr9Pz+/ftr+vTpl2krAQAAAACoOS4N9l26dJExptz5DodDycnJSk5OLreNl5eX0tLSlJaWVm6boKAgpaenX0qpAAAAAADUSrX2GnsAAAAAAHBxBHsAAAAAAGyMYA8AAAAAgI0R7AEAAAAAsDGCPQAAAAAANkawBwAAAADAxgj2AAAAAADYGMEeAAAAAAAbI9gDAAAAAGBjBHsAAAAAAGyMYA8AAAAAgI0R7AEAAAAAsDGCPQAAAAAANkawBwAAAADAxgj2AAAAAADYGMEeAAAAAAAbI9gDAAAAAGBjBHsAAAAAAGyMYA8AAAAAgI0R7AEAAAAAsDGCPQAAAAAANkawBwAAAADAxgj2AAAAAADYGMEeAAAAAAAbI9gDAAAAAGBjBHsAAAAAAGyMYA8AAAAAgI0R7AEAAAAAsDGCPQAAAAAANkawBwAAAADAxgj2AAAAAADYGMEeAAAAAAAbI9gDAAAAAGBjBHsAAAAAAGyMYA8AAAAAgI0R7AEAAAAAsDGCPQAAAAAANkawBwAAAADAxgj2AAAAAADYGMEeAAAAAAAbI9gDAAAAAGBjBHsAAAAAAGyMYA8AAAAAgI0R7AEAAAAAsDGCPQAAAAAANkawBwAAAADAxgj2AAAAAADYGMEeAAAAAAAbI9gDAAAAAGBjBHsAAAAAAGyMYA8AAAAAgI0R7AEAAAAAsDGCPQAAAAAANkawBwAAAADAxgj2AAAAAADYGMEeAAAAAAAbI9gDAAAAAGBjBHsAAAAAAGyMYA8AAAAAgI0R7AEAAAAAsDGCPQAAAAAANkawBwAAAADAxgj2AAAAAADYGMEeAAAAAAAbI9gDAAAAAGBjBHsAAAAAAGyMYA8AAAAAgI0R7AEAAAAAsDGCPQAAAAAANkawBwAAAADAxgj2AAAAAADYGMEeAAAAAAAbI9gDAAAAAGBjBHsAAAAAAGyMYA8AAAAAgI0R7AEAAH6RmpqqDh06yM/PTyEhIbrzzju1e/dupzbGGCUnJysiIkLe3t7q0qWLdu7c6dSmoKBAo0aNUnBwsHx9fdW/f38dPHjwcm4KAKAOIdgDAAD8Yt26dXr88ce1adMmZWRkqKioSAkJCTp16pTVZsqUKZo6daqmT5+uzZs3KywsTD169NCJEyesNklJSVq8eLEWLFig9evX6+TJk0pMTFRxcbErNgsAcIVzc3UBAAAAtcXy5cud7s+ePVshISHKzMzUr371KxljNG3aNE2YMEEDBgyQJM2dO1ehoaGaP3++hg8frtzcXM2aNUvvvPOOunfvLklKT09XZGSkVq5cqZ49e1727QIAXNk4Yg8AAFCO3NxcSVJQUJAkac+ePcrKylJCQoLVxtPTU/Hx8dqwYYMkKTMzU2fOnHFqExERodjYWKvN+QoKCpSXl+d0AwCgogj2AAAAZTDGaMyYMbr11lsVGxsrScrKypIkhYaGOrUNDQ215mVlZcnDw0OBgYHltjlfamqqAgICrFtkZGR1bw4A4ApGsAcAACjDyJEj9c033+jdd98tNc/hcDjdN8aUmna+C7UZP368cnNzrduBAweqXjgAoM6p1cE+OTlZDofD6RYWFmbNZ1RaAABQE0aNGqWPPvpIa9asUdOmTa3pJf2Q84+8Z2dnW0fxw8LCVFhYqJycnHLbnM/T01P+/v5ONwAAKqpWB3tJuu6663T48GHrtn37dmseo9ICAIDqZIzRyJEjtWjRIq1evVoxMTFO82NiYhQWFqaMjAxrWmFhodatW6e4uDhJUrt27eTu7u7U5vDhw9qxY4fVBgCA6lTrR8V3c3NzOkpfglFpAQBAdXv88cc1f/58ffjhh/Lz87OOzAcEBMjb21sOh0NJSUlKSUlR8+bN1bx5c6WkpMjHx0eDBw+22g4bNkxPPvmkGjVqpKCgII0dO1Zt2rSx+iMAAFSnWn/E/rvvvlNERIRiYmJ077336r///a+kmhuVtgSj0wIAUPfMnDlTubm56tKli8LDw63bwoULrTbjxo1TUlKSRowYofbt2+uHH37Qp59+Kj8/P6vNyy+/rDvvvFODBg1S586d5ePjo6VLl6p+/fqu2CwAwBWuVh+x79ixo95++21de+21OnLkiJ599lnFxcVp586dFxyVdt++fZKqNiptidTUVE2aNKkatwYAANR2xpiLtnE4HEpOTlZycnK5bby8vJSWlqa0tLRqrA4AgLLV6iP2vXv31t13322durZs2TJJP59yX6K6R6Utwei0AAAAAAA7qNXB/ny+vr5q06aNvvvuuxoblbYEo9MCAAAAAOzAVsG+oKBAu3btUnh4OKPSAgAAAACgWn6N/dixY9WvXz9dddVVys7O1rPPPqu8vDwNHTqUUWkBAAAAAFAtD/YHDx7Ufffdp//9739q3LixbrnlFm3atElRUVGSfh6VNj8/XyNGjFBOTo46duxY5qi0bm5uGjRokPLz89WtWzfNmTOHUWkBAAAAAFeEWh3sFyxYcMH5jEoLAAAAAKjrbHWNPQAAAAAAcEawBwAAAADAxgj2AAAAAADYGMEeAAAAAAAbI9gDAAAAAGBjBHsAAAAAAGyMYA8AAAAAgI0R7AEAAAAAsDGCPQAAAAAANkawBwAAAADAxgj2AAAAAADYGMEeAAAAAAAbI9gDAAAAAGBjBHsAAAAAAGyMYA8AAAAAgI0R7AEAAAAAsDGCPQAAAAAANkawBwAAAADAxgj2AAAAAADYGMEeAAAAAAAbI9gDAAAAAGBjBHsAAAAAAGyMYA8AAAAAgI0R7AEAAAAAsDGCPQAAAAAANkawBwAAAADAxgj2AAAAAADYGMEeAAAAAAAbI9gDAAAAAGBjBHsAAAAAAGyMYA8AAAAAgI0R7AEAAAAAsDGCPQAAAAAANkawBwAAAADAxgj2AAAAAADYGMEeAAAAAAAbI9gDAAAAAGBjBHsAAAAAAGyMYA8AAAAAgI0R7AEAAAAAsDGCPQAAAAAANkawBwAAAADAxgj2AAAAAADYGMEeAAAAAAAbI9gDAAAAAGBjBHsAAAAAAGyMYA8AAAAAgI0R7AEAAAAAsDGCPQAAAAAANkawBwAAAADAxgj2AAAAAADYGMEeAAAAAAAbI9gDAAAAAGBjBHsAAAAAAGyMYA8AAAAAgI0R7AEAAAAAsDGCPQAAAAAANkawBwAAAADAxgj2AAAAAADYGMEeAAAAAAAbI9gDAAAAAGBjBHsAAAAAAGyMYA8AAAAAgI0R7AEAAAAAsDGCPQAAAAAANkawBwAAAADAxgj2AAAAAADYGMEeAAAAAAAbI9gDAAAAAGBjBHsAAAAAAGyMYA8AAAAAgI0R7AEAAAAAsDGCPQAAAAAANkawBwAAAADAxgj2AAAAAADYGMEeAAAAAAAbI9gDAAAAAGBjBHsAAAAAAGyMYA8AAAAAgI0R7AEAAAAAsDGCPQAAAAAANkawBwAAAADAxupUsJ8xY4ZiYmLk5eWldu3a6fPPP3d1SQAA4ApG3wMAcDnUmWC/cOFCJSUlacKECfr666912223qXfv3tq/f7+rSwMAAFcg+h4AgMulzgT7qVOnatiwYXrooYfUqlUrTZs2TZGRkZo5c6arSwMAAFcg+h4AgMvFzdUFXA6FhYXKzMzUU0895TQ9ISFBGzZsKPMxBQUFKigosO7n5uZKkvLy8i6plrMFpy/p8TWhIttU2+q+UmuOnbjiMlRScTsm9XR1CUCtUvI+Nsa4uBLUdpXte9RUv0OqfZ+HNamq+4t9VDF1ZT9dyj6qbX25mlTVfiL7qHIq2veoE8H+f//7n4qLixUaGuo0PTQ0VFlZWWU+JjU1VZMmTSo1PTIyskZqdKWAaa6uoPKo+fKwY83A5XDixAkFBAS4ugzUYpXte9SlfkdN4nPr4thHF8c+qhj208VV5z66WN+jTgT7Eg6Hw+m+MabUtBLjx4/XmDFjrPtnz57VsWPH1KhRo3Ifc7nk5eUpMjJSBw4ckL+/v0trqQw71k3Nlwc1Xx52rFmqXXUbY3TixAlFRES4tA7YR0X7HrW531FZtek9W5uxny6OfXRx7KOKsfN+qmjfo04E++DgYNWvX7/UN+TZ2dmlvkkv4enpKU9PT6dpDRs2rKkSq8Tf3992L0zJnnVT8+VBzZeHHWuWak/dHKlHRVS272GHfkdl1Zb3bG3Hfro49tHFsY8qxq77qSJ9jzoxeJ6Hh4fatWunjIwMp+kZGRmKi4tzUVUAAOBKRd8DAHA51Ykj9pI0ZswYDRkyRO3bt1enTp30xhtvaP/+/Xr00UddXRoAALgC0fcAAFwudSbY33PPPfrxxx81efJkHT58WLGxsfrkk08UFRXl6tIqzdPTUxMnTix1yl5tZ8e6qfnyoObLw441S/atG7iS+h6VwXu2YthPF8c+ujj2UcXUhf3kMPxmDwAAAAAAtlUnrrEHAAAAAOBKRbAHAAAAAMDGCPYAAAAAANgYwR4AAAAAABsj2NvMZ599pn79+ikiIkIOh0NLlixxdUkVMmPGDMXExMjLy0vt2rXT559/7uqSypWamqoOHTrIz89PISEhuvPOO7V7925Xl2V7FdmvxhglJycrIiJC3t7e6tKli3bu3Omiin92sffcokWL1LNnTwUHB8vhcGjr1q0uqRMAXM2ufRRXsFO/yBXoiwGVR7C3mVOnTqlt27aaPn26q0upsIULFyopKUkTJkzQ119/rdtuu029e/fW/v37XV1amdatW6fHH39cmzZtUkZGhoqKipSQkKBTp065ujRbq8h+nTJliqZOnarp06dr8+bNCgsLU48ePXTixAmX1X2x99ypU6fUuXNnPffcc5e5MgCoXezYR3EFu/WLXIG+GFAFBrYlySxevNjVZVzUzTffbB599FGnaS1btjRPPfWUiyqqnOzsbCPJrFu3ztWlXFHO369nz541YWFh5rnnnrPa/PTTTyYgIMC89tprrirTyYXec3v27DGSzNdff31Za6qon376yYwaNco0btzYeHp6ms6dO5svv/zS1WUBuELZpY/iCnbvF7kCfbGy8dmOc3HEHjWqsLBQmZmZSkhIcJqekJCgDRs2uKiqysnNzZUkBQUFubiSK8v5+3XPnj3Kyspyeq14enoqPj7eNq+V2mzcuHH6xz/+oblz5+qrr77SNddco549e+rYsWOuLg0A6owroV/kCvTFysZnO85FsEeN+t///qfi4mKFhoY6TQ8NDVVWVpaLqqo4Y4zGjBmjW2+9VbGxsa4u56LmzZunBg0aWLfaes1eWfu15PVg19dKbXbq1CnNnDlTL7zwgnr37q3WrVvrzTfflLe3t2bNmuXq8ko5/3U8adIkp/ucrgrAruzeL3IFu/XFLhe7fba7kl36x5fKzdUFoG5wOBxO940xpabVRiNHjtQ333yj9evXu7qUCunfv786duxo3W/SpIkLqynfhfarXV8rtdl//vMfnTlzRp07d7amubu76+abb9auXbtcWFnZzn8de3t76ze/+Y11PyIiwhVlAUC14bOu4uzWF7tc7PbZ7kp26R9fKoI9alRwcLDq169f6lvo7OzsUt9W1zajRo3SRx99pM8++0xNmzZ1dTkV4ufnJz8/P1eXcUHl7dewsDBJPx+5Dw8Pt6bb4bVS2xljJNmnI2mH1zEAVIWd+0WuYMe+2OVit892V6or/QpOxUeN8vDwULt27ZSRkeE0PSMjQ3FxcS6q6sKMMRo5cqQWLVqk1atXKyYmxtUlXREutl9jYmIUFhbm9FopLCzUunXrau1rxS6uueYaeXh4OB3tOHPmjLZs2aJWrVq5sDIAqFvs2C9yBfpiF8dnO87HEXubOXnypL7//nvr/p49e7R161YFBQXpqquucmFl5RszZoyGDBmi9u3bq1OnTnrjjTe0f/9+Pfroo64urUyPP/645s+frw8//FB+fn7Wt+oBAQHy9vZ2cXX2dbH96nA4lJSUpJSUFDVv3lzNmzdXSkqKfHx8NHjwYJfVfbH33LFjx7R//34dOnRIkqzf2Q0LC7POQnA1X19fPfbYY/rDH/5g1T1lyhSdPn1aw4YNc3V5AK4QduyjuILd+kWuQF/s4vhsRykuGYsfVbZmzRojqdRt6NChri7tgl599VUTFRVlPDw8zE033VSrf66krP0rycyePdvVpdlaRfbr2bNnzcSJE01YWJjx9PQ0v/rVr8z27dtdV7S5+Htu9uzZZc6fOHGiS+s+X35+vhk1apQJDg7mJ3EA1Ai79lFcwU79IlegL1YxfLbjXA5jfrlAAwAAAAAA2A7X2AMAAAAAYGMEewAAAAAAbIxgDwAAAACAjRHsAQAAAACwMYI9AAAAAAA2RrAHAAAAAMDGCPYAAAAAANgYwR4AAAAAABsj2AMo05w5c9SwYcNLXo7D4dCSJUsueTkAAODKRb8DuDQEe+AK9sADD+jOO+90dRkAAKAOoN8BuA7BHgAAAAAAGyPYA3XU1KlT1aZNG/n6+ioyMlIjRozQyZMnS7VbsmSJrr32Wnl5ealHjx46cOCA0/ylS5eqXbt28vLyUrNmzTRp0iQVFRWVuc7CwkKNHDlS4eHh8vLyUnR0tFJTU2tk+wAAQO1BvwOoWQR7oI6qV6+eXnnlFe3YsUNz587V6tWrNW7cOKc2p0+f1l//+lfNnTtX//rXv5SXl6d7773Xmr9ixQrdf//9Gj16tL799lu9/vrrmjNnjv7617+Wuc5XXnlFH330kd577z3t3r1b6enpio6OrsnNBAAAtQD9DqCGGQBXrKFDh5o77rijQm3fe+8906hRI+v+7NmzjSSzadMma9quXbuMJPPFF18YY4y57bbbTEpKitNy3nnnHRMeHm7dl2QWL15sjDFm1KhR5vbbbzdnz56t4hYBAIDain4H4DocsQfqqDVr1qhHjx5q0qSJ/Pz89Nvf/lY//vijTp06ZbVxc3NT+/btrfstW7ZUw4YNtWvXLklSZmamJk+erAYNGli3hx9+WIcPH9bp06dLrfOBBx7Q1q1b1aJFC40ePVqffvppzW8oAABwOfodQM0i2AN10L59+9SnTx/FxsbqH//4hzIzM/Xqq69Kks6cOePU1uFwlHp8ybSzZ89q0qRJ2rp1q3Xbvn27vvvuO3l5eZV63E033aQ9e/boL3/5i/Lz8zVo0CANHDiwBrYQAADUFvQ7gJrn5uoCAFx+W7ZsUVFRkV566SXVq/fz93vvvfdeqXZFRUXasmWLbr75ZknS7t27dfz4cbVs2VLSzx+Yu3fv1jXXXFPhdfv7++uee+7RPffco4EDB6pXr146duyYgoKCqmHLAABAbUO/A6h5BHvgCpebm6utW7c6TWvcuLGKioqUlpamfv366V//+pdee+21Uo91d3fXqFGj9Morr8jd3V0jR47ULbfcYn3gPvPMM0pMTFRkZKR+/etfq169evrmm2+0fft2Pfvss6WW9/LLLys8PFw33HCD6tWrp/fff19hYWFq2LBhTWw6AAC4zOh3AK7BqfjAFW7t2rW68cYbnW5vvfWWpk6dqueff16xsbGaN29emT//4uPjoz/+8Y8aPHiwOnXqJG9vby1YsMCa37NnT3388cfKyMhQhw4ddMstt2jq1KmKiooqs5YGDRro+eefV/v27dWhQwft3btXn3zyifXtPQAAsDf6HYBrOIwxxtVFAAAAAACAquHrKgAAAAAAbIxgDwAAAACAjRHsAQAAAACwMYI9AAAAAAA2RrAHAAAAAMDGCPYAAAAAANgYwR4AAAAAABsj2AMAAAAAYGMEewAAAAAAbIxgDwAAAACAjRHsAQAAAACwsf8HDnjg81jID5wAAAAASUVORK5CYII="},"metadata":{}}]},{"cell_type":"code","source":"# Check for missing values in the train dataset\ntrain_na = df_train.isna().sum()\nprint('Train Dataset - Missing Values:')\nprint(train_na)\n\ntest_na = df_test.isna().sum()\nprint('\\nTest Dataset - Missing Values:')\nprint(test_na)","metadata":{"execution":{"iopub.status.busy":"2023-06-23T02:45:47.160842Z","iopub.execute_input":"2023-06-23T02:45:47.161832Z","iopub.status.idle":"2023-06-23T02:45:47.175162Z","shell.execute_reply.started":"2023-06-23T02:45:47.161799Z","shell.execute_reply":"2023-06-23T02:45:47.174251Z"},"trusted":true},"execution_count":4,"outputs":[{"name":"stdout","text":"Train Dataset - Missing Values:\ntext     1\nlabel    0\ndtype: int64\n\nTest Dataset - Missing Values:\ntext     1\nlabel    0\ndtype: int64\n","output_type":"stream"}]},{"cell_type":"code","source":"df_train = df_train.dropna()\ndf_test = df_test.dropna()\n\n# reset indexing \ndf_train = df_train.reset_index(drop=True)","metadata":{"execution":{"iopub.status.busy":"2023-06-23T02:45:47.176905Z","iopub.execute_input":"2023-06-23T02:45:47.177267Z","iopub.status.idle":"2023-06-23T02:45:47.192243Z","shell.execute_reply.started":"2023-06-23T02:45:47.177237Z","shell.execute_reply":"2023-06-23T02:45:47.191011Z"},"trusted":true},"execution_count":5,"outputs":[]},{"cell_type":"code","source":"# Check for missing values in the train dataset\ntrain_na = df_train.isna().sum()\nprint('Train Dataset - Missing Values:')\nprint(train_na)\n\ntest_na = df_test.isna().sum()\nprint('\\nTest Dataset - Missing Values:')\nprint(test_na)\n\nprint(\"Train Dataset Size:\",df_train.shape )\nprint(\"Test Dataset Size:\", df_test.shape)","metadata":{"execution":{"iopub.status.busy":"2023-06-23T02:45:47.194119Z","iopub.execute_input":"2023-06-23T02:45:47.194487Z","iopub.status.idle":"2023-06-23T02:45:47.210035Z","shell.execute_reply.started":"2023-06-23T02:45:47.194455Z","shell.execute_reply":"2023-06-23T02:45:47.209175Z"},"trusted":true},"execution_count":6,"outputs":[{"name":"stdout","text":"Train Dataset - Missing Values:\ntext     0\nlabel    0\ndtype: int64\n\nTest Dataset - Missing Values:\ntext     0\nlabel    0\ndtype: int64\nTrain Dataset Size: (5999, 2)\nTest Dataset Size: (1995, 2)\n","output_type":"stream"}]},{"cell_type":"code","source":"# remove unwanted datas\nprint(df_train['label'].unique())\nprint(df_test['label'].unique())\n\nvalid_labels = ['0', '1', '2']\n\n\n# Filter train dataset\ndf_train = df_train.loc[df_train['label'].isin(valid_labels)]\ndf_train.loc[:, 'label'] = df_train['label'].astype(int)\n\n# Filter test dataset\ndf_test = df_test.loc[df_test['label'].isin(valid_labels)]\ndf_test.loc[:, 'label'] = df_test['label'].astype(int)\n","metadata":{"execution":{"iopub.status.busy":"2023-06-23T02:45:47.211298Z","iopub.execute_input":"2023-06-23T02:45:47.212123Z","iopub.status.idle":"2023-06-23T02:45:47.229412Z","shell.execute_reply.started":"2023-06-23T02:45:47.212089Z","shell.execute_reply":"2023-06-23T02:45:47.228567Z"},"trusted":true},"execution_count":7,"outputs":[{"name":"stdout","text":"['2' '1' '0' '20' '11' '-' 'o' '--']\n['1' '0' '2' 'o' '-']\n","output_type":"stream"},{"name":"stderr","text":"/tmp/ipykernel_29/1816761743.py:10: DeprecationWarning: In a future version, `df.iloc[:, i] = newvals` will attempt to set the values inplace instead of always setting a new array. To retain the old behavior, use either `df[df.columns[i]] = newvals` or, if columns are non-unique, `df.isetitem(i, newvals)`\n  df_train.loc[:, 'label'] = df_train['label'].astype(int)\n/tmp/ipykernel_29/1816761743.py:14: DeprecationWarning: In a future version, `df.iloc[:, i] = newvals` will attempt to set the values inplace instead of always setting a new array. To retain the old behavior, use either `df[df.columns[i]] = newvals` or, if columns are non-unique, `df.isetitem(i, newvals)`\n  df_test.loc[:, 'label'] = df_test['label'].astype(int)\n","output_type":"stream"}]},{"cell_type":"code","source":"print(df_train['label'])\nprint(df_test['label'])","metadata":{"execution":{"iopub.status.busy":"2023-06-23T02:45:47.231022Z","iopub.execute_input":"2023-06-23T02:45:47.231634Z","iopub.status.idle":"2023-06-23T02:45:47.238944Z","shell.execute_reply.started":"2023-06-23T02:45:47.231594Z","shell.execute_reply":"2023-06-23T02:45:47.238059Z"},"trusted":true},"execution_count":8,"outputs":[{"name":"stdout","text":"0       2\n1       1\n2       2\n3       2\n4       1\n       ..\n5994    1\n5995    2\n5996    2\n5997    1\n5998    1\nName: label, Length: 5990, dtype: int64\n0       1\n1       1\n2       1\n3       1\n4       1\n       ..\n1991    0\n1992    0\n1993    0\n1994    0\n1995    0\nName: label, Length: 1993, dtype: int64\n","output_type":"stream"}]},{"cell_type":"code","source":"label_encoder = LabelEncoder()","metadata":{"execution":{"iopub.status.busy":"2023-06-23T02:45:47.240295Z","iopub.execute_input":"2023-06-23T02:45:47.240896Z","iopub.status.idle":"2023-06-23T02:45:47.247755Z","shell.execute_reply.started":"2023-06-23T02:45:47.240866Z","shell.execute_reply":"2023-06-23T02:45:47.246911Z"},"trusted":true},"execution_count":9,"outputs":[]},{"cell_type":"code","source":"tokenizer = AutoTokenizer.from_pretrained('bert-base-multilingual-cased')","metadata":{"execution":{"iopub.status.busy":"2023-06-23T02:45:47.252095Z","iopub.execute_input":"2023-06-23T02:45:47.252672Z","iopub.status.idle":"2023-06-23T02:45:48.466395Z","shell.execute_reply.started":"2023-06-23T02:45:47.252639Z","shell.execute_reply":"2023-06-23T02:45:48.465432Z"},"trusted":true},"execution_count":10,"outputs":[{"output_type":"display_data","data":{"text/plain":"Downloading (…)okenizer_config.json:   0%|          | 0.00/29.0 [00:00<?, ?B/s]","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"3af3e47d8afa4c7cbadd903277b90c32"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":"Downloading (…)lve/main/config.json:   0%|          | 0.00/625 [00:00<?, ?B/s]","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"01690e7742604e85bc461a9c67d459bd"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":"Downloading (…)solve/main/vocab.txt: 0.00B [00:00, ?B/s]","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"a448174ddb4643019c17afe96d710933"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":"Downloading (…)/main/tokenizer.json: 0.00B [00:00, ?B/s]","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"669c2525c53646029c51d00ff09f5f3b"}},"metadata":{}}]},{"cell_type":"code","source":"df_train['text'] = df_train['text'].tolist()\n\nprint(type(df_train['text']))\nprint(df_train['text'].head())","metadata":{"execution":{"iopub.status.busy":"2023-06-23T02:45:48.467692Z","iopub.execute_input":"2023-06-23T02:45:48.468040Z","iopub.status.idle":"2023-06-23T02:45:48.476443Z","shell.execute_reply.started":"2023-06-23T02:45:48.468008Z","shell.execute_reply":"2023-06-23T02:45:48.475412Z"},"trusted":true},"execution_count":11,"outputs":[{"name":"stdout","text":"<class 'pandas.core.series.Series'>\n0    बजार ले जसरी ट्रेन्ड चेन्ज गर्��ो यो हेर्दा तत्...\n1    1000 अंकले घटेको नेप्से 200 अंकले बढ्नु ठूलो क...\n2    होइन यो सानिमा बैंक ले bonus घोसणा गरेको २ महि...\n3    खैँ MBJC प्रति कित्तामा रू,10/-ले बढेर आज रू,1...\n4                                         राम्रो भयो️️\nName: text, dtype: object\n","output_type":"stream"}]},{"cell_type":"code","source":"# tokenizing training data\ntrain_tokens = tokenizer.batch_encode_plus(\n    df_train['text'].tolist(),\n    padding = True,\n    truncation = True,\n    max_length = 512,\n    return_tensors = 'pt'\n)\n\ntrain_labels = label_encoder.fit_transform(df_train['label'])\n","metadata":{"execution":{"iopub.status.busy":"2023-06-23T02:45:48.477697Z","iopub.execute_input":"2023-06-23T02:45:48.478786Z","iopub.status.idle":"2023-06-23T02:45:52.075027Z","shell.execute_reply.started":"2023-06-23T02:45:48.478748Z","shell.execute_reply":"2023-06-23T02:45:52.074019Z"},"trusted":true},"execution_count":12,"outputs":[]},{"cell_type":"code","source":"print(train_labels)","metadata":{"execution":{"iopub.status.busy":"2023-06-23T02:45:52.076254Z","iopub.execute_input":"2023-06-23T02:45:52.078427Z","iopub.status.idle":"2023-06-23T02:45:52.083785Z","shell.execute_reply.started":"2023-06-23T02:45:52.078393Z","shell.execute_reply":"2023-06-23T02:45:52.082715Z"},"trusted":true},"execution_count":13,"outputs":[{"name":"stdout","text":"[2 1 2 ... 2 1 1]\n","output_type":"stream"}]},{"cell_type":"code","source":"# tokenizing test data \ntest_tokens = tokenizer.batch_encode_plus(\n    df_test['text'].tolist(),\n    padding=True,\n    truncation=True,\n    max_length=512,\n    return_tensors='pt'\n)\n\ntest_labels = label_encoder.fit_transform(df_test['label'])","metadata":{"execution":{"iopub.status.busy":"2023-06-23T02:45:52.085197Z","iopub.execute_input":"2023-06-23T02:45:52.086338Z","iopub.status.idle":"2023-06-23T02:45:53.373535Z","shell.execute_reply.started":"2023-06-23T02:45:52.086273Z","shell.execute_reply":"2023-06-23T02:45:53.372541Z"},"trusted":true},"execution_count":14,"outputs":[]},{"cell_type":"code","source":"print(test_labels)","metadata":{"execution":{"iopub.status.busy":"2023-06-23T02:45:53.374844Z","iopub.execute_input":"2023-06-23T02:45:53.375282Z","iopub.status.idle":"2023-06-23T02:45:53.380875Z","shell.execute_reply.started":"2023-06-23T02:45:53.375249Z","shell.execute_reply":"2023-06-23T02:45:53.379982Z"},"trusted":true},"execution_count":15,"outputs":[{"name":"stdout","text":"[1 1 1 ... 0 0 0]\n","output_type":"stream"}]},{"cell_type":"code","source":"# Convert input sequences to tensors\ntrain_input_ids = torch.tensor(train_tokens['input_ids'])\ntrain_attention_masks = torch.tensor(train_tokens['attention_mask'])\ntrain_labels = torch.tensor(df_train['label'].values)\n\ntest_input_ids = torch.tensor(test_tokens['input_ids'])\ntest_attention_masks = torch.tensor(test_tokens['attention_mask'])\ntest_labels = torch.tensor(df_test['label'].values)\n\n# labels\ntrain_labels = torch.tensor(df_train['label'].values)\ntest_labels = torch.tensor(df_test['label'].values)\n\n\n# Create a TensorDataset\ntrain_dataset = TensorDataset(train_input_ids, train_attention_masks, train_labels)\ntest_dataset = TensorDataset(test_input_ids, test_attention_masks, test_labels)","metadata":{"execution":{"iopub.status.busy":"2023-06-23T02:45:53.382130Z","iopub.execute_input":"2023-06-23T02:45:53.382651Z","iopub.status.idle":"2023-06-23T02:45:53.458380Z","shell.execute_reply.started":"2023-06-23T02:45:53.382612Z","shell.execute_reply":"2023-06-23T02:45:53.457465Z"},"trusted":true},"execution_count":16,"outputs":[{"name":"stderr","text":"/tmp/ipykernel_29/3236953423.py:2: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).\n  train_input_ids = torch.tensor(train_tokens['input_ids'])\n/tmp/ipykernel_29/3236953423.py:3: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).\n  train_attention_masks = torch.tensor(train_tokens['attention_mask'])\n/tmp/ipykernel_29/3236953423.py:6: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).\n  test_input_ids = torch.tensor(test_tokens['input_ids'])\n/tmp/ipykernel_29/3236953423.py:7: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).\n  test_attention_masks = torch.tensor(test_tokens['attention_mask'])\n","output_type":"stream"}]},{"cell_type":"code","source":"type(train_dataset)","metadata":{"execution":{"iopub.status.busy":"2023-06-23T02:45:53.459669Z","iopub.execute_input":"2023-06-23T02:45:53.460766Z","iopub.status.idle":"2023-06-23T02:45:53.466777Z","shell.execute_reply.started":"2023-06-23T02:45:53.460731Z","shell.execute_reply":"2023-06-23T02:45:53.465812Z"},"trusted":true},"execution_count":17,"outputs":[{"execution_count":17,"output_type":"execute_result","data":{"text/plain":"torch.utils.data.dataset.TensorDataset"},"metadata":{}}]},{"cell_type":"code","source":"model = BertForSequenceClassification.from_pretrained('bert-base-multilingual-cased', num_labels=3)","metadata":{"execution":{"iopub.status.busy":"2023-06-23T02:45:53.468096Z","iopub.execute_input":"2023-06-23T02:45:53.468685Z","iopub.status.idle":"2023-06-23T02:46:04.217512Z","shell.execute_reply.started":"2023-06-23T02:45:53.468652Z","shell.execute_reply":"2023-06-23T02:46:04.216653Z"},"trusted":true},"execution_count":18,"outputs":[{"output_type":"display_data","data":{"text/plain":"Downloading model.safetensors:   0%|          | 0.00/714M [00:00<?, ?B/s]","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"5ace0edd7b414f73a7974749af582433"}},"metadata":{}},{"name":"stderr","text":"Some weights of the model checkpoint at bert-base-multilingual-cased were not used when initializing BertForSequenceClassification: ['cls.seq_relationship.weight', 'cls.predictions.transform.LayerNorm.bias', 'cls.predictions.transform.LayerNorm.weight', 'cls.predictions.bias', 'cls.predictions.transform.dense.bias', 'cls.predictions.transform.dense.weight', 'cls.seq_relationship.bias']\n- This IS expected if you are initializing BertForSequenceClassification from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n- This IS NOT expected if you are initializing BertForSequenceClassification from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\nSome weights of BertForSequenceClassification were not initialized from the model checkpoint at bert-base-multilingual-cased and are newly initialized: ['classifier.bias', 'classifier.weight']\nYou should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.\n","output_type":"stream"}]},{"cell_type":"code","source":"# Setting device\ndevice = torch.device('cuda' if torch.cuda.is_available() else 'cpu')\nmodel.to(device)","metadata":{"execution":{"iopub.status.busy":"2023-06-23T02:46:04.218899Z","iopub.execute_input":"2023-06-23T02:46:04.219358Z","iopub.status.idle":"2023-06-23T02:46:13.086573Z","shell.execute_reply.started":"2023-06-23T02:46:04.219325Z","shell.execute_reply":"2023-06-23T02:46:13.085532Z"},"trusted":true},"execution_count":19,"outputs":[{"execution_count":19,"output_type":"execute_result","data":{"text/plain":"BertForSequenceClassification(\n  (bert): BertModel(\n    (embeddings): BertEmbeddings(\n      (word_embeddings): Embedding(119547, 768, padding_idx=0)\n      (position_embeddings): Embedding(512, 768)\n      (token_type_embeddings): Embedding(2, 768)\n      (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)\n      (dropout): Dropout(p=0.1, inplace=False)\n    )\n    (encoder): BertEncoder(\n      (layer): ModuleList(\n        (0-11): 12 x BertLayer(\n          (attention): BertAttention(\n            (self): BertSelfAttention(\n              (query): Linear(in_features=768, out_features=768, bias=True)\n              (key): Linear(in_features=768, out_features=768, bias=True)\n              (value): Linear(in_features=768, out_features=768, bias=True)\n              (dropout): Dropout(p=0.1, inplace=False)\n            )\n            (output): BertSelfOutput(\n              (dense): Linear(in_features=768, out_features=768, bias=True)\n              (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)\n              (dropout): Dropout(p=0.1, inplace=False)\n            )\n          )\n          (intermediate): BertIntermediate(\n            (dense): Linear(in_features=768, out_features=3072, bias=True)\n            (intermediate_act_fn): GELUActivation()\n          )\n          (output): BertOutput(\n            (dense): Linear(in_features=3072, out_features=768, bias=True)\n            (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)\n            (dropout): Dropout(p=0.1, inplace=False)\n          )\n        )\n      )\n    )\n    (pooler): BertPooler(\n      (dense): Linear(in_features=768, out_features=768, bias=True)\n      (activation): Tanh()\n    )\n  )\n  (dropout): Dropout(p=0.1, inplace=False)\n  (classifier): Linear(in_features=768, out_features=3, bias=True)\n)"},"metadata":{}}]},{"cell_type":"code","source":"batch_size = 16\nlearning_rate = 2e-5\nepochs = 5\n\ntrain_dataloader = DataLoader(train_dataset, sampler=RandomSampler(train_dataset), batch_size=batch_size)\noptimizer = AdamW(model.parameters(), lr=learning_rate, eps=1e-8)","metadata":{"execution":{"iopub.status.busy":"2023-06-23T02:46:13.088069Z","iopub.execute_input":"2023-06-23T02:46:13.088658Z","iopub.status.idle":"2023-06-23T02:46:13.101795Z","shell.execute_reply.started":"2023-06-23T02:46:13.088622Z","shell.execute_reply":"2023-06-23T02:46:13.100820Z"},"trusted":true},"execution_count":20,"outputs":[{"name":"stderr","text":"/opt/conda/lib/python3.10/site-packages/transformers/optimization.py:411: FutureWarning: This implementation of AdamW is deprecated and will be removed in a future version. Use the PyTorch implementation torch.optim.AdamW instead, or set `no_deprecation_warning=True` to disable this warning\n  warnings.warn(\n","output_type":"stream"}]},{"cell_type":"code","source":"from tqdm import tqdm\n\n# Set random seed for reproducibility\nseed = 42\ntorch.manual_seed(seed)\ntorch.cuda.manual_seed_all(seed)\nnp.random.seed(seed)\n\n# Training\nmodel.train()\n\nfor epoch in range(epochs):\n    total_loss = 0\n    progress_bar = tqdm(train_dataloader, desc=f'Epoch {epoch+1}')\n\n    for batch in progress_bar:\n        batch = tuple(t.to(device) for t in batch)\n        inputs = {\n            'input_ids': batch[0],\n            'attention_mask': batch[1],\n            'labels': batch[2]\n        }\n\n        optimizer.zero_grad()\n        outputs = model(**inputs)\n        loss = outputs.loss\n        total_loss += loss.item()\n\n        loss.backward()\n        torch.nn.utils.clip_grad_norm_(model.parameters(), max_norm=1.0)  # Apply gradient clipping if necessary\n        optimizer.step()\n\n        progress_bar.set_postfix({'Training Loss': loss.item()})\n\n    average_loss = total_loss / len(train_dataloader)\n    print(f'Epoch {epoch+1}: Average Loss: {average_loss:.4f}')","metadata":{"execution":{"iopub.status.busy":"2023-06-23T02:46:13.103179Z","iopub.execute_input":"2023-06-23T02:46:13.103767Z","iopub.status.idle":"2023-06-23T03:28:29.362425Z","shell.execute_reply.started":"2023-06-23T02:46:13.103736Z","shell.execute_reply":"2023-06-23T03:28:29.361487Z"},"trusted":true},"execution_count":21,"outputs":[{"name":"stderr","text":"Epoch 1: 100%|██████████| 375/375 [08:29<00:00,  1.36s/it, Training Loss=0.693]\n","output_type":"stream"},{"name":"stdout","text":"Epoch 1: Average Loss: 0.8459\n","output_type":"stream"},{"name":"stderr","text":"Epoch 2: 100%|██████████| 375/375 [08:25<00:00,  1.35s/it, Training Loss=1.03] \n","output_type":"stream"},{"name":"stdout","text":"Epoch 2: Average Loss: 0.6496\n","output_type":"stream"},{"name":"stderr","text":"Epoch 3: 100%|██████████| 375/375 [08:26<00:00,  1.35s/it, Training Loss=0.491]\n","output_type":"stream"},{"name":"stdout","text":"Epoch 3: Average Loss: 0.5399\n","output_type":"stream"},{"name":"stderr","text":"Epoch 4: 100%|██████████| 375/375 [08:26<00:00,  1.35s/it, Training Loss=0.39] \n","output_type":"stream"},{"name":"stdout","text":"Epoch 4: Average Loss: 0.4444\n","output_type":"stream"},{"name":"stderr","text":"Epoch 5: 100%|██████████| 375/375 [08:26<00:00,  1.35s/it, Training Loss=0.417] ","output_type":"stream"},{"name":"stdout","text":"Epoch 5: Average Loss: 0.3991\n","output_type":"stream"},{"name":"stderr","text":"\n","output_type":"stream"}]},{"cell_type":"code","source":"# Saving the trained model\noutput_dir = './trained_model'\nos.makedirs(output_dir, exist_ok=True)\nmodel.save_pretrained(output_dir)\n\nprint('Training completed!')","metadata":{"execution":{"iopub.status.busy":"2023-06-23T03:28:29.363936Z","iopub.execute_input":"2023-06-23T03:28:29.364514Z","iopub.status.idle":"2023-06-23T03:28:30.530487Z","shell.execute_reply.started":"2023-06-23T03:28:29.364479Z","shell.execute_reply":"2023-06-23T03:28:30.529365Z"},"trusted":true},"execution_count":22,"outputs":[{"name":"stdout","text":"Training completed!\n","output_type":"stream"}]},{"cell_type":"code","source":"model.eval()  # Set the model in evaluation mode\n\ncorrect = 0\ntotal = 0\n\ntest_dataloader = DataLoader(test_dataset, sampler=RandomSampler(test_dataset), batch_size=batch_size)\n\nwith torch.no_grad():  # Disable gradient calculations during evaluation\n    progress_bar = tqdm(test_dataloader, desc=\"Evaluating\")\n    for batch in progress_bar:\n        batch_input_ids, batch_attention_masks, batch_labels = batch\n        \n        batch_input_ids = batch_input_ids.to(device)\n        batch_attention_masks = batch_attention_masks.to(device)\n        batch_labels = batch_labels.to(device)\n\n        outputs = model(input_ids=batch_input_ids, attention_mask=batch_attention_masks)\n        logits = outputs.logits\n        _, predicted_labels = torch.max(logits, dim=1)\n        \n        total += batch_labels.size(0)\n        correct += (predicted_labels == batch_labels).sum().item()\n\n        accuracy = (correct / total) * 100\n        progress_bar.set_postfix({'Accuracy': f'{accuracy:.2f}%'})\n\nprint(f\"Accuracy: {accuracy:.2f}%\")\n","metadata":{"execution":{"iopub.status.busy":"2023-06-23T03:28:30.531897Z","iopub.execute_input":"2023-06-23T03:28:30.532325Z","iopub.status.idle":"2023-06-23T03:29:41.828015Z","shell.execute_reply.started":"2023-06-23T03:28:30.532291Z","shell.execute_reply":"2023-06-23T03:29:41.827108Z"},"trusted":true},"execution_count":23,"outputs":[{"name":"stderr","text":"Evaluating: 100%|██████████| 125/125 [01:11<00:00,  1.75it/s, Accuracy=61.62%]","output_type":"stream"},{"name":"stdout","text":"Accuracy: 61.62%\n","output_type":"stream"},{"name":"stderr","text":"\n","output_type":"stream"}]},{"cell_type":"code","source":"# SAVING MODEL\nmodel.save_pretrained(\"my-model\")\n\n# Download\nimport shutil\n\nmodel_directory = \"my-model\"\n\n# Zip the model directory\nshutil.make_archive(\"my-model\", 'zip', model_directory)","metadata":{"execution":{"iopub.status.busy":"2023-06-23T03:32:44.408184Z","iopub.execute_input":"2023-06-23T03:32:44.408577Z","iopub.status.idle":"2023-06-23T03:33:27.095351Z","shell.execute_reply.started":"2023-06-23T03:32:44.408537Z","shell.execute_reply":"2023-06-23T03:33:27.094441Z"},"trusted":true},"execution_count":24,"outputs":[{"execution_count":24,"output_type":"execute_result","data":{"text/plain":"'/kaggle/working/my-model.zip'"},"metadata":{}}]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]}]}

requirements.txt

@@ -0,0 +1,5 @@
+torch==1.9.0
+transformers==4.8.2
+tqdm==4.62.0
+pandas==1.3.0
+numpy==1.21.0