binary_classification_OpenAI.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h1 align=\"center\"><font color=\"yellow\">Fine-tuning: Classificação Binária</font></h1>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<font color=\"yellow\">Data Scientist.: Dr.Eddy Giusepe Chirinos Isidro</font>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Outros links des estudo para Fine-Tuning:\n",
    "\n",
    "* [Fine Tuning OpenAI’s GPT3 model](https://medium.com/@_sumitsaha_/fine-tuning-openais-gpt3-model-fd9cb06517f6)\n",
    "\n",
    "* [datacamp: Fine-Tuning GPT-3 Using the OpenAI API and Python](https://www.datacamp.com/tutorial/fine-tuning-gpt-3-using-the-open-ai-api-and-python)\n",
    "\n",
    "\n",
    "Também: nomes personalizados do `fine-tuning`:\n",
    "\n",
    "$ openai api fine_tunes.create -t test.jsonl -m ada --suffix \"custom model name\"    ----> criará: `ada:ft-your-org:custom-model-name-2022-02-15-04-21-04`\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<font color=\"orange\">Neste Notebook vamos realizar o `Fine-Tuning` de um Classificador `ADA` para distinguir entre os dois esportes: `Baseball` e `Hockey`.</font>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# <font color=\"red\">Importação e exploração de Dados</font>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "O conjunto de dados do `newsgroup` pode ser carregado usando sklearn. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.datasets import fetch_20newsgroups\n",
    "import pandas as pd\n",
    "import openai\n",
    "\n",
    "\n",
    "categories = ['rec.sport.baseball', 'rec.sport.hockey']\n",
    "sports_dataset = fetch_20newsgroups(subset='train', shuffle=True, random_state=42, categories=categories)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      ".. _20newsgroups_dataset:\n",
      "\n",
      "The 20 newsgroups text dataset\n",
      "------------------------------\n",
      "\n",
      "The 20 newsgroups dataset comprises around 18000 newsgroups posts on\n",
      "20 topics split in two subsets: one for training (or development)\n",
      "and the other one for testing (or for performance evaluation). The split\n",
      "between the train and test set is based upon a messages posted before\n",
      "and after a specific date.\n",
      "\n",
      "This module contains two loaders. The first one,\n",
      ":func:`sklearn.datasets.fetch_20newsgroups`,\n",
      "returns a list of the raw texts that can be fed to text feature\n",
      "extractors such as :class:`~sklearn.feature_extraction.text.CountVectorizer`\n",
      "with custom parameters so as to extract feature vectors.\n",
      "The second one, :func:`sklearn.datasets.fetch_20newsgroups_vectorized`,\n",
      "returns ready-to-use features, i.e., it is not necessary to use a feature\n",
      "extractor.\n",
      "\n",
      "**Data Set Characteristics:**\n",
      "\n",
      "    =================   ==========\n",
      "    Classes                     20\n",
      "    Samples total            18846\n",
      "    Dimensionality               1\n",
      "    Features                  text\n",
      "    =================   ==========\n",
      "\n",
      "|details-start|\n",
      "**Usage**\n",
      "|details-split|\n",
      "\n",
      "The :func:`sklearn.datasets.fetch_20newsgroups` function is a data\n",
      "fetching / caching functions that downloads the data archive from\n",
      "the original `20 newsgroups website`_, extracts the archive contents\n",
      "in the ``~/scikit_learn_data/20news_home`` folder and calls the\n",
      ":func:`sklearn.datasets.load_files` on either the training or\n",
      "testing set folder, or both of them::\n",
      "\n",
      "  >>> from sklearn.datasets import fetch_20newsgroups\n",
      "  >>> newsgroups_train = fetch_20newsgroups(subset='train')\n",
      "\n",
      "  >>> from pprint import pprint\n",
      "  >>> pprint(list(newsgroups_train.target_names))\n",
      "  ['alt.atheism',\n",
      "   'comp.graphics',\n",
      "   'comp.os.ms-windows.misc',\n",
      "   'comp.sys.ibm.pc.hardware',\n",
      "   'comp.sys.mac.hardware',\n",
      "   'comp.windows.x',\n",
      "   'misc.forsale',\n",
      "   'rec.autos',\n",
      "   'rec.motorcycles',\n",
      "   'rec.sport.baseball',\n",
      "   'rec.sport.hockey',\n",
      "   'sci.crypt',\n",
      "   'sci.electronics',\n",
      "   'sci.med',\n",
      "   'sci.space',\n",
      "   'soc.religion.christian',\n",
      "   'talk.politics.guns',\n",
      "   'talk.politics.mideast',\n",
      "   'talk.politics.misc',\n",
      "   'talk.religion.misc']\n",
      "\n",
      "The real data lies in the ``filenames`` and ``target`` attributes. The target\n",
      "attribute is the integer index of the category::\n",
      "\n",
      "  >>> newsgroups_train.filenames.shape\n",
      "  (11314,)\n",
      "  >>> newsgroups_train.target.shape\n",
      "  (11314,)\n",
      "  >>> newsgroups_train.target[:10]\n",
      "  array([ 7,  4,  4,  1, 14, 16, 13,  3,  2,  4])\n",
      "\n",
      "It is possible to load only a sub-selection of the categories by passing the\n",
      "list of the categories to load to the\n",
      ":func:`sklearn.datasets.fetch_20newsgroups` function::\n",
      "\n",
      "  >>> cats = ['alt.atheism', 'sci.space']\n",
      "  >>> newsgroups_train = fetch_20newsgroups(subset='train', categories=cats)\n",
      "\n",
      "  >>> list(newsgroups_train.target_names)\n",
      "  ['alt.atheism', 'sci.space']\n",
      "  >>> newsgroups_train.filenames.shape\n",
      "  (1073,)\n",
      "  >>> newsgroups_train.target.shape\n",
      "  (1073,)\n",
      "  >>> newsgroups_train.target[:10]\n",
      "  array([0, 1, 1, 1, 0, 1, 1, 0, 0, 0])\n",
      "\n",
      "|details-end|\n",
      "\n",
      "|details-start|\n",
      "**Converting text to vectors**\n",
      "|details-split|\n",
      "\n",
      "In order to feed predictive or clustering models with the text data,\n",
      "one first need to turn the text into vectors of numerical values suitable\n",
      "for statistical analysis. This can be achieved with the utilities of the\n",
      "``sklearn.feature_extraction.text`` as demonstrated in the following\n",
      "example that extract `TF-IDF`_ vectors of unigram tokens\n",
      "from a subset of 20news::\n",
      "\n",
      "  >>> from sklearn.feature_extraction.text import TfidfVectorizer\n",
      "  >>> categories = ['alt.atheism', 'talk.religion.misc',\n",
      "  ...               'comp.graphics', 'sci.space']\n",
      "  >>> newsgroups_train = fetch_20newsgroups(subset='train',\n",
      "  ...                                       categories=categories)\n",
      "  >>> vectorizer = TfidfVectorizer()\n",
      "  >>> vectors = vectorizer.fit_transform(newsgroups_train.data)\n",
      "  >>> vectors.shape\n",
      "  (2034, 34118)\n",
      "\n",
      "The extracted TF-IDF vectors are very sparse, with an average of 159 non-zero\n",
      "components by sample in a more than 30000-dimensional space\n",
      "(less than .5% non-zero features)::\n",
      "\n",
      "  >>> vectors.nnz / float(vectors.shape[0])\n",
      "  159.01327...\n",
      "\n",
      ":func:`sklearn.datasets.fetch_20newsgroups_vectorized` is a function which\n",
      "returns ready-to-use token counts features instead of file names.\n",
      "\n",
      ".. _`20 newsgroups website`: http://people.csail.mit.edu/jrennie/20Newsgroups/\n",
      ".. _`TF-IDF`: https://en.wikipedia.org/wiki/Tf-idf\n",
      "\n",
      "|details-end|\n",
      "\n",
      "|details-start|\n",
      "**Filtering text for more realistic training**\n",
      "|details-split|\n",
      "\n",
      "It is easy for a classifier to overfit on particular things that appear in the\n",
      "20 Newsgroups data, such as newsgroup headers. Many classifiers achieve very\n",
      "high F-scores, but their results would not generalize to other documents that\n",
      "aren't from this window of time.\n",
      "\n",
      "For example, let's look at the results of a multinomial Naive Bayes classifier,\n",
      "which is fast to train and achieves a decent F-score::\n",
      "\n",
      "  >>> from sklearn.naive_bayes import MultinomialNB\n",
      "  >>> from sklearn import metrics\n",
      "  >>> newsgroups_test = fetch_20newsgroups(subset='test',\n",
      "  ...                                      categories=categories)\n",
      "  >>> vectors_test = vectorizer.transform(newsgroups_test.data)\n",
      "  >>> clf = MultinomialNB(alpha=.01)\n",
      "  >>> clf.fit(vectors, newsgroups_train.target)\n",
      "  MultinomialNB(alpha=0.01, class_prior=None, fit_prior=True)\n",
      "\n",
      "  >>> pred = clf.predict(vectors_test)\n",
      "  >>> metrics.f1_score(newsgroups_test.target, pred, average='macro')\n",
      "  0.88213...\n",
      "\n",
      "(The example :ref:`sphx_glr_auto_examples_text_plot_document_classification_20newsgroups.py` shuffles\n",
      "the training and test data, instead of segmenting by time, and in that case\n",
      "multinomial Naive Bayes gets a much higher F-score of 0.88. Are you suspicious\n",
      "yet of what's going on inside this classifier?)\n",
      "\n",
      "Let's take a look at what the most informative features are:\n",
      "\n",
      "  >>> import numpy as np\n",
      "  >>> def show_top10(classifier, vectorizer, categories):\n",
      "  ...     feature_names = vectorizer.get_feature_names_out()\n",
      "  ...     for i, category in enumerate(categories):\n",
      "  ...         top10 = np.argsort(classifier.coef_[i])[-10:]\n",
      "  ...         print(\"%s: %s\" % (category, \" \".join(feature_names[top10])))\n",
      "  ...\n",
      "  >>> show_top10(clf, vectorizer, newsgroups_train.target_names)\n",
      "  alt.atheism: edu it and in you that is of to the\n",
      "  comp.graphics: edu in graphics it is for and of to the\n",
      "  sci.space: edu it that is in and space to of the\n",
      "  talk.religion.misc: not it you in is that and to of the\n",
      "\n",
      "\n",
      "You can now see many things that these features have overfit to:\n",
      "\n",
      "- Almost every group is distinguished by whether headers such as\n",
      "  ``NNTP-Posting-Host:`` and ``Distribution:`` appear more or less often.\n",
      "- Another significant feature involves whether the sender is affiliated with\n",
      "  a university, as indicated either by their headers or their signature.\n",
      "- The word \"article\" is a significant feature, based on how often people quote\n",
      "  previous posts like this: \"In article [article ID], [name] <[e-mail address]>\n",
      "  wrote:\"\n",
      "- Other features match the names and e-mail addresses of particular people who\n",
      "  were posting at the time.\n",
      "\n",
      "With such an abundance of clues that distinguish newsgroups, the classifiers\n",
      "barely have to identify topics from text at all, and they all perform at the\n",
      "same high level.\n",
      "\n",
      "For this reason, the functions that load 20 Newsgroups data provide a\n",
      "parameter called **remove**, telling it what kinds of information to strip out\n",
      "of each file. **remove** should be a tuple containing any subset of\n",
      "``('headers', 'footers', 'quotes')``, telling it to remove headers, signature\n",
      "blocks, and quotation blocks respectively.\n",
      "\n",
      "  >>> newsgroups_test = fetch_20newsgroups(subset='test',\n",
      "  ...                                      remove=('headers', 'footers', 'quotes'),\n",
      "  ...                                      categories=categories)\n",
      "  >>> vectors_test = vectorizer.transform(newsgroups_test.data)\n",
      "  >>> pred = clf.predict(vectors_test)\n",
      "  >>> metrics.f1_score(pred, newsgroups_test.target, average='macro')\n",
      "  0.77310...\n",
      "\n",
      "This classifier lost over a lot of its F-score, just because we removed\n",
      "metadata that has little to do with topic classification.\n",
      "It loses even more if we also strip this metadata from the training data:\n",
      "\n",
      "  >>> newsgroups_train = fetch_20newsgroups(subset='train',\n",
      "  ...                                       remove=('headers', 'footers', 'quotes'),\n",
      "  ...                                       categories=categories)\n",
      "  >>> vectors = vectorizer.fit_transform(newsgroups_train.data)\n",
      "  >>> clf = MultinomialNB(alpha=.01)\n",
      "  >>> clf.fit(vectors, newsgroups_train.target)\n",
      "  MultinomialNB(alpha=0.01, class_prior=None, fit_prior=True)\n",
      "\n",
      "  >>> vectors_test = vectorizer.transform(newsgroups_test.data)\n",
      "  >>> pred = clf.predict(vectors_test)\n",
      "  >>> metrics.f1_score(newsgroups_test.target, pred, average='macro')\n",
      "  0.76995...\n",
      "\n",
      "Some other classifiers cope better with this harder version of the task. Try the\n",
      ":ref:`sphx_glr_auto_examples_model_selection_plot_grid_search_text_feature_extraction.py`\n",
      "example with and without the `remove` option to compare the results.\n",
      "|details-end|\n",
      "\n",
      ".. topic:: Data Considerations\n",
      "\n",
      "  The Cleveland Indians is a major league baseball team based in Cleveland,\n",
      "  Ohio, USA. In December 2020, it was reported that \"After several months of\n",
      "  discussion sparked by the death of George Floyd and a national reckoning over\n",
      "  race and colonialism, the Cleveland Indians have decided to change their\n",
      "  name.\" Team owner Paul Dolan \"did make it clear that the team will not make\n",
      "  its informal nickname -- the Tribe -- its new team name.\" \"It's not going to\n",
      "  be a half-step away from the Indians,\" Dolan said.\"We will not have a Native\n",
      "  American-themed name.\"\n",
      "\n",
      "  https://www.mlb.com/news/cleveland-indians-team-name-change\n",
      "\n",
      ".. topic:: Recommendation\n",
      "\n",
      "  - When evaluating text classifiers on the 20 Newsgroups data, you\n",
      "    should strip newsgroup-related metadata. In scikit-learn, you can do this\n",
      "    by setting ``remove=('headers', 'footers', 'quotes')``. The F-score will be\n",
      "    lower because it is more realistic.\n",
      "  - This text dataset contains data which may be inappropriate for certain NLP\n",
      "    applications. An example is listed in the \"Data Considerations\" section\n",
      "    above. The challenge with using current text datasets in NLP for tasks such\n",
      "    as sentence completion, clustering, and other applications is that text\n",
      "    that is culturally biased and inflammatory will propagate biases. This\n",
      "    should be taken into consideration when using the dataset, reviewing the\n",
      "    output, and the bias should be documented.\n",
      "\n",
      ".. topic:: Examples\n",
      "\n",
      "   * :ref:`sphx_glr_auto_examples_model_selection_plot_grid_search_text_feature_extraction.py`\n",
      "\n",
      "   * :ref:`sphx_glr_auto_examples_text_plot_document_classification_20newsgroups.py`\n",
      "\n",
      "   * :ref:`sphx_glr_auto_examples_text_plot_hashing_vs_dict_vectorizer.py`\n",
      "\n",
      "   * :ref:`sphx_glr_auto_examples_text_plot_document_clustering.py`\n",
      "\n"
     ]
    }
   ],
   "source": [
    "print(sports_dataset['DESCR'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['rec.sport.baseball', 'rec.sport.hockey']\n"
     ]
    }
   ],
   "source": [
    "print(sports_dataset['target_names'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "From: dougb@comm.mot.com (Doug Bank)\n",
      "Subject: Re: Info needed for Cleveland tickets\n",
      "Reply-To: dougb@ecs.comm.mot.com\n",
      "Organization: Motorola Land Mobile Products Sector\n",
      "Distribution: usa\n",
      "Nntp-Posting-Host: 145.1.146.35\n",
      "Lines: 17\n",
      "\n",
      "In article <1993Apr1.234031.4950@leland.Stanford.EDU>, bohnert@leland.Stanford.EDU (matthew bohnert) writes:\n",
      "\n",
      "|> I'm going to be in Cleveland Thursday, April 15 to Sunday, April 18.\n",
      "|> Does anybody know if the Tribe will be in town on those dates, and\n",
      "|> if so, who're they playing and if tickets are available?\n",
      "\n",
      "The tribe will be in town from April 16 to the 19th.\n",
      "There are ALWAYS tickets available! (Though they are playing Toronto,\n",
      "and many Toronto fans make the trip to Cleveland as it is easier to\n",
      "get tickets in Cleveland than in Toronto.  Either way, I seriously\n",
      "doubt they will sell out until the end of the season.)\n",
      "\n",
      "-- \n",
      "Doug Bank                       Private Systems Division\n",
      "dougb@ecs.comm.mot.com          Motorola Communications Sector\n",
      "dougb@nwu.edu                   Schaumburg, Illinois\n",
      "dougb@casbah.acns.nwu.edu       708-576-8207                    \n",
      "\n"
     ]
    }
   ],
   "source": [
    "print(sports_dataset['data'][0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'rec.sport.baseball'"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sports_dataset.target_names[sports_dataset['target'][0]]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Total examples: 1197, Baseball examples: 597, Hockey examples: 600\n"
     ]
    }
   ],
   "source": [
    "len_all, len_baseball, len_hockey = len(sports_dataset.data), len([e for e in sports_dataset.target if e == 0]), len([e for e in sports_dataset.target if e == 1])\n",
    "\n",
    "\n",
    "print(f\"Total examples: {len_all}, Baseball examples: {len_baseball}, Hockey examples: {len_hockey}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# <font color=\"red\">Preparação de dados</font>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<font color=\"orange\">Transformamos o conjunto de dados em um dataframe do `pandas`, com uma coluna para `prompt` e `completion`. O `prompt` contém o `e-mail` da lista de discussão e a `completion` é o nome do esporte, seja `hockey` ou `baseball`. Apenas para fins de demonstração e velocidade de ``Fine-Tuning, tomamos apenas `300` exemplos. `Em um caso de uso real, quanto mais exemplos melhor será o desempenho.`</font>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>prompt</th>\n",
       "      <th>completion</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>From: dougb@comm.mot.com (Doug Bank)\\nSubject:...</td>\n",
       "      <td>baseball</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>From: gld@cunixb.cc.columbia.edu (Gary L Dare)...</td>\n",
       "      <td>hockey</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>From: rudy@netcom.com (Rudy Wade)\\nSubject: Re...</td>\n",
       "      <td>baseball</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>From: monack@helium.gas.uug.arizona.edu (david...</td>\n",
       "      <td>hockey</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Subject: Let it be Known\\nFrom: &lt;ISSBTL@BYUVM....</td>\n",
       "      <td>baseball</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                              prompt completion\n",
       "0  From: dougb@comm.mot.com (Doug Bank)\\nSubject:...   baseball\n",
       "1  From: gld@cunixb.cc.columbia.edu (Gary L Dare)...     hockey\n",
       "2  From: rudy@netcom.com (Rudy Wade)\\nSubject: Re...   baseball\n",
       "3  From: monack@helium.gas.uug.arizona.edu (david...     hockey\n",
       "4  Subject: Let it be Known\\nFrom: <ISSBTL@BYUVM....   baseball"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "\n",
    "\n",
    "labels = [sports_dataset.target_names[x].split('.')[-1] for x in sports_dataset['target']]\n",
    "\n",
    "texts = [text.strip() for text in sports_dataset['data']]\n",
    "\n",
    "\n",
    "df = pd.DataFrame(zip(texts, labels), columns = ['prompt','completion']) #[:300]\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(1197, 2)"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<font color=\"orange\">Tanto o `baseball` quanto o `hockey` são tokens únicos. Salvamos o conjunto de dados como um arquivo `jsonl`.</font>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "# orient='records' --> Significa que cada linha do DataFrame será convertida em um registro JSON separado no arquivo. Cada registro JSON conterá os dados de uma linha do DataFrame.\n",
    "# lines=True --> Este argumento indica que os registros JSON devem ser escritos em linhas separadas no arquivo JSONL.\n",
    "df.to_json(\"sport2.jsonl\", orient='records', lines=True)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# <font color=\"red\">Tool de preparação de dados</font>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<font color=\"orange\">Agora podemos usar uma `tool de preparação de dados` que irá sugerir algumas melhorias em nosso conjunto de dados antes do `Fine-Tuning`. Antes de lançar a Tool, atualizamos a biblioteca `openai` para garantir que estamos usando a Tool de preparação de dados mais recente. Além disso, especificamos `-q` que aceita automaticamente todas as sugestões.</font>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: openai in ./venv_Classification/lib/python3.10/site-packages (0.28.1)\n",
      "Requirement already satisfied: aiohttp in ./venv_Classification/lib/python3.10/site-packages (from openai) (3.8.5)\n",
      "Requirement already satisfied: tqdm in ./venv_Classification/lib/python3.10/site-packages (from openai) (4.66.1)\n",
      "Requirement already satisfied: requests>=2.20 in ./venv_Classification/lib/python3.10/site-packages (from openai) (2.31.0)\n",
      "Requirement already satisfied: certifi>=2017.4.17 in ./venv_Classification/lib/python3.10/site-packages (from requests>=2.20->openai) (2023.7.22)\n",
      "Requirement already satisfied: charset-normalizer<4,>=2 in ./venv_Classification/lib/python3.10/site-packages (from requests>=2.20->openai) (3.3.0)\n",
      "Requirement already satisfied: idna<4,>=2.5 in ./venv_Classification/lib/python3.10/site-packages (from requests>=2.20->openai) (3.4)\n",
      "Requirement already satisfied: urllib3<3,>=1.21.1 in ./venv_Classification/lib/python3.10/site-packages (from requests>=2.20->openai) (2.0.6)\n",
      "Requirement already satisfied: attrs>=17.3.0 in ./venv_Classification/lib/python3.10/site-packages (from aiohttp->openai) (23.1.0)\n",
      "Requirement already satisfied: async-timeout<5.0,>=4.0.0a3 in ./venv_Classification/lib/python3.10/site-packages (from aiohttp->openai) (4.0.3)\n",
      "Requirement already satisfied: aiosignal>=1.1.2 in ./venv_Classification/lib/python3.10/site-packages (from aiohttp->openai) (1.3.1)\n",
      "Requirement already satisfied: frozenlist>=1.1.1 in ./venv_Classification/lib/python3.10/site-packages (from aiohttp->openai) (1.4.0)\n",
      "Requirement already satisfied: yarl<2.0,>=1.0 in ./venv_Classification/lib/python3.10/site-packages (from aiohttp->openai) (1.9.2)\n",
      "Requirement already satisfied: multidict<7.0,>=4.5 in ./venv_Classification/lib/python3.10/site-packages (from aiohttp->openai) (6.0.4)\n",
      "Note: you may need to restart the kernel to use updated packages.\n"
     ]
    }
   ],
   "source": [
    "%pip install --upgrade openai"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Analyzing...\n",
      "\n",
      "- Your file contains 1197 prompt-completion pairs\n",
      "- Based on your data it seems like you're trying to fine-tune a model for classification\n",
      "- For classification, we recommend you try one of the faster and cheaper models, such as `ada`\n",
      "- For classification, you can estimate the expected model performance by keeping a held out dataset, which is not used for training\n",
      "- There are 11 examples that are very long. These are rows: [134, 200, 281, 320, 404, 595, 704, 838, 1113, 1139, 1174]\n",
      "For conditional generation, and for classification the examples shouldn't be longer than 2048 tokens.\n",
      "- Your data does not contain a common separator at the end of your prompts. Having a separator string appended to the end of the prompt makes it clearer to the fine-tuned model where the completion should begin. See https://platform.openai.com/docs/guides/fine-tuning/preparing-your-dataset for more detail and examples. If you intend to do open-ended generation, then you should leave the prompts empty\n",
      "- The completion should start with a whitespace character (` `). This tends to produce better results due to the tokenization we use. See https://platform.openai.com/docs/guides/fine-tuning/preparing-your-dataset for more details\n",
      "\n",
      "Based on the analysis we will perform the following actions:\n",
      "- [Recommended] Remove 11 long examples [Y/n]: Y\n",
      "- [Recommended] Add a suffix separator `\\n\\n###\\n\\n` to all prompts [Y/n]: Y\n",
      "- [Recommended] Add a whitespace character to the beginning of the completion [Y/n]: Y\n",
      "- [Recommended] Would you like to split into training and validation set? [Y/n]: Y\n",
      "\n",
      "\n",
      "Your data will be written to a new JSONL file. Proceed [Y/n]: Y\n",
      "\n",
      "Wrote modified files to `sport2_prepared_train.jsonl` and `sport2_prepared_valid.jsonl`\n",
      "Feel free to take a look!\n",
      "\n",
      "Now use that file when fine-tuning:\n",
      "> openai api fine_tunes.create -t \"sport2_prepared_train.jsonl\" -v \"sport2_prepared_valid.jsonl\" --compute_classification_metrics --classification_positive_class \" baseball\"\n",
      "\n",
      "After you’ve fine-tuned a model, remember that your prompt has to end with the indicator string `\\n\\n###\\n\\n` for the model to start generating completions, rather than continuing with the prompt.\n",
      "Once your model starts training, it'll approximately take 30.8 minutes to train a `curie` model, and less for `ada` and `babbage`. Queue will approximately take half an hour per job ahead of you.\n"
     ]
    }
   ],
   "source": [
    "# Ver também: openai tools fine_tunes.prepare_data -f qa.txt\n",
    "\n",
    "!openai tools fine_tunes.prepare_data -f sport2.jsonl -q"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<font color=\"orange\">A ferramenta sugere algumas melhorias no conjunto de dados e divide o conjunto de dados em conjunto de `Treinamento` e `Validação`.\n",
    "\n",
    "Um `suffix` entre um `prompt` e o `completion` é necessário para informar ao modelo que o texto de entrada foi interrompido e que agora ele precisa prever a `classe`. Como usamos o mesmo separador em cada exemplo, o modelo é capaz de aprender que se destina a prever o `baseball` or `hockey` seguindo o separador. Um `prefix` de espaço em branco nas `completions` é útil, pois a maioria dos tokens de palavras são tokenizados com um `prefix` de espaço. A ferramenta (`Tool`) também reconheceu que esta é provavelmente uma tarefa de `classificação`, por isso sugeriu dividir o conjunto de dados em conjuntos de dados de Treinamento e Validação. Isso nos permitirá medir facilmente o desempenho esperado em novos dados.</font>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# <font color=\"red\">Fine-Tuning</font>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<font color=\"orange\">A ferramenta sugere que executemos o seguinte comando para treinar o conjunto de dados. Como esta é uma `task de classificação`, gostaríamos de saber qual é o desempenho de `generalização` no conjunto de `validação` fornecido para nosso caso de uso de classificação. A ferramenta sugere adicionar `--compute_classification_metrics --classification_positive_class \"baseball\"` para calcular as métricas de classificação.\n",
    "\n",
    "Podemos simplesmente copiar o comando sugerido da ferramenta CLI. Adicionamos especificamente `-m ada` para o `Fine-Tuning` do modelo `ada` mais barato e mais rápido, que geralmente é comparável em desempenho a modelos mais lentos e mais caros em casos de uso de classificação.</font>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training File ID: file-iGfQPTiV6hwy5fV2J7Y4YQpW\n"
     ]
    }
   ],
   "source": [
    "# Data_train:\n",
    "Data_train = openai.File.create(file=open(\"/home/eddygiusepe/1_Eddy_Giusepe/6_REPO_HuggingFace/15_Binary_classification_fine-tuning/sport2_prepared_train.jsonl\", \"rb\"),\n",
    "                                purpose='fine-tune'\n",
    "                               )\n",
    "\n",
    "training_file_id = Data_train[\"id\"]\n",
    "print(f\"Training File ID: {training_file_id}\")\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Validation File ID: file-9Ur6ctNs3Pfyfywql1lp7PO4\n"
     ]
    }
   ],
   "source": [
    "# Data_valid:\n",
    "Data_train = openai.File.create(file=open(\"/home/eddygiusepe/1_Eddy_Giusepe/6_REPO_HuggingFace/15_Binary_classification_fine-tuning/sport2_prepared_valid.jsonl\", \"rb\"),\n",
    "                                purpose='fine-tune'\n",
    "                               )\n",
    "\n",
    "validation_file_id = Data_train[\"id\"]\n",
    "print(f\"Validation File ID: {validation_file_id}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Excluir os arquivos:\n",
    "\n",
    "openai.File.delete(\"file-WyKeDhoOpOz6DryMemOimyvW\")\n",
    "\n",
    "openai.File.delete(\"file-mzyk3q314rgKyQxQnr5oBx0j\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [],
   "source": [
    "#DEPRECATED ---> !openai api fine_tunes.create -t \"sport2_prepared_train.jsonl\" -v \"sport2_prepared_valid.jsonl\" --compute_classification_metrics --classification_positive_class \" baseball\" -m ada\n",
    "\n",
    "# Descomentar para treinar: (CUIDADO NÃO EXECUTAR VARIAS VEZES NÃO❗❗❗)\n",
    "\n",
    "Fine_Tuning =openai.FineTuningJob.create(\n",
    "    \t\t\t\t\t\t\t\t\ttraining_file=training_file_id,\n",
    "                            \t\t\tvalidation_file=validation_file_id,\n",
    "                            \t\t\tmodel=\"davinci-002\",\n",
    "                            \t\t\thyperparameters={\"n_epochs\":4},\n",
    "                                        suffix=\"Eddy_BinaryClass\")\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<FineTuningJob fine_tuning.job id=ftjob-ytgarjLNymFEcrKYtDUup2yL at 0x7f7ff49ff600> JSON: {\n",
       "  \"object\": \"fine_tuning.job\",\n",
       "  \"id\": \"ftjob-ytgarjLNymFEcrKYtDUup2yL\",\n",
       "  \"model\": \"davinci-002\",\n",
       "  \"created_at\": 1696536241,\n",
       "  \"finished_at\": null,\n",
       "  \"fine_tuned_model\": null,\n",
       "  \"organization_id\": \"org-cSA8PDGrKC1wEWr130vOKCM0\",\n",
       "  \"result_files\": [],\n",
       "  \"status\": \"validating_files\",\n",
       "  \"validation_file\": \"file-9Ur6ctNs3Pfyfywql1lp7PO4\",\n",
       "  \"training_file\": \"file-iGfQPTiV6hwy5fV2J7Y4YQpW\",\n",
       "  \"hyperparameters\": {\n",
       "    \"n_epochs\": 4\n",
       "  },\n",
       "  \"trained_tokens\": null,\n",
       "  \"error\": null\n",
       "}"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Fine_Tuning"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'ftjob-ytgarjLNymFEcrKYtDUup2yL'"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Fine_Tuning_id = Fine_Tuning[\"id\"]\n",
    "Fine_Tuning_id"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Para cancelar o Treinamento:\n",
    "\n",
    "openai.FineTuningJob.cancel(\"ftjob-MYm8Gj37kEI3ynM6tHimWTGA\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "O modelo é treinado com sucesso em cerca de dez minutos. Podemos ver que o nome do modelo é `ada:ft-openai-2021-07-30-12-26-20`, que podemos usar para fazer inferências."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# <font color=\"red\">Resultados e desempenho esperado do Modelo</font>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<font color=\"orange\">Agora podemos fazer `download` do arquivo de resultados para observar o desempenho esperado em um conjunto de `validação` retido.</font>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<FineTuningJob fine_tuning.job id=ftjob-ytgarjLNymFEcrKYtDUup2yL at 0x7f7ff4ac1210> JSON: {\n",
       "  \"object\": \"fine_tuning.job\",\n",
       "  \"id\": \"ftjob-ytgarjLNymFEcrKYtDUup2yL\",\n",
       "  \"model\": \"davinci-002\",\n",
       "  \"created_at\": 1696536241,\n",
       "  \"finished_at\": 1696536406,\n",
       "  \"fine_tuned_model\": \"ft:davinci-002:personal:eddy-binaryclass:86OuN8we\",\n",
       "  \"organization_id\": \"org-cSA8PDGrKC1wEWr130vOKCM0\",\n",
       "  \"result_files\": [\n",
       "    \"file-3ZqS55VbHhldKCTQNzIUQoFK\"\n",
       "  ],\n",
       "  \"status\": \"succeeded\",\n",
       "  \"validation_file\": \"file-9Ur6ctNs3Pfyfywql1lp7PO4\",\n",
       "  \"training_file\": \"file-iGfQPTiV6hwy5fV2J7Y4YQpW\",\n",
       "  \"hyperparameters\": {\n",
       "    \"n_epochs\": 4\n",
       "  },\n",
       "  \"trained_tokens\": 41764,\n",
       "  \"error\": null\n",
       "}"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Usamos o seguinte comando ---> https://platform.openai.com/docs/api-reference/fine-tuning/retrieve\n",
    "\n",
    "openai.FineTuningJob.retrieve(\"ftjob-ytgarjLNymFEcrKYtDUup2yL\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<File file id=file-3ZqS55VbHhldKCTQNzIUQoFK at 0x7f7ff4aecc20> JSON: {\n",
       "  \"object\": \"file\",\n",
       "  \"id\": \"file-3ZqS55VbHhldKCTQNzIUQoFK\",\n",
       "  \"purpose\": \"fine-tune-results\",\n",
       "  \"filename\": \"step_metrics.csv\",\n",
       "  \"bytes\": 1811,\n",
       "  \"created_at\": 1696536409,\n",
       "  \"status\": \"processed\",\n",
       "  \"status_details\": null\n",
       "}"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "xxx= openai.File.retrieve(\"file-3ZqS55VbHhldKCTQNzIUQoFK\")\n",
    "\n",
    "xxx"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "b'step,train_loss,train_accuracy,valid_loss,valid_mean_token_accuracy\\n1,15.81906,0.0,14.67002,0.0\\n2,13.07276,0.0,15.50881,0.0\\n3,13.87567,0.0,16.25,0.0\\n4,13.3373,0.0,12.77298,0.0\\n5,14.30848,0.0,12.6235,0.0\\n6,13.32132,0.0,11.60394,0.0\\n7,12.9074,0.0,11.67611,0.0\\n8,12.5918,0.0,9.70959,0.0\\n9,11.56537,0.0,8.65894,0.0\\n10,8.10063,0.0,15.49833,0.0\\n11,4.67463,0.0,4.4466,0.0\\n12,5.78812,0.0,3.59255,0.0\\n13,4.91947,0.0,1.81759,0.0\\n14,1.67461,0.0,1.37271,1.0\\n15,0.45382,1.0,0.02368,1.0\\n16,0.89798,1.0,0.05735,1.0\\n17,0.03383,1.0,0.03698,1.0\\n18,0.20682,1.0,0.00484,1.0\\n19,0.03163,1.0,0.00079,1.0\\n20,0.00083,1.0,0.00017,1.0\\n21,0.00013,1.0,4e-05,1.0\\n22,1e-05,1.0,1e-05,1.0\\n23,1e-05,1.0,0.0,1.0\\n24,0.0,1.0,0.0,1.0\\n25,0.0,1.0,2e-05,1.0\\n26,0.0,1.0,0.0,1.0\\n27,0.0,1.0,0.0,1.0\\n28,0.0,1.0,0.0,1.0\\n29,0.0,1.0,-0.0,1.0\\n30,0.0,1.0,0.99869,0.0\\n31,0.0,1.0,0.0,1.0\\n32,0.0,1.0,-0.0,1.0\\n33,0.0,1.0,-0.0,1.0\\n34,0.0,1.0,-0.0,1.0\\n35,0.0,1.0,-0.0,1.0\\n36,0.0,1.0,-0.0,1.0\\n37,0.0,1.0,-0.0,1.0\\n38,0.0,1.0,-0.0,1.0\\n39,0.0,1.0,-0.0,1.0\\n40,0.0,1.0,-0.0,1.0\\n41,0.0,1.0,-0.0,1.0\\n42,0.0,1.0,-0.0,1.0\\n43,0.0,1.0,-0.0,1.0\\n44,0.0,1.0,-0.0,1.0\\n45,0.0,1.0,0.0,1.0\\n46,0.0,1.0,-0.0,1.0\\n47,0.0,1.0,-0.0,1.0\\n48,1e-05,1.0,-0.0,1.0\\n49,0.0,1.0,0.0,1.0\\n50,0.0,1.0,9.84829,0.0\\n51,0.0,1.0,-0.0,1.0\\n52,0.0,1.0,-0.0,1.0\\n53,0.0,1.0,0.0,1.0\\n54,0.0,1.0,0.0,1.0\\n55,0.0,1.0,-0.0,1.0\\n56,0.0,1.0,0.0,1.0\\n57,0.0,1.0,-0.0,1.0\\n58,0.0,1.0,-0.0,1.0\\n59,0.0,1.0,-0.0,1.0\\n60,0.0,1.0,-0.0,1.0\\n61,0.0,1.0,0.0,1.0\\n62,0.0,1.0,-0.0,1.0\\n63,0.0,1.0,0.0,1.0\\n64,0.0,1.0,-0.0,1.0\\n65,0.0,1.0,-0.0,1.0\\n66,0.0,1.0,-0.0,1.0\\n67,0.0,1.0,-0.0,1.0\\n68,0.0,1.0,-0.0,1.0\\n69,0.0,1.0,0.0,1.0\\n70,0.0,1.0,9.45719,0.0\\n71,0.0,1.0,-0.0,1.0\\n72,0.0,1.0,-0.0,1.0\\n73,0.0,1.0,0.0,1.0\\n74,0.0,1.0,3e-05,1.0\\n75,0.0,1.0,0.0,1.0\\n76,0.0,1.0,0.0,1.0\\n77,0.0,1.0,0.0,1.0\\n78,0.0,1.0,0.0,1.0\\n79,0.0,1.0,-0.0,1.0\\n80,0.0,1.0,0.0,1.0\\n'"
      ]
     },
     "execution_count": 62,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# !openai api fine_tunes.results -i ft-vAbAQIPmy69zOyssgt5zI3VE > result.csv\n",
    "\n",
    "# Aqui usamos para recuperar nosso RESULTADOS --->  https://platform.openai.com/docs/api-reference/files/retrieve-contents\n",
    "content = openai.File.download(\"file-3ZqS55VbHhldKCTQNzIUQoFK\")\n",
    "\n",
    "content"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'step,train_loss,train_accuracy,valid_loss,valid_mean_token_accuracy\\n1,15.81906,0.0,14.67002,0.0\\n2,13.07276,0.0,15.50881,0.0\\n3,13.87567,0.0,16.25,0.0\\n4,13.3373,0.0,12.77298,0.0\\n5,14.30848,0.0,12.6235,0.0\\n6,13.32132,0.0,11.60394,0.0\\n7,12.9074,0.0,11.67611,0.0\\n8,12.5918,0.0,9.70959,0.0\\n9,11.56537,0.0,8.65894,0.0\\n10,8.10063,0.0,15.49833,0.0\\n11,4.67463,0.0,4.4466,0.0\\n12,5.78812,0.0,3.59255,0.0\\n13,4.91947,0.0,1.81759,0.0\\n14,1.67461,0.0,1.37271,1.0\\n15,0.45382,1.0,0.02368,1.0\\n16,0.89798,1.0,0.05735,1.0\\n17,0.03383,1.0,0.03698,1.0\\n18,0.20682,1.0,0.00484,1.0\\n19,0.03163,1.0,0.00079,1.0\\n20,0.00083,1.0,0.00017,1.0\\n21,0.00013,1.0,4e-05,1.0\\n22,1e-05,1.0,1e-05,1.0\\n23,1e-05,1.0,0.0,1.0\\n24,0.0,1.0,0.0,1.0\\n25,0.0,1.0,2e-05,1.0\\n26,0.0,1.0,0.0,1.0\\n27,0.0,1.0,0.0,1.0\\n28,0.0,1.0,0.0,1.0\\n29,0.0,1.0,-0.0,1.0\\n30,0.0,1.0,0.99869,0.0\\n31,0.0,1.0,0.0,1.0\\n32,0.0,1.0,-0.0,1.0\\n33,0.0,1.0,-0.0,1.0\\n34,0.0,1.0,-0.0,1.0\\n35,0.0,1.0,-0.0,1.0\\n36,0.0,1.0,-0.0,1.0\\n37,0.0,1.0,-0.0,1.0\\n38,0.0,1.0,-0.0,1.0\\n39,0.0,1.0,-0.0,1.0\\n40,0.0,1.0,-0.0,1.0\\n41,0.0,1.0,-0.0,1.0\\n42,0.0,1.0,-0.0,1.0\\n43,0.0,1.0,-0.0,1.0\\n44,0.0,1.0,-0.0,1.0\\n45,0.0,1.0,0.0,1.0\\n46,0.0,1.0,-0.0,1.0\\n47,0.0,1.0,-0.0,1.0\\n48,1e-05,1.0,-0.0,1.0\\n49,0.0,1.0,0.0,1.0\\n50,0.0,1.0,9.84829,0.0\\n51,0.0,1.0,-0.0,1.0\\n52,0.0,1.0,-0.0,1.0\\n53,0.0,1.0,0.0,1.0\\n54,0.0,1.0,0.0,1.0\\n55,0.0,1.0,-0.0,1.0\\n56,0.0,1.0,0.0,1.0\\n57,0.0,1.0,-0.0,1.0\\n58,0.0,1.0,-0.0,1.0\\n59,0.0,1.0,-0.0,1.0\\n60,0.0,1.0,-0.0,1.0\\n61,0.0,1.0,0.0,1.0\\n62,0.0,1.0,-0.0,1.0\\n63,0.0,1.0,0.0,1.0\\n64,0.0,1.0,-0.0,1.0\\n65,0.0,1.0,-0.0,1.0\\n66,0.0,1.0,-0.0,1.0\\n67,0.0,1.0,-0.0,1.0\\n68,0.0,1.0,-0.0,1.0\\n69,0.0,1.0,0.0,1.0\\n70,0.0,1.0,9.45719,0.0\\n71,0.0,1.0,-0.0,1.0\\n72,0.0,1.0,-0.0,1.0\\n73,0.0,1.0,0.0,1.0\\n74,0.0,1.0,3e-05,1.0\\n75,0.0,1.0,0.0,1.0\\n76,0.0,1.0,0.0,1.0\\n77,0.0,1.0,0.0,1.0\\n78,0.0,1.0,0.0,1.0\\n79,0.0,1.0,-0.0,1.0\\n80,0.0,1.0,0.0,1.0\\n'"
      ]
     },
     "execution_count": 63,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "CSV_content = content.decode(\"utf-8\")\n",
    "CSV_content"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {},
   "outputs": [],
   "source": [
    "with open(\"CSV_content_RESULT.csv\", \"w\", newline='') as file:\n",
    "    file.write(CSV_content)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>step</th>\n",
       "      <th>train_loss</th>\n",
       "      <th>train_accuracy</th>\n",
       "      <th>valid_loss</th>\n",
       "      <th>valid_mean_token_accuracy</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>15.81906</td>\n",
       "      <td>0.0</td>\n",
       "      <td>14.67002</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>13.07276</td>\n",
       "      <td>0.0</td>\n",
       "      <td>15.50881</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>13.87567</td>\n",
       "      <td>0.0</td>\n",
       "      <td>16.25000</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>13.33730</td>\n",
       "      <td>0.0</td>\n",
       "      <td>12.77298</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>14.30848</td>\n",
       "      <td>0.0</td>\n",
       "      <td>12.62350</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>6</td>\n",
       "      <td>13.32132</td>\n",
       "      <td>0.0</td>\n",
       "      <td>11.60394</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   step  train_loss  train_accuracy  valid_loss  valid_mean_token_accuracy\n",
       "0     1    15.81906             0.0    14.67002                        0.0\n",
       "1     2    13.07276             0.0    15.50881                        0.0\n",
       "2     3    13.87567             0.0    16.25000                        0.0\n",
       "3     4    13.33730             0.0    12.77298                        0.0\n",
       "4     5    14.30848             0.0    12.62350                        0.0\n",
       "5     6    13.32132             0.0    11.60394                        0.0"
      ]
     },
     "execution_count": 69,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# DEPRECATED ---> results[results['classification/accuracy'].notnull()].tail(1)\n",
    "\n",
    "results = pd.read_csv('/home/eddygiusepe/1_Eddy_Giusepe/6_REPO_HuggingFace/15_Binary_classification_fine-tuning/CSV_content_RESULT.csv')\n",
    "df = results\n",
    "df.head(6) \n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA0oAAAIjCAYAAAA9VuvLAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy81sbWrAAAACXBIWXMAAA9hAAAPYQGoP6dpAACUYUlEQVR4nOzdd3gU1cIG8He2ZNMJNYUEAgjSERERIRRpAiIQQSkq6FWvV1S4WNFPFBuKVy5YsV3xeilSAjZEIhAIivRYAGmGFhICQghJSLLZne+PyWy2Jrubna3v73ny7Gb27Ozh5BDm5Zw5RxBFUQQRERERERGZqHxdASIiIiIiIn/DoERERERERGSFQYmIiIiIiMgKgxIREREREZEVBiUiIiIiIiIrDEpERERERERWGJSIiIiIiIisMCgRERERERFZYVAiIiIiIiKywqBERER2TZ06Fampqb6uRkB44YUXIAiCxbHU1FRMnTrV5/UgIiL3MCgREZk5duwY/v73v6N169YIDw9HbGws+vTpg4ULF+LKlSu+rl7QePLJJyEIAu644w5fV4U8ZPHixRAEAbt37/Z1VYiIPELj6woQEfmLb7/9FuPHj4dOp8Pdd9+Nzp07o7KyEtu2bcMTTzyB/fv348MPP/R1NQOeKIpYtmwZUlNT8fXXX+Py5cuIiYnxdbU87tChQ1Cp+P+RRESBikGJiAhAbm4uJkyYgJYtW2LTpk1ITEw0vTZt2jQcPXoU3377rUc+q7S0FFFRUR45VyDKysrC6dOnsWnTJgwbNgwZGRmYMmWKRz+jrKwMkZGRHj2nq3Q6nU8/n4iI6of/1UVEBGDevHkoKSnBJ598YhGSZFdddRWmT58OADh+/DgEQcDixYttygmCgBdeeMH0vXzPyIEDBzBp0iQ0bNgQffv2xb/+9S8IgoATJ07YnGPWrFkICwvDxYsXAQDZ2dkYP348WrRoAZ1Oh5SUFPzzn/+0mQpYUFCAe+65B8nJydDpdEhMTMTo0aNx/PjxOv/8a9euRefOnREeHo7OnTtjzZo1dssZjUYsWLAAnTp1Qnh4OOLj4/H3v//dVFdnLFmyBB07dsTAgQMxePBgLFmyxKZMVlYWBEHAF198gWeeeQYJCQmIiorCrbfeilOnTlmUHTBgADp37ow9e/agX79+iIyMxDPPPAMAqKiowPPPP4+rrrrK1HZPPvkkKioqLM4hCAIefvhhUzvodDp06tQJ69evt6nbtm3b0LNnT4SHh6NNmzb44IMP7P45re9REgTB4Zf8M/r1118xdepU09TPhIQE3Hvvvfjrr7/crkdVVRVeeukltGnTBjqdDqmpqXjmmWds2sBb9u3bh+HDhyM2NhbR0dEYNGgQfv75Z4syer0ec+bMQdu2bREeHo7GjRujb9++yMzMNJWpT38nInIGR5SIiAB8/fXXaN26NW688UZFzj9+/Hi0bdsWr776KkRRxC233IInn3wSK1aswBNPPGFRdsWKFRg6dCgaNmwIAFi5ciXKysrwj3/8A40bN8bOnTvx9ttv4/Tp01i5cqXpfbfddhv279+PRx55BKmpqSgsLERmZiZOnjxZ66IMGzZswG233YaOHTti7ty5+Ouvv0wXoNb+/ve/Y/Hixbjnnnvw6KOPIjc3F++88w727duHH3/8EVqtttZ2qKiowOrVq/HYY48BACZOnIh77rkHBQUFSEhIsCn/yiuvQBAEPPXUUygsLMSCBQswePBg5OTkICIiwlTur7/+wvDhwzFhwgTceeediI+Ph9FoxK233opt27bhgQceQIcOHfDbb7/h3//+Nw4fPoy1a9dafNa2bduQkZGBhx56CDExMXjrrbdw22234eTJk2jcuDEA4LfffsPQoUPRtGlTvPDCC6iqqsLzzz+P+Pj4Wv/cAPD555/bHPu///s/FBYWIjo6GgCQmZmJP//8E/fccw8SEhJM0z3379+Pn3/+2bRQgyv1uO+++/DZZ59h3LhxeOyxx7Bjxw7MnTsXBw8edBiIlbJ//36kpaUhNjYWTz75JLRaLT744AMMGDAAW7ZsQa9evQBI/8Ewd+5c3Hfffbj++utRXFyM3bt3Y+/evRgyZAgA9/s7EZHTRCKiEHfp0iURgDh69Ginyufm5ooAxE8//dTmNQDi888/b/r++eefFwGIEydOtCnbu3dvsUePHhbHdu7cKQIQ//vf/5qOlZWV2bx37ty5oiAI4okTJ0RRFMWLFy+KAMQ33njDqT+DuWuuuUZMTEwUi4qKTMc2bNggAhBbtmxpOpadnS0CEJcsWWLx/vXr19s9bs+qVatEAOKRI0dEURTF4uJiMTw8XPz3v/9tUW7z5s0iALF58+ZicXGx6fiKFStEAOLChQtNx/r37y8CEBctWmRxjs8//1xUqVRidna2xfFFixaJAMQff/zRdAyAGBYWJh49etR07JdffhEBiG+//bbp2JgxY8Tw8HBTu4uiKB44cEBUq9Wi9T+pLVu2FKdMmeKwLebNm+fUz3rZsmUiAHHr1q0u1yMnJ0cEIN53330W53z88cdFAOKmTZsc1s9Vn376qQhA3LVrl8MyY8aMEcPCwsRjx46Zjp05c0aMiYkR+/XrZzrWrVs3ceTIkQ7PU5/+TkTkLE69I6KQV1xcDACKLijw4IMP2hy74447sGfPHhw7dsx07IsvvoBOp8Po0aNNx8xHTkpLS3H+/HnceOONEEUR+/btM5UJCwtDVlaWS9Pg8vPzkZOTgylTpqBBgwam40OGDEHHjh0tyq5cuRINGjTAkCFDcP78edNXjx49EB0djc2bN9f5eUuWLMF1112Hq666CoDU5iNHjrQ7/Q4A7r77boufy7hx45CYmIh169ZZlNPpdLjnnnts6tuhQwe0b9/eor433XQTANjUd/DgwWjTpo3p+65duyI2NhZ//vknAMBgMOD777/HmDFj0KJFC1O5Dh06YNiwYXX+2c1t3rwZs2bNwiOPPIK77rrLdNz8Z11eXo7z58/jhhtuAADs3bvX5XrI7TRz5kyL4/KInqfuu3OGwWDAhg0bMGbMGLRu3dp0PDExEZMmTcK2bdtMfxfj4uKwf/9+HDlyxO653O3vRESuYFAiopAXGxsLALh8+bJin9GqVSubY+PHj4dKpcIXX3wBQFoNbuXKlab7N2QnT57E1KlT0ahRI0RHR6Np06bo378/AODSpUsApKDw+uuv47vvvkN8fDz69euHefPmoaCgoNZ6yfdItW3b1ua1q6++2uL7I0eO4NKlS2jWrBmaNm1q8VVSUoLCwsJaP6uoqAjr1q1D//79cfToUdNXnz59sHv3bhw+fNjmPdb1EgQBV111lc19KM2bN0dYWJhNfffv329T13bt2gGATX3NQ4esYcOGpgvxc+fO4cqVK061VW1Onz6NO+64A3369MH8+fMtXrtw4QKmT5+O+Ph4REREoGnTpqa+I/+sXanHiRMnoFKpTMFUlpCQgLi4OLv3yMmuXLmCgoICi6/6OHfuHMrKyuy2VYcOHWA0Gk33n7344osoKipCu3bt0KVLFzzxxBP49ddfTeXd7e9ERK7gPUpEFPJiY2ORlJSE33//3anyjjb0NBgMDt9jPlIgS0pKQlpaGlasWIFnnnkGP//8M06ePInXX3/d4pxDhgzBhQsX8NRTT6F9+/aIiopCXl4epk6dCqPRaCo7Y8YMjBo1CmvXrsX333+P5557DnPnzsWmTZvQvXt3p/5stTEajWjWrJnD0Z+mTZvW+v6VK1eioqICb775Jt58802b15csWYI5c+a4VTd77Ws0GtGlSxebMCJLSUmx+F6tVtstJ4qiW3Wyp7KyEuPGjYNOp8OKFSug0Vj+M3z77bfjp59+whNPPIFrrrkG0dHRMBqNuPnmmy1+1q5yZxPaL774wmaUzpNtUZt+/frh2LFj+PLLL7FhwwZ8/PHH+Pe//41FixbhvvvuA6B8fyciYlAiIgJwyy234MMPP8T27dvRu3fvWsvKiywUFRVZHK/tf+cdueOOO/DQQw/h0KFD+OKLLxAZGYlRo0aZXv/tt99w+PBhfPbZZ7j77rtNx81X/zLXpk0bPPbYY3jsscdw5MgRXHPNNXjzzTfxv//9z275li1bAoDdKU6HDh2yOfcPP/yAPn362A0mdVmyZAk6d+6M559/3ua1Dz74AEuXLrUJStb1EkURR48eRdeuXev8vDZt2uCXX37BoEGD3AoK1po2bYqIiAin2sqRRx99FDk5Odi6davNwgsXL17Exo0bMWfOHMyePdt03PrzXKlHy5YtYTQaceTIEXTo0MF0/OzZsygqKjL9/O0ZNmyYw37mjqZNmyIyMtJuW/3xxx9QqVQW4bVRo0a45557cM8996CkpAT9+vXDCy+8YApKgOv9nYjIFZx6R0QE4Mknn0RUVBTuu+8+nD171ub1Y8eOYeHChQCkEagmTZpg69atFmXee+89lz/3tttug1qtxrJly7By5UrccsstFnssyaMc5v+TL4qiqS6ysrIylJeXWxxr06YNYmJial0GOjExEddccw0+++wz09QuQApiBw4csCh7++23w2Aw4KWXXrI5T1VVlU1wNHfq1Cls3boVt99+O8aNG2fzdc899+Do0aPYsWOHxfv++9//WkyJXLVqFfLz8zF8+HCHn2Ve37y8PHz00Uc2r125cgWlpaV1nsOcWq3GsGHDsHbtWpw8edJ0/ODBg/j+++/rfP+nn36KDz74AO+++y6uv/56u+cHbEdtFixY4HY9RowYYfcc8ijbyJEjHdY3MTERgwcPtviqD7VajaFDh+LLL7+0mDp59uxZLF26FH379jVNObVeDj06OhpXXXWVqS+729+JiFzBESUiIkgXWUuXLsUdd9yBDh064O6770bnzp1RWVmJn376CStXrrTYE+e+++7Da6+9hvvuuw/XXXcdtm7davcem7o0a9YMAwcOxPz583H58mXccccdFq+3b98ebdq0weOPP468vDzExsZi9erVNjewHz58GIMGDcLtt9+Ojh07QqPRYM2aNTh79iwmTJhQax3mzp2LkSNHom/fvrj33ntx4cIFvP322+jUqRNKSkpM5fr374+///3vmDt3LnJycjB06FBotVocOXIEK1euxMKFCzFu3Di7n7F06VKIoohbb73V7usjRoyARqPBkiVLTEtEA9KoQt++fXHPPffg7NmzWLBgAa666ircf//9tf6ZAOCuu+7CihUr8OCDD2Lz5s3o06cPDAYD/vjjD6xYsQLff/89rrvuujrPY27OnDlYv3490tLS8NBDD6GqqsrUVub30Fg7f/48HnroIXTs2BE6nc5mxGPs2LGIjY013Wuj1+vRvHlzbNiwAbm5uW7Xo1u3bpgyZQo+/PBDFBUVoX///ti5cyc+++wzjBkzBgMHDnTpz++M//znP3b3n5o+fTpefvllZGZmom/fvnjooYeg0WjwwQcfoKKiAvPmzTOV7dixIwYMGIAePXqgUaNG2L17N1atWoWHH34YQP36OxGR03y23h4RkR86fPiweP/994upqaliWFiYGBMTI/bp00d8++23xfLyclO5srIy8W9/+5vYoEEDMSYmRrz99tvFwsJCh8uDnzt3zuFnfvTRRyIAMSYmRrxy5YrN6wcOHBAHDx4sRkdHi02aNBHvv/9+09LV8hLl58+fF6dNmya2b99ejIqKEhs0aCD26tVLXLFihVN/7tWrV4sdOnQQdTqd2LFjRzEjI0OcMmWKxfLgsg8//FDs0aOHGBERIcbExIhdunQRn3zySfHMmTMOz9+lSxexRYsWtdZhwIABYrNmzUS9Xm9aHnzZsmXirFmzxGbNmokRERHiyJEjLZbEFkVpefBOnTrZPWdlZaX4+uuvi506dRJ1Op3YsGFDsUePHuKcOXPES5cumcoBEKdNm2bzfntLfG/ZskXs0aOHGBYWJrZu3VpctGiR6efs6L3ykvKOvnJzc0VRFMXTp0+LY8eOFePi4sQGDRqI48ePF8+cOWPTr1yph16vF+fMmSO2atVK1Gq1YkpKijhr1iyL/uwJ8vLgjr5OnToliqIo7t27Vxw2bJgYHR0tRkZGigMHDhR/+ukni3O9/PLL4vXXXy/GxcWJERERYvv27cVXXnlFrKysFEWx/v2diMgZgih66c5MIiIiJ2VlZWHgwIFYuXKlw1EqIiIiJfEeJSIiIiIiIisMSkRERERERFYYlIiIiIiIiKzwHiUiIiIiIiIrHFEiIiIiIiKywqBERERERERkJeg3nDUajThz5gxiYmIgCIKvq0NERERERD4iiiIuX76MpKQkqFS1jxkFfVA6c+YMUlJSfF0NIiIiIiLyE6dOnUJycnKtZYI+KMXExACQGiM2Nlaxz9Hr9diwYQOGDh0KrVar2OeEOraz8tjGymMbK49trDy2sfLYxt7BdlaeP7VxcXExUlJSTBmhNkEflOTpdrGxsYoHpcjISMTGxvq8AwQztrPy2MbKYxsrj22sPLax8tjG3sF2Vp4/trEzt+RwMQciIiIiIiIrDEpERERERERWGJSIiIiIiIisBP09SkREREREdRFFEXq9HgaDwddVCTp6vR4ajQbl5eWKt69arYZGo/HItkAMSkREREQU0lQqFfLy8lBeXu7rqgQlURSRkJCAU6dOeWVf08jISCQmJiIsLKxe52FQIiIiIqKQZTQa0bRpU1RVVSEpKQlhYWFeuZgPJUajESUlJYiOjq5zk9f6EEURlZWVOHfuHHJzc9G2bdt6fR6DEhERERGFLL1eD61Wi8TERERHR/u6OkHJaDSisrIS4eHhigYlAIiIiIBWq8WJEydMn+kuLuZARERERCFLFEUAUPwCnrzHUz9L9ggiIiIiIiIrDEpERERERERWGJSIiIiIiOrJYACysoBly6THQFtlPDU1FQsWLPDIubKysiAIAoqKijxyPl/hYg5ERERERPWQkQFMnw6cPl1zLDkZWLgQSE9X7nMHDBiAa665xiMBZ9euXYiKiqp/pYIIR5SIiIiIiNyUkQGMG2cZkgAgL086npHhm3oB0kIVVVVVTpVt2rQpIiMjFa5RYGFQCjSBPq5LRERE5OdEESgtrfuruBh49FGpvL1zANJIU3Gxc+ezdx5Hpk6dii1btmDhwoUQBAGCIGDx4sUQBAHfffcdevToAZ1Oh23btuHYsWMYPXo04uPjER0djZ49e+KHH36wOJ/11DtBEPDxxx9j7NixiIyMRNu2bfHVV1+50ZqSr776Cl26dIFOp0NqairefPNNi9ffe+89tG3bFuHh4YiPj8e4ceNMr61atQpdunRBREQEGjdujMGDB6O0tNTtujiLQSmQZGQAqanAwIHApEnSY2qqb/+rgoiIiCjIlJUB0dF1fzVoII0cOSKK0khTgwbOna+szPk6Lly4EL1798b999+P/Px85OfnIyUlBQDw9NNP47XXXsPBgwfRtWtXlJSUYMSIEdi4cSP27duHm2++GaNGjcLJkydr/Yw5c+bg9ttvx6+//ooRI0Zg8uTJuHDhgvOVrLZnzx7cc889uOOOO/Dbb7/hhRdewHPPPYfFixcDAHbv3o1HH30UL774Ig4dOoT169ejX79+AID8/HxMnDgR9957Lw4ePIisrCykp6eblnVXEu9RChTyuK51p5DHdVetUnYSLBERERH5jQYNGiAsLAyRkZFISEgAAPzxxx8AgBdffBFDhgwxlW3UqBG6detm+v6ll17CmjVr8NVXX+Hhhx92+BlTp07FxIkTAQCvvvoq3nrrLezcuRM333yzS3X997//jf79++P//u//oFKp0K5dOxw4cABvvPEGpk6dipMnTyIqKgq33HILYmJi0LJlS3Tv3h2AFJSqqqqQnp6Oli1bAgC6dOni0ue7iyNKgcBgkMZtaxvXnTGD0/CIiIiIPCAyEigpqftr3TrnzrdunXPn89QtQtddd53F9yUlJXj88cfRoUMHxMXFITo6GgcPHqxzRKlr166m51FRUYiNjUVhYaHL9fnjjz/Qq1cvi2N9+vTBkSNHYDAYMGTIELRs2RKtW7fGXXfdhSVLlqCsenitW7duGDRoELp06YLx48fjo48+wsWLF12ugzsYlAJBdrbtHYLmRBE4dUoqR0RERET1IghAVFTdX0OHSqvbCYLj86SkSOWcOZ+j87jKevW6xx9/HGvWrMGrr76K7Oxs5OTkoEuXLqisrKz1PFqt1urPI8BoNHqmkmZiYmKwd+9eLFu2DImJiZg9eza6deuGoqIiqNVqZGZm4rvvvkPHjh3x9ttv4+qrr0Zubq7H62GNQSkQ5Od7thwRERER1ZtaLS0BDtiGHPn7BQukckoICwuDwYkZRT/++COmTp2KsWPHokuXLkhISMDx48eVqZQd7du3x44dO2zq1K5dO6irG0ej0WDw4MGYN28efv31Vxw/fhybNm0CIAW0Pn36YM6cOdi3bx/CwsKwZs0axevNe5QCQWKiZ8sRERERkUekp0u3itvbR2nBAmVvIU9NTcWOHTtw/PhxREdHOxztadu2LTIyMjBq1CgIgoDnnntOkZEhR2bOnIlevXrh5ZdfxoQJE7B9+3a88847eO+99wAA33zzDf7880/069cPDRs2xLp162A0GnH11Vdjx44d2LhxI4YOHYpmzZphx44dOHfuHDp06KB4vX06orR161aMGjUKSUlJEAQBa9eutSlz8OBB3HrrrWjQoAGioqLQs2fPOudTBp20NOlvmyPyuG5amvfqREREREQApDB0/DiweTOwdKn0mJur/Dpbjz/+ONRqNTp27IimTZs6vEaeP38+GjZsiBtvvBGjRo3CsGHDcO211ypbOTPXXnstPv30U3zxxRfo3LkzZs+ejRdffBFTp04FAMTFxSEjIwM33XQTOnTogEWLFmHZsmXo1KkTYmNjsXXrVowYMQLt2rXD//3f/+HNN9/E8OHDFa+3T0eUSktL0a1bN9x7771It9OTjh07hr59++Jvf/sb5syZg9jYWOzfvx/h4eE+qK0PqdXSf0mYrSdv4o1xXSIiIiKqlVoNDBjg3c9s164dtm/fbnFMDh/mUlNTTdPYZNOmTbP43noqnr3lt4uKipyq14ABA2zef+utt+LOO++ESmU7TtO3b19kZWXZPVeHDh2wfv16pz7X03walIYPH15rGnz22WcxYsQIzJs3z3SsTZs23qia/5GXQREEy9XvvDGuS0REREQUYvz2HiWj0Yhvv/0WTz75JIYNG4Z9+/ahVatWmDVrFsaMGePwfRUVFaioqDB9X1xcDADQ6/XQ6/WK1Vc+t1KfoX7tNagAGB55BNBooJ4/H8aePWHYulX6LwwF/2z+ROl2JraxN7CNlcc2Vh7bWHlsY++oqqoCII2gePO+nUD2j3/8A0uWLLH72uTJk/H+++9bHJNHl7zVxkajEaIoQq/XmxaLkLny90kQvbGtrRMEQcCaNWtMIaigoACJiYmIjIzEyy+/jIEDB2L9+vV45plnsHnzZvTv39/ueV544QXMmTPH5vjSpUsR6anF6d1gMAAHDjTGxYvhaNiwHB07/uX0TLmGhw+j35NPwqjRIHPRIsSeOIHeL72ES6mpyFqwQNF6ExEREQUzjUaDhIQEpKSkICwszNfVCQjnzp3D5cuX7b4WExODpk2berlGliorK3Hq1CkUFBSYgrCsrKwMkyZNwqVLlxAbG1vrefw2KJ05cwbNmzfHxIkTsXTpUlO5W2+9FVFRUVi2bJnd89gbUUpJScH58+frbIz60Ov1yMzMxJAhQ2zWnF+zRsDMmWrk5dWsG9m8uYj58w0YO7bu5lePHw/Vl1/COGUKDB99BOTkQHv99RCbNUNVbfsrBaHa2pk8g22sPLax8tjGymMbK49t7B0lJSX4888/0b59e0RERPi6OkFJFEVcvnwZMTExEDy1WVQtysvLcfz4caSkpNisbVBcXIwmTZo4FZT8dupdkyZNoNFo0LFjR4vjHTp0wLZt2xy+T6fTQafT2RzXarVe+SVj/TkZGcCECZa3FQHAmTMCJkzQYNWqOm4v+uMP4KuvAACqp56CSqsFWrQAAAjnzkErfahn/xABwFs/z1DGNlYe21h5bGPlsY2VxzZWlkYjXQ4LgmB3oQGqP3m6nbfaWKVSQRAEu393XPm75Le9ISwsDD179sShQ4csjh8+fBgtW7b0Ua1cYzBIa+rbG7OTj82YIZVz6I03pMKjRwPyevFNm0r3JYkicPasp6tNRERERBTyfDqiVFJSgqNHj5q+z83NRU5ODho1aoQWLVrgiSeewB133IF+/fqZ7lH6+uuvHS4f6G+ysy03HrMmisCpU1I5u8tJ5uUBn38uPX/qqZrjKhWQkCC9np9f+x5LRERERETkMp+OKO3evRvdu3dH9+7dAUi79nbv3h2zZ88GAIwdOxaLFi3CvHnz0KVLF3z88cdYvXo1+vbt68tqOy0/v57lFiyQVrNLSwN697Z8LSlJejxzxt3qERERERGRAz4dUbK3GZW1e++9F/fee6+XauRZiYn1KHfxIrBokfTcfDTJ+k3OpjEiIiIiInKa396jFAzS0qRZcY4W9xAEICVFKmfj/feBkhKgc2dgxAjb1xmUiIiIiPyHwQBkZQHLlkmPtd6E7h9SU1OxwGyrGUEQsHbtWofljx8/DkEQkJOTU+e5s7KyIAgCioqK6l1PX/HbVe+CgVoNLFwIjBsnhSLzwTM5PC1YgJr9lAwG6YalEyekRRwAaTTJXtIKxal3BgOELVvQfOtWCFFRwMCBcHozKiIiIiKlZGRIK3iZ35yenCxdCNa6vLF/yc/PR8OGDX1dDb/BESWFpacDq1YBzZtbHk9OhuXS4BkZQGqqdPE/dSpQVCSFAEcbn4XaiFJ1+2iGDMF18+dDM2SI1F4ZGb6uGREREYWyjAzpf8WtV/DKy5OOB9C1SkJCgt1tdkIVg5IXpKcDx48Dq1fXHDt0yCok2fsLZjBImzDZ+wsmjyiFQlAKol9AREREFABEESgtrfuruBh49NHa94KZPl0q58z56rh339yHH36IpKQk0x5FstGjR+Pee+/FsWPHMHr0aMTHxyM6Oho9e/bEDz/8UOs5rafe7dy5E927d0d4eDiuu+467Nu3z+n62bN69Wp06tQJOp0OqampePPNNy1ef++999C2bVuEh4cjPj4e48aNM722atUqdOnSBREREWjcuDEGDx6M0tLSetWnLgxKXqJWA2PHAvLmwHl51S/UttmSzN5mS/KIUrBPvfPIZlRERERELigrA6Kj6/5q0MDsos4OUZT+o7dBA+fOV1bmdBXHjx+Pv/76C5s3bzYdu3DhAtavX4/JkyejpKQEI0aMwMaNG7Fv3z7cfPPNGDVqFE6ePOnU+UtKSnDLLbegY8eO2LNnD1544QU8/vjjTtfPWk5ODiZMmIAJEybgt99+wwsvvIDnnnsOixcvBiCthv3oo4/ixRdfxKFDh7B+/Xr069cPgDQlcOLEibj33ntx8OBBZGVlIT09vc5F4eqL9yh5kSBIs8X++EMaYbrqKri/2ZI8olRYCFRVAZog/VHWezMqIiIiouDTsGFDDB8+HEuXLsWgQYMASKMuTZo0wcCBA6FSqdCtWzdT+Zdeeglr1qzBV199hYcffrjO8y9duhRGoxGffPIJwsPD0alTJ5w+fRr/+Mc/3Krvu+++i5tuugnPPfccAKBdu3Y4cOAA3njjDUydOhUnT55EVFQUbrnlFsTExKBly5amLYTy8/NRVVWF9PR0tGzZEgDQpUsXt+rhCo4oeVlqqvR4/Hj1AXc3W2raVNp41miUwlKwqvdmVEREREQuioyUVh+u62vdOufOt26dc+eLjHSpmpMnT8bq1atRUVEBAFiyZAkmTJgAlUqFkpISPP744+jQoQPi4uIQHR2NgwcPOj2idPDgQXTt2hXh8nQoAL2t9/V0weHDh9GnTx+LY3369MGRI0dgMBgwZMgQtGzZEq1bt8Zdd92FJUuWoKx6hK1bt24YNGgQunTpgvHjx+Ojjz7CxYsX3a6LsxiUvKxVK+nRFJTc3WxJrQbi46XnwRwS6rUZFREREZEbBAGIiqr7a+hQ5/aCGTrUufM5Oo8Do0aNgiiK+Pbbb3Hq1ClkZ2dj8uTJAIDHH38ca9aswauvvors7Gzk5OSgS5cuqKysrG/rKCImJgZ79+7FsmXLkJiYiNmzZ6Nbt24oKiqCWq1GZmYmvvvuO3Ts2BFvv/02rr76auTm5ipaJwYlL7MZUarPZkuhsKBDvTajIiIiIlKQvBcMYHutYncvGM8KDw9Heno6lixZgmXLluHqq6/GtddeCwD48ccfMXXqVIwdOxZdunRBQkICjpsuQOvWoUMH/PrrrygvLzcd+/nnn92ua7t27fDjjz9aHPvxxx/Rrl07qKvbR6PRYPDgwZg3bx5+/fVXHD9+HJs2bQIgLTTRp08fzJkzB/v27UNYWBjWrFnjdn2cwaDkZTZBSf4LZu9mtLr+goXCgg7mv4CseeEXEBEREVGtnN4LRhmTJ0/Gt99+i//85z+m0SQAaNu2LTIyMpCTk4NffvkFkyZNslkhrzaTJk2CIAi4//77ceDAAaxbtw7/+te/3K7nww8/jE2bNuGll17C4cOH8dlnn+Gdd94xLRDxzTff4K233kJOTg5OnDiB//73vzAajbj66quxY8cOvPrqq9i9ezdOnjyJjIwMnDt3Dh06dHC7Ps5gUPIyOShZjBSOGQMkJNgWrusvWCiMKAE1v4Cs1/X30i8gIiIiolrJe8Fs3gwsXSo95uZ65RrlpptuQqNGjXDo0CFMmjTJdHz+/Plo2LAhbrzxRowaNQrDhg0zjTY5Izo6Gl9//TV+++03dO/eHc8++yxef/11t+vZrVs3LF++HMuXL0fnzp0xe/ZsvPjii5g6dSoAIC4uDhkZGbjpppvQoUMHLFq0CMuWLUOnTp0QGxuLrVu3YsSIEWjXrh3+7//+D2+++SaGDx/udn2cEaRLpfkvOSidOQNUVFRf+2/cCBQUSMtCrlghbTabmChNJ6ttpCQURpRk6elAp07A3r241KIFoj75BJqBAzmSRERERP5BrfbJCrwqlQpn7FwLpqammqatyaZNm2bxvfVUPOvltm+44Qbk5OTUWsaRAQMGmMrKI1m33XYbxo8fb7d83759kZWVZfe1Dh06YP369U59ricxKHlZ06bSgiZlZcDJk0DbtgDef196cepUwJVkLAelYB9RkpWUAACqoqIg9u/PkEREREREiuHUOy+T91ICqu9TOn0a+PJL6cCDD7p2slCZeie7fBkAoKqq8nFFiIiIiELbgw8+iOjoaLtfD7p6TeunOKLkA6mpwIED1UFp20fSXkj9+klTy1wRSlPvANOIkspPl7UkIiIiChUvvviiaSEGa7GxsV6ujTIYlHxAHlE6eUwP/Pcj6ZuHHnL9RPKI0tmzgMEQ3FPRRNEUlNR6vY8rQ0RERBTamjVrhmbNmvm6Gori1DsfkINSox+/kqbNxccDY8e6fqJmzaS5fAYDcO6cR+vod0pLTUuoc0SJiIiIPEWo3m7E2UUKyP956mfJoOQDclBK+/U96cnf/gaEhbl+Io1GCktA8N+nVH1/EsARJSIiIvIcjUYDo9GIsrIyX1eFPET+WWq12nqdh1PvfKBVK6AdDuG64k2ASgU88ID7J0tKkqbe5ecD3bt7rpL+xiwocUSJiIiIPEWtVuPy5cs4d+4cVCoVIiMjTaNM5BlGoxGVlZUoLy+HSqXcOI0oiigrK0NhYSHi4uKgrudtKQxKPpCaCjyIRQAAw/CRULds6f7JEhOBffuCf0EHqxElgw+rQkRERMHl8uXLaNeuHQoLC31dlaAkiiKuXLmCiIgIr4TQuLg4JCQk1Ps8DEo+0DiiDFOxGACQP/ofSK7PyUJliXDzEaWqKhgMBqCew6lEREREsvj4eCQmJkLPKf4ep9frsXXrVvTr16/e0+HqotVq6z2SJGNQ8gHhi+VoiCL8iVY4mjKsfkEpVJYINwtKAICKCiA83Dd1ISIioqCkVqs9dpFNNdRqNaqqqhAeHq54UPIkLubgC++/DwBYhAeRe6KePwI5KIXQiBIAoLzcN/UgIiIiopDAESVvMRiA7Gxg2zZg927oBS0+Fe/Bfcfred4QnHoHgEGJiIiIiBTFoOQNGRnA9OnA6dM1xzRapOmzcfx4ev3OHapT7xiUiIiIiEhBnHqntIwMYNw4y5AEQKMvwyqMQ8u9GfU7vzyiVFAAGI31O5c/Y1AiIiIiIi9iUFKSwSCNJNnZHVheGPGRozOkcu6KjwcEAaiqAs6fd/88/q6kxPL7igrf1IOIiIiIQgKDkpKys21GksypIKK58RQqfsh2/zO0WqBJE+l5MN+nZDWiJHBEiYiIiIgUxKCkJCeDy/nf6xlwQmFBB069IyIiIiIvYlBSkrzQQh3yDM6Vq/NzgnlBBwYlIiIiIvIiBiUlpaUBycnSPUR2GCHgJFKwLzqtfp/DESUiIiIiIo9iUFKSWg0sXCg9tw5LggABwAwsQO7Jeu4AzRElIiIiIiKPYlBSWno6sGoV0Ly55fHkZHw9ZRXWIB3Hj9fzM+Sg5MKIksEAZGUBy5ZJj/VZeM8rqoOSGBcnfc9V74iIiIhIQdxw1hvS04HRo6VV8PLzpWCTloaqL9XAZ6h/UHJx6p29/W+Tk6XBr/R67n+rGHlEqUkToKiIq94RERERkaIYlLxFrQYGDLA4lJoqPXpsRMmJqXfy/rfWWzvl5UnHV63y07Akjyg1aQLh6FFOvSMiIiIiRXHqnQ/JQensWaCsrB4nMh9RsrO5rayW/W9Nx2bM8MNpeAZDTQPJe0YxKBERERGRghiUfKhhQyA2Vnp+4kQ9TpSQID3q9cBffzksVsf+txBF4NQpqZxfKSmped60qfTIoERERERECmJQ8iFB8ND0u7CwmpGWWu5TcnatB79bZVy+P0mjgdiggfScQYmIiIiIFMSg5GMev0+plpTj5P63TpfzGnlEKSYGCA+XnnPVOyIiIiJSEIOSj3lzQYc69r+FIAApKVI5vyKPKJkFJa56R0RERERKYlDyMY8FJSeWCJf3v61lvQcsWCCV8yt2ghKn3hERERGRknwalLZu3YpRo0YhKSkJgiBg7dq1Dss++OCDEAQBCxYs8Fr9vKFVK+kxN7eeJ3JyifD0dKBlS9vjkZH+vzQ4gxIREREReYtPg1JpaSm6deuGd999t9Zya9aswc8//4wkedQkiHhzRAkAfv9dWmFPowG+/BKYM0c6LgjAzTfXsw5KMQtKIoMSEREREXmBTzecHT58OIYPH15rmby8PDzyyCP4/vvvMXLkSC/VzHvkoHTuHFBaCkRF1V7eYJCW787PlwaR0tKqp8o5sZgDACxZIj2OGAHceiswahSweLE0ovXNN8Dtt9fnT6MQ8xElnU56zsUciIiIiEhBPg1KdTEajbjrrrvwxBNPoFOnTk69p6KiAhVmF9HFxcUAAL1eD71er0g95fObPzorKgpo0ECDS5cEHD2qR8eOjsuuWSNg5kw18vJqVmNo3lzE/PkGpCc0hQaAeOYMqhzUwWgEli7VABBwxx1V0Oulm5Vuv12F119XY8kSI8aO9bfdZgFVURHUAIxRUTBotdKf88oVRX+eoczdvkzOYxsrj22sPLax8tjG3sF2Vp4/tbErdfDroPT6669Do9Hg0Ucfdfo9c+fOxRx5PpmZDRs2IDIy0pPVsyszM9Pl9zRsOACXLjXAqlW7cd11hXbLbN+eiNdf72lzPC8PuOMONV578AKeAmDMy8O6b7+1u7Td/v2NcPJkGiIi9NBo1mPdOiMAICkpBsBNWLdOxIoVGxAdXeXyn0FJ7fftw9UAjv/1FwoPHsQNAIoLC7F13TpfVy2oudOXyTVsY+WxjZXHNlYe29g72M7K84c2Lisrc7qs3walPXv2YOHChdi7dy8ER+tZ2zFr1izMnDnT9H1xcTFSUlIwdOhQxMbGKlFVAFI6zczMxJAhQ6DVal167yefqHH8ONCs2fUYMcJo87rBAEybJv+orNtCgCCI+OjrYXgKgFqvx4jevYFGjWzO8+230i1p48apMXas5Q1JH30k4vff1SgrG4bbb69lWTwfUG3cCABo2akTkm64AQDQICwMI0aM8GW1glZ9+jI5h22sPLax8tjGymMbewfbWXn+1MbybDNn+G1Qys7ORmFhIVq0aGE6ZjAY8Nhjj2HBggU47mD1A51OB518H4sZrVbrlR+MO5/Tpo30ePKkGlqt7drcP/4ojRw5IooCjuVFQB/bCNriC9CePw/Ex1uUqawEVq+Wnt91lwpareU6HpMmAc88A3zxhQb33+9S9ZVXWgoAUMfFQYyOBgAIFRU+/4sW7Lz1dyaUsY2VxzZWHttYeWxj72A7K88f2tiVz/fbfZTuuusu/Prrr8jJyTF9JSUl4YknnsD333/v6+p5VF0r39WxPoNJaazjBR3WrwcuXAASEoCbbrJ974QJ0uPmzc5/ntfYWx6cizkQERERkYJ8OqJUUlKCo0ePmr7Pzc1FTk4OGjVqhBYtWqBx48YW5bVaLRISEnD11Vd7u6qKqisoyQva1cUYnwic3m93LyV5tbsJE+xvKNuqFdC7N7B9O7BiBTB9unOf6RXmy4PLo4VcHpyIiIiIFOTTEaXdu3eje/fu6N69OwBg5syZ6N69O2bPnu3LanldXUEpLQ1ITra7PgMA6XhKChDX0f5eSpcvA199JT2fPNlxPSZOlB6XLXOq2t5TUiI9csNZIiIiIvISn44oDRgwAKLo/MIBju5LCnQtW0qP589LmaD6NhwTtRpYuBC47Tb77xdFYMECQLWzeujJakRpzRopV7RrB/To4bget98OzJgB7NgBHDtWc++Uz9mbesegREREREQK8tt7lEJJXJz0BQAnTtgvM2IE4GjRPpUKaNsWQJL9ESV52t3kyY5HpQBp/YdBg6Tny5c7U3MvsROUBL1eWg6QiIiIiEgBDEp+olUr6TE31/7rX3wBFBcDzZsDP/wALF0qLbwwZoy0kex99wGGeNvFHAoKpPKAtLJdXeQyS5dKI1V+wd6IEsAFHYiIiIhIMQxKfqK2+5REUZp6BwAPPyyN+kycCAwYALz7rjTStHMnkPGT7dS7L76QglSvXsBVV9Vdj7FjAZ0OOHAA+O23+vyJPMhRUOL0OyIiIiJSCIOSn6gtKP30E7Bvn5QR7rvP8rWkJOCNN6Tncz4ym3pXPRxkPu3OGQ0aACNHSs+XLnW6+sqpqqoJRDExgEYDo6q62zIoEREREZFCGJT8RG1B6a23pMfJk4EmTWxfv+8+oF8/4NiV6hGlK1eAS5dw5Aiwa5e0GMQddzhfF3n1u+XLpdEon5JHkwApKAEwhoVJ3zMoEREREZFCGJT8hKOgdPo0sHq19PyRR+y/V6UCPvwQEHURuIg4AMDOL/Px9NPS64MHA82aOV+XkSOlTHLihDS1b9kyICvLR2snyEFJpwOqd1I2yDsqMygRERERkUIYlPyEo6D0/vtSQOnfH+jWzfH7r74amD0byIc0qvT01HxkZEiv7doF03NnREQA114rPX/0UWmBh4EDpTq6ch6PkIOS2ZrpHFEiIiIiIqUxKPkJOSj99VdNNigvl0aKACmw1OWqq2qCUhJqFnS4eBEYN875kJORAWzZYns8L8+183iE+UIO1TiiRERERERKY1DyE7GxQKNG0nN5VGn5cmkT2hYtgFtvrf39BgPw2GPAGUgLOiSiZolweZnvGTPqnj5nMADTp9t/zZXzeIydoMQRJSIiIiJSGoOSHzGffieKNYs4TJsGaDS1vzc7W7qfyd6IEiCd79QpqZwz53HE2fN4jL2gxBElIiIiIlIYg5IfMQ9KP/4oLQkeEWG7JLg98h6z9kaU7JWr6zzOfp7iSkqkR/OpdxxRIiIiIiKF1TFOQd5kHpTkEZs776yZklebxOqVweURJUdBSS5X13mc/TzFcUSJiIiIiHyAQcmPtGwpPa5dW3OfkqMlwa2lpQHJyUDBaftT7wRBej0tzbnz5OXV3JPkznk8xt5iDhxRIiIiIiKFceqdn8jIAF58UXr+55/SRq86HXDkiHPvV6uBhQutp95JSUcQpDILFkjlnDmP+fvMiSLwr3/VfR6P4YgSEREREfkAg5IfyMiQlt3+6y/L4xUVri3HnZ4O/GuJNKIUjVLEQAoZycnAqlXS686eZ9UqoHlzy+NycMrMtD/apAiOKBERERGRDzAo+Zi8HHdtwcOV5bhHT4qCGBsLAFj6r3xs3gzk5jofkmTp6dL0v82bgaVLpccvvwRUKuDjj2tGnRTHESUiIiIi8gHeo+RjrizHPWCAc+cUEhOB4mLc0iMfGHC123VTq20/81//AmbOlPZsuvpqYOhQqW75+dICD2lpHp6WxxElIiIiIvIBBiUfU2Q57sRE4NAh4MyZusu6aMYMYP9+4JNPgNtukzbKPXu25vXkZGm0ydURLIc4okREREREPsCpdz6myHLcSdKCDkpsdiQIwHvvAR07AleuWIYkQFotz5X7quokB6XoaNMhjigRERERkdIYlHxMXo7b3gpzgHQ8JcXF5bjlVKXAiBIgTa0rKrL/mnyvlSv3VdWKI0pERERE5AMMSj5W23LcrizrbSEhQXrcvh3IyvJQYqmRnV17BjO/r6reeI8SEREREfkAg5IfcLQct6vLegOQ5rzNnSs9374dGDgQSE314Fw4he6rcoQjSkRERETkA1zMwU+kpwOjR9dzBTl5QybrtcblG4dcTl32KXJflSMlJdIjR5SIiIiIyIsYlPyIveW4nVbbhkyiKM3jmzFDSmNqtVTezVQm31eVl+d4/6ekJBfvq7KnslL6AjiiRERERERexal3wcKVDZkyMqTpeAMHApMmuTw9r7b7qmR6PfDnn9Jzg0G6VWrZMhdvmZKn3QFc9Y6IiIiIvIpBKVg4e0PQf/4jTcOzDlUuruvt6L6qxERpLYlz54C+faUNat3OZHJQiogANDWDnxxRIiIiIiKlMSgFC2dvCPr8c8fT8wCX1vVOTweOHwc2bwaWLpUeT50CcnKA7t2BwkLgiSfqkcnsLOQAAEaOKBERERGRwhiUgkVdGzIBgBwwHHFjXW/5vqqJE6VHtRqIjwd++AHQ6Rx/DOBEJnMQlAwcUSIiIiIihTEoBYu6NmQSBOChh5w7lwfW9f71V6CiwvHrTmUyjigRERERkY8wKAWTujZkGj3aufN4YF1vj+y1JAcls4UcAI4oEREREZHyuDx4sKltQyaDofZ1vQVBer3e63p7aK8ljigRERERkY8wKAUjRxsyydPzxo2TQpF5WJKn6y1Y4OIut/bVtdeSU5mM9ygRERERkY9w6l2ocTQ9Ly5OOp6e7pGPqeuWKcCJTFbXiJJe78KmTEREREREzmNQCkXm63rfcYd0rHt3j4Uk84+xl8maNXMyk9U1ogTUvmIEEREREZGbGJRClTw979VXpe+zsoCzZz3+MeaZ7IYbpGN33ulkJispkR4djSgBnH5HRERERIpgUAp1rVsDPXsCRqMTO8C6R85kjz8ufb9qlf37lmw4GFES1WqI8pw9BiUiIiIiUgCDEgG33y49fvGFoh8zfDgQFQWcOAHs2uXEGxwEJQBAeLj0yKBERERERApgUKKaoLR1K3DmjGIfExkJjBolPV+xwok3MCgRERERkY8wKBHQogXQu7c0H271akU/Ss5kK1Y4Mf2OQYmIiIiIfIRBiSRemn53881AdDRw6hSwY0cdhRmUiIiIiMhHGJRIMn689Pjjj1KKUUhEBHDrrdLzOqff1RaUdDrpkUGJiIiIiBTg06C0detWjBo1CklJSRAEAWvXrjW9ptfr8dRTT6FLly6IiopCUlIS7r77bpxR8B6akNa8OdC3r/R81SpFP0oevFq5UlpszyE5KEVH27wkckSJiIiIiBTk06BUWlqKbt264d1337V5raysDHv37sVzzz2HvXv3IiMjA4cOHcKt8nAEeZ68+azC0++GDQNiY4HTp4Gff3ZQSBQ59Y6IiIiIfEbjyw8fPnw4hg8fbve1Bg0aIDMz0+LYO++8g+uvvx4nT55EixYtvFHF0DJuHPDoo9LNQ8ePA6mpinxMeDgwejTw+efS9Lsbb7RTqLwcMBik5wxKRERERORlPg1Krrp06RIEQUBcXJzDMhUVFaioqDB9X1xcDECayqfX6xWrm3xuJT9DcY0bQ92vH1RbtsCwfDmMjz2m2Eelpwv4/HMNVq4U8frrVVBZj21euABt9VO9TgdYta8YFgYAqCothRjIbe6HgqIv+zm2sfLYxspjGyuPbewdbGfl+VMbu1KHgAlK5eXleOqppzBx4kTExsY6LDd37lzMmTPH5viGDRsQGRmpZBUBwGYULNCkduiAblu24PLHH2NLhw6KfU5VlQqRkTfjzBkt3nzzZ3TqdMHi9cj8fAwBUBUejnXr19u8/+ylS0gCsH/3bhxv1kyxeoayQO/LgYBtrDy2sfLYxspjG3sH21l5/tDGZWVlTpcVRLHO3Wy8QhAErFmzBmPGjLF5Ta/X47bbbsPp06eRlZVVa1CyN6KUkpKC8+fP1/q++tLr9cjMzMSQIUOg1WrrfoO/OncOmpQUCEYj9AcOAFddpdhH/e1vanz+uQoPPWTAggVWqzr88gu0PXtCTEhA1cmTpsNyO49ctgyaL76AYd48GGfMUKyOoSho+rIfYxsrj22sPLax8tjG3sF2Vp4/tXFxcTGaNGmCS5cu1ZkN/H5ESa/X4/bbb8eJEyewadOmOv9AOp0OOnnpaDNardYrPxhvfY5ikpKAm24CfvgB2rVrgVmzFPuoCROk+5QyMtR46y011GqzF6vvPRJiYuy2p6p6dFCt10MdyO3txwK+LwcAtrHy2MbKYxsrj23sHWxn5flDG7vy+X69j5Icko4cOYIffvgBjRs39nWVQoMrq98ZDEBWFrBsmfQoL8DghMGDgbg4oKAA2LbN6sXaVrwDlwcnIiIiImX5NCiVlJQgJycHOTk5AIDc3Fzk5OTg5MmT0Ov1GDduHHbv3o0lS5bAYDCgoKAABQUFqKys9GW1g9/YsYBGA/zyC3DokONyGRnSyngDBwKTJkmPqanScSeEhUkfBdjZfLaOoMRV74iIiIhIST4NSrt370b37t3RvXt3AMDMmTPRvXt3zJ49G3l5efjqq69w+vRpXHPNNUhMTDR9/fTTT76sdvBr3Fga7gGA11+3P1qUkSEtJ376tOV78/Kk406GJXnz2VWrrAaj6gpK8vRKBiUiIiIiUoBP71EaMGAAaltLwk/WmQhNbdpIj59+Kn0BQHIysHChtAnS9OnSprDWRBEQBGDGDKmcxY1HtgYNAho2BAoLgbfeAhISgMREoN+ly1KK54gSEREREfmA3y/mQD6QkQG8957tcXm06J57bEeSzIkicOoUkJ0NDBhQ60dptcC11wIbNwIzZ9YcfyP2Mh4HGJSIiIiIyCf8ejEH8gGDofbRIlEE/vMf586Vn19nkYwMKSTZfFSxNPXuSH60/TcyKBERERGRghiUyFJ2du2jRa5ITKz1ZTmT2RMNKSh9syXG7kJ6XPWOiIiIiJTEoESWnBgFAgA0aiTdi2SPIAApKUBaWq2nqC2TxVQHpdPFMcjOtlOAizkQERERkYIYlMhSHaNAJvJQkHVYkr9fsKDOhRxqy2RyULqMGPvlOKJERERERApiUCJLaWnS6nZ1jRY9+6y0pnfz5pavJydLx9PT6/yo2jKZeVCyW45BiYiIiIgUxKBEltRqaQlwoO7RovR04PhxYM4c6Xi7dkBurlMhCag9k0WjBAAQ3iTG/gw+BiUiIiIiUhCDEtlKT3d+tEitBiZOlJ6fPOnSx9SWyeQRpXsejbE/g49BiYiIiIgUxKBE9smjRZs3A0uXSo+ORotatwYiIqTQcuyYyx9jL5PFqaWg1G8k91EiIiIiIu9jUCLH1Gppw9iJE6VHR4szqNVAp07S899+c/lj5EzWrJn0/bvvAglRUlBytOGsyFXviIiIiEhBDErkGV26SI9uBCVAylqtWknPkxJFCCXSPUqOghJHlIiIiIhISQxK5Bn1DEoAkJQkPRYeLwOMRukbBiUiIiIi8gEGJfIMOSj9/rvbp5CXAb94snranSAAkZH2C8tBSa8HDAa3P5OIiIiIyB4GJfKMzp2lx6NHgStX3DqFPKJUdKo6KEVHO97PSQ5KAFBR4dbnERERERE5wqBEnhEfDzRpIk2ZO3DArVPII0ol+bUv5ADAMihx+h0REREReRiDEnmGINT7PiV5RKnsrBNBSaOpWYWPI0pERERE5GEMSuQ59QxK8ohSxXknghLABR2IiIiISDEMSuQ5HhpRMlxiUCIiIiIi32JQIs+pZ1Bq3FiaUReNOvZQkjEoEREREZFCGJTIczp1kh4LCoDz511+u0olTb+LAUeUiIiIiMi3GJTIc6Kjgdatpef1uE+JQYmIiIiIfI1BiTzLA/cpMSgRERERka8xKJFneWDlOwYlIiIiIvI1BiXyLI4oEREREVEQYFAiz+rcWXr8/XfAaHT57RxRIiIiIiJ/wKBEntW2LRAWBpSWAidOuPx2ixGl6OjaCzMoEREREZFCGJTIs7RaoEMH6bkb0+84okRERERE/oBBiTyvHvcpmY8oVUUyKBERERGRbzAokefVIyg1aVITlC5UMigRERERkW8wKJHn1SMoqVRArCAFpYJSBiUiIiIi8g0GJfI8OSgdOgRUVLj2XqMRUWIpAODMZQYlIiIiIvINBiXyvObNgbg4wGAA/vjDtfeWlpqenipiUCIiIiIi32BQIs8TBPen312uXsgBapw+H157WQYlIiIiIlIIgxIpo55B6TJikF8g1F6WQYmIiIiIFMKgRMrwQFA6c6aOsgxKRERERKQQBiVSRufO0mN9RpTy6yjLoERERERECmFQImXIQen0aeDiReffVx2UShDNESUiIiIi8hkGJVJGXByQkiI9//13599nNqJUWAhUVdVSlkGJiIiIiBTCoETKke9TciMolSAGRiNQWFhLWQYlIiIiIlIIgxIpx50FHeTlwSOkPZRqvU+JQYmIiIiIFOLToLR161aMGjUKSUlJEAQBa9eutXhdFEXMnj0biYmJiIiIwODBg3HkyBHfVJZcV4+gJEZLQanW+5QYlIiIiIhIIT4NSqWlpejWrRveffddu6/PmzcPb731FhYtWoQdO3YgKioKw4YNQzkvjAODeVASRefeUx2UVA04okREREREvqPx5YcPHz4cw4cPt/uaKIpYsGAB/u///g+jR48GAPz3v/9FfHw81q5diwkTJnizquSO9u0BjQa4dEla/U5e3KE2JSUAAE1DjigRERERke/4NCjVJjc3FwUFBRg8eLDpWIMGDdCrVy9s377dYVCqqKhARUWF6fvi4mIAgF6vh16vV6y+8rmV/IyAIwjQtGsH4cABVO3bBzEhoc63qC9dggqAtlEUACAvzwC93mh63aKd1WpoAYjl5ahiu3sM+7Ly2MbKYxsrzGCAISsLzbduhUGnAwYMANRqX9cq6LAfewfbWXn+1Mau1MFvg1JBQQEAID4+3uJ4fHy86TV75s6dizlz5tgc37BhAyIjIz1bSTsyMzMV/4xA0qNxYyQDOLR6NY46Mf2u159/IgHAX5WXAAA5Oeewbt0Om3KZmZkIKy7GcABCZSXWffMNoOLaJJ7Evqw8trHy2Mael7h9O7p8/DEi/voL1wHA/Pm40rgxfrvvPuT37u3r6gUl9mPvYDsrzx/auKyszOmyfhuU3DVr1izMnDnT9H1xcTFSUlIwdOhQxMbGKva5er0emZmZGDJkCLRarWKfE2hUv/4KZGejg16PdiNG1Fle/a9/AQCu7tke2AwYjc0wwux9Fu1sNnI4YtAgICLC83+AEMS+rDy2sfLYxsoQ1qyBet48m/tOwy9cQM9582BYvhzi2LE+ql3wYT/2Draz8vypjeXZZs7w26CUUD1N6+zZs0hMTDQdP3v2LK655hqH79PpdNDpdDbHtVqtV34w3vqcgNGtGwBAtX8/VM60S/U9Sg1T4gAA+fkqaLW2I0VarRbasLCa7w0GgO3uUezLymMbK49t7EEGA/DYY3YX5xFEUZpu/fjjwG23cRqeh7EfewfbWXn+0MaufL7fzlVq1aoVEhISsHHjRtOx4uJi7NixA705tB845JXv9u8HPv8cyMqS/rF1pHrVu0YtpcUczp6tpbhGU/OPMRd0ICJSVna2tDCPI6IInDollSMiCgI+HVEqKSnB0aNHTd/n5uYiJycHjRo1QosWLTBjxgy8/PLLaNu2LVq1aoXnnnsOSUlJGDNmjO8qTa7ZswcQBCnt3H23dCw5GVi4EEhPty1fHZTikqOhUgFGI1BYCJgNKloKDwdKSxmUiIiUVut+DW6UIyLycz4NSrt378bAgQNN38v3Fk2ZMgWLFy/Gk08+idLSUjzwwAMoKipC3759sX79eoTLy0KTf8vIAG6/3XaaRl4eMG4csGqVbViqDkrquBgkJEjLg+fnMygREfmcw1/EbpYjIvJzPp16N2DAAIiiaPO1ePFiAIAgCHjxxRdRUFCA8vJy/PDDD2jXrp0vq0zOMhiA6dPtbzQrH5sxw3JeXVUVcOWK9DwmxvRvba17Kcn3ozEoEREpKy1NmhEgCPZfFwRpv7y0NO/Wi4hIIX57jxIFOHfmslcv5AAAiIlBUpL0tNZZHNx0lojIO9Rqado0YBuW5O8XLOBCDkQUNBiUSBnuzGWvnnYHrRbQ6ZwbUWJQIiLynvR0adp08+aWx5OT7U+nJiIKYAxKpAx35rLLQSlGWvGOI0pERH4oPR04fhxi9SbfhqlTgdxchiQKTAYDhC1b0HzrVghbttS+Mi+FHAYlUoY7c9nlqXfVQYkjSkREfkoUIRiN0vOEBE63o8CUkQGkpkIzZAiumz8fmiFDgNRU6TgRGJRIKe7MZeeIEhFRYJAX3gH4+5cCU0aGtAKv9f3U8sq8DEsEBiVSkqO57I0a1bo0OEeUiIj8nHlQMn9OFAjcWZmXQhKDEimrei47Nm8GRo2Sjg0cWOtms9YjSmfP1vK7ikGJiMj7zMKRwKBEgcadlXkpJPl0w1kKEWo1MGAAEB0NfP018N13QFkZEBlpWc4qKDVrBqhUUkg6d06aBm+DQYmIyPs4okSBzJ2VeSkkcUSJvKdHD6BlS6C0FPj+e9vXrYKSWg3Ex0uHHP6uYlAiIvI+BiUKZO6szEshiUGJvEcQpBskAekeJWtyUIqONh2q8z4lBiUiIu/jYg4UyNxZmZdCEoMSedf48dLj11/b/uNqNaIEOLHyHYMSEZH3cUSJApn5yrzWHK3MSyGJQYm86/rrpf+luXwZ2LDB8jU7QYkjSkREfohBiQKdvDJvVJTl8eRk+yvzUkhiUCLvEgTgttuk59bT7ziiREQUGLjqHQWD9HTTLQHlcXGoyswEcnMZksiEQYm8T75P6auvgIqKmuMcUSIiCgy8R4mCRfV1iKhWQ+zfn9PtyAKDEnlf795SArp0Cdi4seY4R5SIiAIDp95RsCgrAwCoKit9XBHyRwxK5H0qlf3pdyUl0iNHlIiI/BuDEgWL6v6rNp/hQlSNQYl8Q55+t3YtoNdLz2sZUSooAIxGO+dhUCIi8j4GJQoW1f1XU1kJiKKPK0P+hkGJfKNvX6BZM+DiRWDzZumYnaDUrJm0/oPBAJw7Z+c8DEpERN5nvphDZaX0S5ooEFVPvQPAawmywaBEvqFW16wqI0+/sxOUNBopLAEO7lNiUCIi8j7rUST+DqZAxdFRqgWDEvmOPP1uzRrpl5M8P9gsKAE10+/s3qfEoERE5H3Wv3P5O5gCFYMS1YJBiXynf3+gcWPg/Hng229rjkdHWxSTF3TgiBIRkZ+wvqDkBSYFKvOpd+bPicCgRL6k0QBjx0rPP/1UetTpAK3WohhHlIiI/AyDEgULjihRLRiUyLfk6Xfr10uPVtPuAI4oERH5HQYlChbmC5PwWoKsMCiRb910E9CwYc3a34IAZGVZrKDEESUiIj/DoETBQK8HqqpqvufUO7LCoES+pdUC11xT8/25c8DAgUBqKpCRAYAjSkREfodBiYIB+zHVgUGJfCsjo2YfJXN5edK0vIwM50aUKisd7EhLREQexwtMCgbsx1QHBiXyHYMBmD7d/mvy7tgzZiCxmTQNr6DAThaSgxJQs7w4EREpixeYFAysp9px6h1ZYVAi38nOBk6fdvy6KAKnTiHhSDYEQZpGfP68VRnzoMTpd0RE3lEdjKrCwiy+Jwoo3DiZ6sCgRL5j96YjW5pz+Wja1MFbNBpAVd2N+QuOiMg7qi8w9fJKpQxKFIis+q3AfkxWGJTId+RVGpwoJ9+nlJ8vWL4mCFzQgYjI26ovKCvlDcJ5gUmBiFPvqA5uBaVTp07htNmUqZ07d2LGjBn48MMPPVYxCgFpaUByshR27BEEICUFSEszZaqCAjvlGJSIiLxLHlFiUKJAxnvtqA5uBaVJkyZhc/VKZQUFBRgyZAh27tyJZ599Fi+++KJHK0hBTK0GFi6UnluHJfn7BQsAtdps5Ts7oYpBiYjIe0SxZkSJU+8okPEeJaqDW0Hp999/x/XXXw8AWLFiBTp37oyffvoJS5YsweLFiz1ZPwp26enAqlVA8+aWx5OTpePp6QDAESUiIn9htsIoR5QooHHqHdVB486b9Ho9dDodAOCHH37ArbfeCgBo37498p28QZ/IJD0dGD1aWgUvP19KRWlp0ohTNY4oERH5CbNQxHuUKKBxMQeqg1tBqVOnTli0aBFGjhyJzMxMvPTSSwCAM2fOoHHjxh6tIIUItRoYMMDhyxxRIiLyE9UXk6JajaqICItjRAGF9yhRHdyaevf666/jgw8+wIABAzBx4kR069YNAPDVV1+ZpuQReZLDVe8ABiUiIm+SLyYjImConl3CC0wKSJx6R3Vwa0RpwIABOH/+PIqLi9GwYUPT8QceeACRkZEeqxyRzHxEyWi0epFBiYjIe8yCkpEbzlIgk0dHo6IglJbyOoJsuDWidOXKFVRUVJhC0okTJ7BgwQIcOnQIzZo182gFiQAgIUF61OsFbNjQElu2CDAYql9kUCIi8h7zESUGJQpkcr9t1Mjye6JqbgWl0aNH47///S8AoKioCL169cKbb76JMWPG4P333/doBYkA4OuvAVV1b1206BoMGaJBaiqQkQEGJSIib5IvJsPDGZQosMlT7aqDksCpd2TFraC0d+9epKWlAQBWrVqF+Ph4nDhxAv/973/x1ltvebSCRBkZwLhxtlPu8vKk4yfPMSgREXkNR5QoWMhT7+SFyNiPyYpbQamsrAwx1ZvMbdiwAenp6VCpVLjhhhtw4sQJj1aQQpvBAEyfLu1vaE0+9tNeBiUiIq+RLy65mAMFOuupd7yOICtuBaWrrroKa9euxalTp/D9999j6NChAIDCwkLExsZ6rHIGgwHPPfccWrVqhYiICLRp0wYvvfQSRHtXzRSUsrOB06cdvy6KwF+lDEpERF7DxRwoWFRPtTONKHHqHVlxa9W72bNnY9KkSfjnP/+Jm266Cb179wYgjS51797dY5V7/fXX8f777+Ozzz5Dp06dsHv3btxzzz1o0KABHn30UY99DvkvZ/YvLgeDEhGR13DqHQULud/KKzizH5MVt4LSuHHj0LdvX+Tn55v2UAKAQYMGYezYsR6r3E8//YTRo0dj5MiRAIDU1FQsW7YMO3fu9NhnkH+TlwWvDYMSEZEXcTEHChZyv+U9SuSAW0EJABISEpCQkIDT1fOikpOTPb7Z7I033ogPP/wQhw8fRrt27fDLL79g27ZtmD9/vsP3VFRUoKKiwvR9cXExAECv10Ov13u0fubkcyv5GaHohhuA5s01OHMGEEXbzWYFQYQuRgcUA4ayMhjZ/vXGvqw8trHy2MbKUZWUQA3AqNOZgpJ45Qqq2NYex36sLHVpKVQAqho0gBqAUFUFfVkZoNX6umpBx5/6sit1cCsoGY1GvPzyy3jzzTdRUlICAIiJicFjjz2GZ599FiqVW7c+2Xj66adRXFyM9u3bQ61Ww2Aw4JVXXsHkyZMdvmfu3LmYM2eOzfENGzZ4ZTPczMxMxT8j1Nx5ZyJef70nABGAeVgSIYpAx2vPAVlA3rFj2LdunW8qGYTYl5XHNlYe29jz2v3yCzoAOP3XXzCaLeaw7ttvAcH2P7So/tiPlTGgsBANAOw7cQI3VB/b8OWXqPLC9WKo8oe+XObCvWhuBaVnn30Wn3zyCV577TX06dMHALBt2za88MILKC8vxyuvvOLOaW2sWLECS5YswdKlS9GpUyfk5ORgxowZSEpKwpQpU+y+Z9asWZg5c6bp++LiYqSkpGDo0KEeXWjCml6vR2ZmJoYMGQIt/yfCo0aMAK691oDp09UoKKg5npwMvPmmAcPOtAGygOTGjZE4YoTP6hks2JeVxzZWHttYOaqffgIANL/qKhyoHlESjEaMGDIEkKfikUewHytL8/jjAIBrBg4Eqq9dh6alAfHxvqxWUPKnvizPNnOGW0Hps88+w8cff4xbb73VdKxr165o3rw5HnroIY8FpSeeeAJPP/00JkyYAADo0qULTpw4gblz5zoMSjqdDjr5f7jMaLVar/xgvPU5oeb224FbbwUiIqRRpZUrqzB2rAZqtQb4KAoAoKqshIpt7zHsy8pjGyuPbayAykoAgCoysuYeJQDaqiogKspXtQpq7McKqR5ZUMfEoCosDJrKSmj1ek69U5A/9GVXPt+tOXIXLlxA+/btbY63b98eFy5ccOeUdpWVldlM41Or1TBa7zxKISE8HGjWTHqemipCrTZ7AeBiDkRE3mC+PLj5BQdvhKdAY96XuScY2eFWUOrWrRveeecdm+PvvPMOunbtWu9KyUaNGoVXXnkF3377LY4fP441a9Zg/vz5Hl1ZjwKLvApefr7ZPHgGJSIi7zG7uIQgQJR/B/MCkwKN3GfNR0fZj8mMW1Pv5s2bh5EjR+KHH34w7aG0fft2nDp1Cus8eDP922+/jeeeew4PPfQQCgsLkZSUhL///e+YPXu2xz6DAktSkohffhEs7lViUCIi8iLzoCQ/lpfzApMCi9FYc90QEQGDPKLETWfJjFsjSv3798fhw4cxduxYFBUVoaioCOnp6di/fz8+//xzj1UuJiYGCxYswIkTJ3DlyhUcO3YML7/8MsJ4s2jIkkeUzpzhiBIRkU9UByLRPCiZHScKCObXDNw8mRxwex+lpKQkm0UbfvnlF3zyySf48MMP610xInsSEkQAQH6+2UEGJSIi75F/18q/e+WgxN/BFEjMR47MR5QYlMiMZzY8IvKSpCTpkSNKREQ+Yj31jvcoUSCS+2tYGKBW14woceodmWFQooCSmMgRJSIin7IKSiKn3lEgsurHnHpH9jAoUUCRR5QKCjiiRETkE/YWczA/ThQI5JGjyEgA4NQ7ssule5TS09Nrfb2oqKg+dSGqk/k9SkYjoFKBQYmIyJvMg1JpKYMSBSarwG/k1Duyw6Wg1KBBgzpfv/vuu+tVIaLaxMcDgiDCYBBw7pz0vSkoVVaapSciIlKEvOqd/LuX9yhRIOLUO3KCS0Hp008/VaoeRE7RaIC4uApcvBiOM2esghIAVFTU/O8mERF5HqfeUTDg1DtyAv/rnQJOw4bSFDvTgg7mQYnT74iIlMWgRMHA0YgSp96RGQYlCjiNGklh6MyZ6gMaTc10OwYlIiLlGAyAXi8956p3FMg49Y6cwKBEAUceUTIFJUHggg5ERN5gfhHJESUKZJx6R05gUKKAYzOiBDAoERF5g/lFJBdzoEDGqXfkBAYlCjgMSkREPiJfXOp0NVOeOaJEgch6eXCOKJEdDEoUcOSgZFrMAWBQIiLyBuuFHMyf8wKTAonV1LsqBiWyg0GJAg5HlIiIfIRBiYIFN5wlJzAoUcCRF3MoKJAWYALAoERE5A12gpLIe5QoEHHVO3ICgxIFnAYNKqBSiTAagcLC6oMMSkREyuOIEgULrnpHTmBQooCjVgMJCdJz0/Q7BiUiIuUxKFGw4Kp35AQGJQpIiYkiALMFHRiUiIiUx6BEwYJT78gJDEoUkDiiRETkAwxKFCw49Y6cwKBEASkpSRpRsglKFRW+qRARUSjgYg4ULByNKJWXA0ajjypF/oZBiQJSYqL0yBElIiIv4ogSBQtHG84CvJYgEwYlCkgOR5T4y42ISDkMShQsrKfeySNKAPsymTAoUUCSR5S4mAMRkRfVFpQqKjhliQKHVV8W1WqIWq10jCvfUTUGJQpICQkcUSIi8rraghLA38EUOOS+XD2iZPGcI0pUjUGJAlJSkvR49ixQVQUGJSIib6grKPECkwKFPGrEaaRUCwYlCkhNm0obz4qiFJYYlIiIvEC+gJR/5wKARiN9mb9O5M9EsfbQz6l3VI1BiQKSSmV1nxKDEhGR8uxdXJp/z6BEgUCvr7mfznzqHfsxWWFQooBlsUQ4gxIRkfIYlCgYmI8Yme8Jxn5MVhiUKGDJ9ykxKBEReQmDEgUDuZ+qVIC80h3AqXdkg0GJAhaDEhGRlzEoUTAwX/FOEGqOc9U7ssKgRAGLQYmIyMsYlCgY2FvxDqi5lmA/pmoMShSw5KDExRyIiLyEQYmCQV39mFPvqBqDEgUsLuZARORl8u9YR/8Tz9/BFAjsbTZr/j0DP1VjUKKAxal3RERexhElCgYOpt5x1TuyxqBEAUsOSoWFgF7NoEREpDgGJQoGjvqx/J+unHpH1RiUKGA1blyzquf5EgYlIiLFMShRMODUO3ISgxIFLJWq5j6ls5cYlIiIFCWKDEoUHBytesd+TFYYlCigmRZ0uFAdlCoqpH/MiYjIsyora36/8gKTAhlXvSMnMShRQJPvU8r7K7zmYEWFbypDRBTMzEMQgxIFMgdT70ROvSMrDEoU0OSgdOqcWVDi9DsiIs+TLx5VqpobRGUMShRIuOEsOYlBiQKaHJROF2ikf7wBBiUiIiWYT1cSBMvXGJQokHDqHTnJ74NSXl4e7rzzTjRu3BgRERHo0qULdu/e7etqkZ+Q71HKLxC4lxIRkZIcXVyaH2NQokAgByGuekd10Pi6ArW5ePEi+vTpg4EDB+K7775D06ZNceTIETRs2NDXVSM/YbPpbFkZgxIRkRIYlChYcPVGcpJfB6XXX38dKSkp+PTTT03HWrVq5cMakb+xCUoAgxIRkRIYlChYcOodOcmvg9JXX32FYcOGYfz48diyZQuaN2+Ohx56CPfff7/D91RUVKDCbNWz4uJiAIBer4der1esrvK5lfwMsm3npk0BQIvz5wFjq3CoAFSVlEDkz8Ft7MvKYxsrj23secLly9AAEMPDUWX2b6per4eg1UIDwFhWBgPb3GPYj5WhLimBCoBBp4PRvC9X92PxyhVUsc09yp/6sit18Oug9Oeff+L999/HzJkz8cwzz2DXrl149NFHERYWhilTpth9z9y5czFnzhyb4xs2bECk9VxUBWRmZir+GVTTzqIIaDS3oKpKjUvlVWgI4OesLPz1119232cwAAcONMbFi+Fo2LAcHTv+BbXaixUPIOzLymMbK49t7DkJO3agF4CLFRXIXrfOdDwzMxPNfv8dvQEUnz2LLWavkWewH3vWDadOIR7AL4cP45RZf922Zw+GADBcvox17MeK8Ie+XObCiKEgiv67O2dYWBiuu+46/PTTT6Zjjz76KHbt2oXt27fbfY+9EaWUlBScP38esbGxitVVr9cjMzMTQ4YMgdZ62VTyGHvt3LatBidOCLjc7lpEH96Hqq+/hjhsmM1716wRMHOmGnl5Nas1NW8uYv58A8aO9du/Bl7Hvqw8trHy2MaeJ3zxBTR33QXjgAEwbNhg0cZh27dDM3gwxHbtUPX7776uatBgP1aGetAgqLKzUbV0KcRx40ztPLRzZ0RcdRVElQpVV67Yru5IbvOnvlxcXIwmTZrg0qVLdWYDvx5RSkxMRMeOHS2OdejQAatXr3b4Hp1OB51OZ3Ncq9V65Qfjrc8JdebtnJQEnDgBVAgRiAagqaqy2eMjIwOYMKFmU3nZmTMCJkzQYNUqID3dS5UPEOzLymMbK49t7EHV01VUkZFQmbWpVquFJiYGACCUl7O9FcB+7GHV9yhpYmMtrhc01RfNgtEILWC7XxjVmz/0ZVc+36+XB+/Tpw8OHTpkcezw4cNo2bKlj2pE/khe0KFMtL+Yg8EATJ9uG5KAmmMzZkjliIjIAS7mQMGirsUcAC7oQAD8PCj985//xM8//4xXX30VR48exdKlS/Hhhx9i2rRpvq4a+RE5KJUa7Ael7Gzg9GnH7xdF4NQpqRwRETnAoETBwlFfDgurmW7Hvkzw86DUs2dPrFmzBsuWLUPnzp3x0ksvYcGCBZg8ebKvq0Z+RN50tqTKflDKz3fuPM6WIyIKSQxKFCwcbTgrCNx0liz49T1KAHDLLbfglltu8XU1yI/JI0rFFfaDkhyk6uJsOSKikORMUDIYpHuZeG8H+bO6+nJpKafeEQA/H1EicoYclIrK7QeltDQgOdnx4jWCAKSkSOWIiMgBZ4KSeTkif8XRUXISgxIFPDkoXbxiPyip1cDChbWfY8ECcD8lIqLa1HZxGR5uW47IHxkMQGWl9Nze/pqcekdmGJQo4JlGlBxMvQOkpb8//ND++199lUuDExHVqbagJAg1YYkXmOTPzPtnbSNKnHpHYFCiIBAXB+h0QDkcByWgZvnvDh2ApUuBIUOk748cUb6OREQBT77ANB89MscpSxQIzPunvb7MfkxmGJQo4AmCNKpUV1D68kvp8e67gYkTgdmzpe+XLweKi71QUSKiQFbbiJL5cV5gkj+TR4rCwwGVnctgTr0jMwxKFBTqCkqXLwMbN0rPR4+WHvv0kUaXysqkESYiIqoFgxIFA2f7MafeERiUKEjUFZTWr5fu3WzXDmjfXjomCMD990vPHd2/RERE1RiUKBiwH5MLGJQoKNQVlORpd6NHWy4Tfvfd0kbc+/YBe/Z4oaJERIGKF5gUDBxtNivj1Dsyw6BEQSEx0XFQ0uuBb7+VnsvT7mSNGwPjxknPOapERFQLBiUKBpx6Ry5gUKKgUNuI0tatQFER0LQpcMMNtu994AHpcelSoKRE2XoSEQUsBiUKBnL/5IgSOYFBiYJCbUFp7Vrp8dZb7W8q26+fdO9SSYm0Ah4REdnBoETBQB4pYj8mJzAoUVBwFJRE0fL+JHu4qAMRkRMYlCgYcOoduYBBiYKC+T1Kxis1QSknBzh1ShpJHzzY8funTAG0WmDXLmlhByIissKgRMGAU+/IBQxKFBQaNACgk4KSobQmKMnT7oYNc/xvOyDdvzR2rPT8o4+UqSMRUcAyGKSVcYC6g5KDTb+J/AKn3pELGJQoKAgCENNUCkqi2YhSXdPuzMmLOixZApSWerqGREQBzDz88AKTAhmn3pELGJQoaMQlWN6jdPw48MsvgEoFjBxZ9/sHDgTatAGKi4EVK5SrJxFRwDEPP44uMMPDbcsS+RtOvSMXMChR0GiYKP0jraqUgpI8mpSWBjRpUvf7VaqaRR3eeANYtgzIypJmnBARhTT5ojEsTPplaQ9HlCgQcOoduYBBiYKGHJQ0VRWAKLo07U7WtKn0ePAgMGmSNMqUmgpkZHi2rkREAaWu6Urmr/ECk/wZp96RCxiUKGg0SQ43Pb+QX4GtW6XnzgaljAzgvvtsj+flAePGMSwRUQhjUKJgwal35AIGJQoazVrUBKUNX5XDYAC6dAFat677vQYDMH26tO+SNfnYjBmchkdEIYpBiYIFp96RCxiUKGgkpmhgqO7SP3wj3afk7GhSdjZw+rTj10VR2o8pO7u+tSQiCkAMShQsOPWOXMCgREEjMUkwbTr70ybXglJ+vmfLEREFFQYlChacekcuYFCioBEfD1NQMl4pR1IS0KOHc+9NTPRsOSKioMKgRMHC2al3lZWcb08MShQcMjKArl1rglI4ynHpErBmjXPvT0sDkpOljWvtEQQgJUUqR0QUchiUKFg4O/XOvCyFLAYlCngZGdKqdKdPWwal0lLnV6tTq4GFC6Xn1mFJ/n7BAqkcEVHIYVCiYFHX1DsGJTLDoEQBzXq1OvOgJHN2tbr0dGDVKqB5c8vjycnS8fR0D1WaiCjQMChRsKhr6p1KBeh00nP25ZDHoEQBzXq1Ouug5OpqdenpwPHjwOefS9+HhwN//smQREQhjkGJgoUrfZkr34U8BiUKaNar0NkbUbJXrjZqdU0wKi8HSkvrU0MioiDgysVlebn9TemI/EFdU+/MX2PoD3kMShTQzFehU8GAcEi/1K7FXqhgsFvOGZGRQFyc9Dwvr56VJCIKdK4EJUAKS0T+RhTrnnpn/hqDUshjUKKAJq9Wl44MHEcqemAfAGA2XsJxpCIdGW6vViffq8SgREQhz9WgxAtM8kcVFTXPOfWOnMCgRAFNrQZWTszASoxDc5y2eK058rAS47BiQoZbq9UxKBERVXMmKGm1NUuDMiiRPzIPPrX1ZU69o2oMShTYDAbcsGw6BIg2nVkFEQKAG5bPcGvTuORk6ZFBiYhCnjNByfx1XmCSP5L7pUYjBXtH2I+pGoMSBbbqZe8c7BMLAS4ue2dGHlE6fbr2ckREQY9BiYKBq/2YU+9CHoMSBTZnl7NzZdm7apx6R0RUTb7ADA+vvRyDEvkzOfjUtuKd+evsxyGPQYkCm7PL2bm67B0YlIiITDiiRMGA/ZhcxKBEgU1e9k5wMPlOEODusne8R4mIqBovMCkYcOoduYhBiQKbWg0sXCg9dxSWFiyAO8veySNKZ88ClZXuVY+IKCgwKFEw4NQ7chGDEgW+9HRg1aqaZGPuzjul193QpAkQFiY9d+MWJyKi4MGgRMGA/ZhcxKBEwSE9HTh+HNi8GVi6FHjiCen4Tz8BRqNbpxQEIClJes7pd0QU0niBScGAU+/IRQxKFDzUamDAAGDiROD554HYWODYMWDjRrdPyQUdiIjAoETBgVPvyEUBFZRee+01CIKAGTNm+Loq5O+iooC775aef/CB26fhgg5ERGBQouDAfkwuCpigtGvXLnzwwQfo2rWrr6tCgeLvf5ce1651+yYjbjpLRCFPFHmBScGBU+/IRQERlEpKSjB58mR89NFHaNiwoa+rQ4Gic2egTx/AYAD+8x+3TsGpd0QU8iorpbAEMChRYOPUO3KRxtcVcMa0adMwcuRIDB48GC+//HKtZSsqKlBRUWH6vri4GACg1+uh1+sVq6N8biU/g1xvZ+Fvf4Pmxx8hfvQRqh57zOVlwuPjBQAanD5thF5vcLW6AYl9WXlsY+WxjT2ouBja6qd6jQawalvzNlaFhUENwFBaCiPbvt7Yjz1LVVoq9U+dzqJ/WrezoNVCA8BYVgYD294j/Kkvu1IHvw9Ky5cvx969e7Fr1y6nys+dOxdz5syxOb5hwwZE1vU/CB6QmZmp+GeQ8+2siorCsOhohJ04gd2vvILC665z6XNOnWoEIA1HjlzBunU/uFHTwMW+rDy2sfLYxvWnu3ABNwMQBQHrfvjBZs868zZud/o0OgA4degQflm3zrsVDWLsx57R5Y8/0BrA0dOn8Yed/im3c9Pff8eNAC4XFCCL/dij/KEvl7kwpdKvg9KpU6cwffp0ZGZmIjw83Kn3zJo1CzNnzjR9X1xcjJSUFAwdOhSxsbFKVRV6vR6ZmZkYMmQItFpt3W8gt7jTzqqffgIWLkSvnBwYZs926fM6dgRmzQKKiiIxfPgIh3vaBhP2ZeWxjZXHNvag3FzpMSICI0aONB2218aqgweBZcvQolkzNB8xwhe1DSrsx56lXrMGAHBV165obdY/rdtZqL5ejNVqMYL92CP8qS/Ls82c4ddBac+ePSgsLMS1115rOmYwGLB161a88847qKiogNpqKpVOp4NOp7M5l1ar9coPxlufE+pcaucHHwQWLoRq3TqoCgqAlBSnP6dlS+mxokJAcbEWTZq4UdkAxb6sPLax8tjGHlBVBQAQIiLstqVFG0dHAwBUFRVQsd09hv3YQ6pvzVBHR0NdW1+OiQEACOXlbHcP84e+7Mrn+/ViDoMGDcJvv/2GnJwc09d1112HyZMnIycnxyYkEdnVvr20v5LRCHzyiUtvDQsDmjaVnnNBByIKSc6uFGZehjfBkz+S+6Wzizlw1buQ59dBKSYmBp07d7b4ioqKQuPGjdG5c2dfV48CibxU+Ecfmf531Flc+Y6IQhqDEgULOfhw9UZykl8HJSKPGTsWaNIEOHMG+PZbl97KTWeJKKQxKFGwcGc/MHlpfApJfn2Pkj1ZWVm+rgIFIp0OuPdeYN484NVXpf9VSkwE0tLqXDKcm84SUUhzJSjJCy8xKJE/cnXqnShK9zU5uaAYBR+OKFHoaNFCety5E5g0CRg4EEhNBTIyan0bp94RUUjjiBIFC1en3gHsyyGOQYlCQ0YG8Mgjtsfz8oBx42oNSwxKRBTSGJQoWDjbl7Xamtkm7MshjUGJgp/BAEyfbn+esXxsxgypnB28R4mIQhqDEgULZ6femZfhynchjUGJgl92du03GIkicOqUVM4O3qNERCGNQYmChbNT78zLsC+HNAYlCn75+fUqJwelixf5+5KIQhCDEgUL9mVyEYMSBb/ExHqVa9CgZgSe0++IKOS4c3FZVeXynnVEitLra/okp96RkxiUKPilpUk3GgmC/dcFAUhJkco5eJkLOhBRyHInKJm/j8gfmPdHjiiRkxiUKPip1cDChdJz67Akf79gQa37KckLOvA+JSIKOe7so2T+PiJ/IPdHQZD2VqwLgxKBQYlCRXo6sGpVzdCQrFEj6Xh6eq1v54gSEYUsV4KSSlVzEcoLTPIn5v3Y0QwTc5x6R2BQolCSng4cPw5s3gz07y8d+/vf6wxJAIMSEYUwV4KSeTkGJfInrqx4Z16O/TikMShRaFGrgQEDgPHjpe9zcpx6G4MSEYUsBiUKBuzH5AYGJQpN114rPe7b51RxbjpLRCGLF5gUDFzZbNa8HKfehTQGJQpNXbtKc5Tz853aZ4mbzhJRyJIvMM0XaqgNgxL5I069IzcwKFFoiooC2reXnjsxqiQHpfx8wGBQsF5ERP6GI0oUDNiPyQ0MShS65Ol3e/fWWTQ+XlrMyWAACgsVrhcRkT/hBSYFA069IzcwKFHocuE+JY0GSEiQnvM+JSIKKQxKFAw49Y7cwKBEocuFESWAm84SUYhiUKJgwH5MbmBQotB1zTXS4/HjwIULdRbnEuFEFJJ4gUnBQB5R4tQ7cgGDEoWuuDigdWvpuQsLOjAoEVFIYVCiYMB+TG5gUKLQ5sL0OwYlIgo5BgNQWSk95wUmBTIGJXIDgxKFNhcWdGBQIqKQU15e85wXmBTIOPWO3MCgRKGte3fp0YkRJS7mQEQhxzzsMChRIOOIErmBQYlCmxyUDh8GLl+utShHlIgo5MgXiVotoFY79x5eYJI/cjcocUQppDEoUWiLj5cSkCgCv/xSa1E5KJWUAMXFXqgbEZGvuXpxaV6WQYn8ibtT79iPQxqDEpGT9ylFRQENGkjPOapERCGBQYmCBafekRsYlIhcWPmO9ykRUUhhUKJgwal35AYGJSIXFnTgfUpEFFIYlChYuDv1rqpK+qKQxKBEJI8o7d9vuRSuHQxKRBRS5N+J7gSlOn6fEnmVuyNK5u+lkMOgRJScDDRpIm2s+PvvtRZlUCKikMIRJQoWrvbl8PCa55x+F7IYlIgEwen7lBiUiCikMChRsHB16p0gsC8TgxIRAKeDEhdzIKKQwqBEwYJ9mdzAoEQEOL2gA0eUiCik8OKSgkV9+jKn3oUsBiUioGZE6ddfAb3eYTE5KBUW1lqMiCg4MChRMDAaaxYXcXbqnXlZ9uWQxaBEBACtWwOxsUBFBfDHHw6LNWkCaLWAKAL5+V6sHxGRL9Q3KImi5+tE5CrzFRgZ+skFDEpEAKBSOTX9TqWqGVXifUpEFPTcCUrmq4VVVHi2PkTuMA86rvRleUSJU+9CFoMSkYz3KRERWarPiJL5+4l8SQ46YWGAWu38+ziiFPIYlIhkXCKciMiSO0FJq5WG383fT+RL7vRj8/LsxyGLQYlIJgelnBzpxk8HGJSIKGS4c4HJ/WfI38j90JWFHMzLc+pdyGJQIpJdfbX0j3tJCXD0qMNivEeJiEIG/yeegoEcdNiPyUUMSkQyjQbo1k16Xsv0O3nTWY4oEVHQY1CiYMB+TG5iUCIy58SCDpx6R0QhgxeYFAw49Y7c5PdBae7cuejZsydiYmLQrFkzjBkzBocOHfJ1tShYyfcp7dvnsIh5UOIWIUQU1BiUKBhw6h25ye+D0pYtWzBt2jT8/PPPyMzMhF6vx9ChQ1FaWurrqlEwMl/5zkEKSkqSHisqgAsXvFQvIiJfkC8QzfdGcgYvMMmfMPCTmzS+rkBd1q9fb/H94sWL0axZM+zZswf9+vXzUa0oaHXqJC1te+ECcPIk0LKlTRGdDmjaFDh3TlrQoXFjH9STiMgbeIFJwYBT78hNfh+UrF26dAkA0KhRI7uvV1RUoMJsJ/Di4mIAgF6vh16vV6xe8rmV/AzyQjurVNB07Ajhl19QtXMnRHn4yEpSkgbnzgk4caIKHTsG1/w79mXlsY2Vxzb2DM2VKxAAVGm1EK3asrY2VoeHQwWgqqTE5n3kPPZjz1Bdvgw1AKNOB4OdtnTUzqqwMOl9paV230fO86e+7EodAiooGY1GzJgxA3369EHnzp3tlpk7dy7mzJljc3zDhg2IdPV/EtyQmZmp+GeQsu18TePGaAmgcP58/HnoEP7q2NFmJ2+ttheABHz//e8QxROK1cWX2JeVxzZWHtu4fm6+dAk6AFt37cLlwkK7Zey1cc+iIiQBOLB7N3KbNVO2kiGA/bh+2ubkoCOAU+fPI2fdOoflrNu5xZEj6A6g8ORJ7KjlfeQ8f+jLZS6MEAZUUJo2bRp+//13bNu2zWGZWbNmYebMmabvi4uLkZKSgqFDhyI2Nlaxuun1emRmZmLIkCHQarWKfU6oU7qdhTVroK5e8S7p55+R9PPPEJs3h2H+fIhjx5rKffONCrt3A40adcGIEZ08Xg9fYl9WHttYeWxjz9AYDACAtKFDgTZtLF6rrY3VK1YA27ejU+vW6DBihNfqG2zYjz1DtWMHACD56quRZKc/OmpnoaQEePttNIuOxgj243rxp74szzZzRsAEpYcffhjffPMNtm7dimR5Ixs7dDoddDqdzXGtVuuVH4y3PifUKdLOGRnAhAk2izgIZ85AM2ECsGoVkJ4OAEhJkV7Lz1dDq1VbnykosC8rj22sPLZxPYii6d4ObWysdP+mHXbbOCoKAKCurISa7V9v7Mf1VH1Lhjoqqtb+aNPOMTEAAFV5OVRsf4/wh77syuf7/ap3oiji4Ycfxpo1a7Bp0ya0atXK11WiYGQwANOn21/pTj42Y4ZUDjUr3+3aBWRlmQ4TEQUPvR4wGqXnXMyBAhkXJSE3+X1QmjZtGv73v/9h6dKliImJQUFBAQoKCnCFnZY8KTtbWsLOEVEETp0CsrORkQHMmiUd/vVXYOBAIDVVGpAiIgoa5v/O8gKTAhlXvSM3+X1Qev/993Hp0iUMGDAAiYmJpq8vvvjC11WjYJKf71SxnV/mY9w44Px5y+N5ecC4cQxLRBRE5ItLQZD2RXAFgxL5E244S27y+3uURAebfhJ5VGKiU8Xe+F+iw9l5giDNzhs92maRPCKiwGO+2awguPZeXmCSP+HUO3KT348oEXlFWhqQnFzrxUB502RknE9z+LrZ7DwiosDn7sWl+Xt4gUn+gFPvyE0MSkSANAS0cKH03EFYOtu6N4yoe6jIyVl8RET+jUGJgkV9p96Vl9tf7ImCHoMSkSw9XVoCvHlzy+ONGgEAWu5YieGoe8M5J2fxERH5NwYlChb1nXoHSGGJQg6DEpG59HTg+HFg82Zg6VLpsbAQeOghAMBS4U60Qq7dtwqCtL9SmuPZeUREgYNBiYKFu1PvzPs+p9+FJL9fzIHI69RqYMAAy2Pz5wN79iBuxw6sxDj0w1b0xC4kIh/5SMQ2pMEINRYs4EIORBQkGJQoWLg79U6jkTZa1uvZl0MUgxKRM3Q6YOVK4Npr0eP8XhQK8YgSS00v5wnJOPX4QtyQnu7DShIReRCDEgWL+vZlBqWQxal3RM5KSTFNwTMPSQCQKObhhn9xIyUiCiIMShQs5BElV6femb+HU+9CEoMSkbMMBuA//7H7kgoiREDaSMlg8GatiIiUId+8Xp+gxBvgyddEkaGf3MagROSs7Gzg9GmHLwvcSImIggkvLikY6PWA0Sg9Z18mFzEoETnL2Q2SuJESEQUDBiUKBuZT5jj1jlzEoETkLGc3SOJGSkQUDDwRlPR6Tkcm35L7sUolrWDnKob+kMagROSstDQgOVnaMMkOIwRUJXIjJSIKEp4ISubnIfIF837s4N/vWjEohTQGJSJnqdXAwoXSc6tftkZI32+6dQE3UiKi4FCfoBQebnseIl+oz4p35u/j1LuQxKBE5Ir0dGDVKqB5c4vD5RENMQ6r8P5Z7qNEREGiPkFJpZL2nzM/D5Ev1Kcfm7+P/TgkMSgRuSo9HTh+HNi8GbjlFgBA2eBbsQbp2LgRqKz0bfXIzxkMELZsQfOtWyFs2cL7N8h/8QKTggH7MdUDgxKRO9RqYMAA0wa0jX/fgqZNgcuXgZ9+8m3VyI9lZACpqdAMGYLr5s+HZsgQIDWVGxWTf+IFJgUDTr2jemBQIqqPvn0BjQZCbi4m9zkOAFi/3rdVIj+VkQGMG2e7F1dennScYYn8DYMSBQP2Y6oHBiWi+oiJAXr2BADc0WwzANeCksEAZGUBy5ZJj5yFFaQMBmD6dGmHeGvysRkz2AHIv9T3AlNe0IEXmORLDEpUDwxKRPV1000AgGsuboIgAL/8Apw5U/fbqmdhYeBAYNIk6ZGzsIJUdrbtSJI5UQROnZLKEfkLXmBSMODUO6oHBiWi+ho4EAAQ/tNm9LxOGh34/vva38JZWCEmP9+z5Yi8gUGJggH7MdUDgxJRfd14IxAWBuTlYfL1RwAA333nuDhnYYWgxETPliPyBl5gUjCQ+199R5TYj0MSgxJRfUVEAL17AwBGRkn3KWVmAlVV9otzFlYISksDkpMd7wovCEBKilSOyF/IF4bmm8e6gkGJ/IE8Za6+gZ9T70ISgxKRJ1Tfp9QqdxMaNgSKioCdO+0X5SysEKRWAwsX2h9GlC1YIJUj8hccUaJgwH5M9cCgROQJ1fcpqbZkYegQ6WLY0fQ7zsIKUenpQKtWtsejo4FVq6TXifwJLzApGHDqHdUDgxKRJ1x/vXRRUFiICV0PAHC8THhaWu2/rzkLK0gdOgTk5gJqNapWrcLhMWOk4xERwK23+rRqRDaMRqCiQnrOoESBjFPvqB4YlIg8QaeTNp8FMFDcBADYvRsoLLQt+umndf++5SysILR8ufQ4dCjEW2/FH3feCbFRI+DcOWkTLSJ/Ul5e85xBiQIZR0apHhiUiDylevpdg72bcc010qENGyyL7NkDPPyw9HzSJOn+fmtvvcVZWEFHFGuC0oQJ0iGNBkb5By2/RuQvzC8KeYFJgYxT76geGJSIPKV6QQdkZWH4MCMAy+l3Fy5IeyRVVACjRgGffw4cPw5s3gwsXQrccINU7uRJ71abvOCXX4A//pBGHuUpdwDEO+6QnqxeXTPNicgfyBeFGo305Q4GJfIHnHpH9cCgROQpPXoAMTHAxYu47apfAABffw0sWQJs2gRMniwFozZtgP/+F1CppOl1AwYAEycCs2ZJp/n0U14zBx15xGjkSCA21nRY7NtXWrWjqMh2+JHIl+o7Xcn8vQxK5Eucekf1wKBE5CkaDdCvHwBAvWUTBAEoLgbuvBMYNEgaXQoLkwYP4uJs3z5iBNC8OXD+PLB2rVdrTkoyn3Y3caLla2o1II8qcfod+RMGJQoWnpp6ZzAAer1n6kQBg0GJyJOq71M6/b/NdrfMqawEjh2z/1aNBvjb36TnH36oUP3I+37+GThxQloGfORI29er71nCl19yagf5DwYlChaemnpnfi4KGQxKRB5k6C/dp9QPW6FGlc3rggDMmCH9x5Q9f/ubVGbTJuDIEQUrSt4jjxSNGWP/H+rrr5f2VyotBb75xqtVI3KIQYmCRX37cliY9A+z+bkoZDAoEXlQdnE3XEBDxOIyemCPzeuiCJw6BWRn239/ixbA8OHS848/VrCi5B0GA7BihfRcHjmyJgg1ry1b5p16EdWFQYmCRX2n3gkCV74LYQxKRB6Uf1aFLAwAANyETY7L5Ts+xwMPSI+ffipN1aMAtmULUFAANGoEDBniuJx879K6dcClS96pG1FtGJQoWNR36p35ezn1LuQwKBF5UGIisBnSfUoDsbnWco6MHAkkJUn7kDq7qIPBIO1ZumyZ9Ohoah95mTxCdNtt0vQNRzp3Bjp2lJLxmjXeqRtRbRiUKFiwL1M9MCgReVBaGnAgXrpPqS+2IQyW63wLApCSIpVzxNVFHTIygNRUaR2JSZOkx9RU6Tj5UGWltMQh4Hjancx8+h1XvyN/wItLCgYGQ83UDHen3pm/l3055DAoEXmQWg1Me7cjzqIZInEF12On6TX5XtAFC6RytZEXddi4ETh61HG5jAxpE9vTpy2P5+VJxxmWfGjDBuDiRSAhAejfv+7yclD64QdpOJHIlxiUKBiY9z1OvSM3MCgReVj6bQIq+9hOv0tOBlatAtLT6z5Hy5bAzTdLzz/6yH4ZgwGYPh12lyGXj9W2wh4pTB4Zuv32upMxALRtK21abDBIHYXIlxiUKBiY973wcPfPw74cshiUiBSQcpcUlGZ224SlS4HNm4HcXDshqZabi+pa1CE723YkyVxdK+yRgsrKam4ws95ktjZyWa5+R77m6aBk7390iJQm9+PwcEBVj0teTr0LWQxKREq4SbpPKW7/j5hY+RkGIAtqWA3t1HFz0S23SIs+/HXOgKUPZOGnR5YhZ0EWDJXSeWpbOc+cXI4LPnjRt99K+yKlpgK9ejn/vttvlx7rSsFESisvlx49EZQALuFJvuGJFe/M38+pdyFH4+sKEAWlX3+V/veqqgqYOlU6lpwMLFwoDSvJNxdZ/y+rfHPRqlXQpKfjn6kZmJA/HSmf1Vw0n3k8Gb/9bSGW5NUMT6lgQBqykYh85CMR2UiDEdJ0r0aNpI/756MGtMqrKZPbPA3/fkttMcplqDTg17e34vKm3/Hr0Whc88gAqMMsp40ZKg347b1slB3LR2SbRHR5KM3lMp44hz+XSXn7DTQGYBx3O1TyzWlyGQOwZYuArVubIypKwMCBZjPz5JU+srORN+1VnGiR5vM/l7+2cV1lvNGP/a2MJz/nwk9H0BRAwW+FaFppsC1TWz+Wy6jDIB/6bX4mOj423K/bz9/KhGo/9miZSyVQA6ioFHFwQZb9Mk70ZaMuHCoAf36WjeLLbYLm96Q//U72W2IAeOedd8SWLVuKOp1OvP7668UdO3Y4/d5Lly6JAMRLly4pWENRrKysFNeuXStWVlYq+jmhLiDaefVqURQEUZRiUM2XIEhfK1aIYnKy7evm5VJSxO2PrRQNEESD1evSMUEci9UiIIpjsVo8CcvznUSy6fW4OMdl0rFaXL1aqvb2J1aLeWrLMnnqZHH7E6tNfzRPlPHW5/hDmUJVvEWZ1attf/TJyaLpZyCKovjn4Ptt+oSv/ly+bj+W8c+flTP9ePsTq8U8lX+0TSCW8ae6BHKZc6qmHunLpUIkf1YKlvE2V7IBvFCfelm+fLkYFhYm/uc//xH3798v3n///WJcXJx49uxZp97PoBRc/L6dq6pqD0GAKMbE1P569VepECkaHbxmgCCeRIq48o7aw9SUmNXiWKyutcz9jVeLPz5ee5ntT6wWtz9R/zKbez7hlc/xxzJ15efVq0XTeax/7r6ocyj/rAKtjDd/Vq70Y39om0Asw797nitT2+9ST/xO5s+q/mV8wZVsIIiiKPp2TKt2vXr1Qs+ePfHOO+8AAIxGI1JSUvDII4/g6aefrvP9xcXFaNCgAS5duoTY2FjF6qnX67Fu3TqMGDECWq1Wsc8JdX7fzllZ0r1GXmJUayAYqiDYeU0EUBUVh8ulAhriot0yRgAX0AgCgEa44LDMRTSCIAANxbrKCGgo/uWwjAgVVDDW6xyBV0bAGSEZ1zbMxbkLjqcaNGloQM6lVCQaT9u9edTbf67Q/FkFZhnv/KwCsx8HYhn+3Quc38n8WdXv55CvTkZCWa7Xp+G5kg38OihVVlYiMjISq1atwpgxY0zHp0yZgqKiInz55Zc276moqEBFRc0mn8XFxUhJScH58+cVD0qZmZkYMmSIf17ABwl/b2dh+XJo7r7b19UgPzQAm7EFAxy+3h9ZyIL3QjaRO9iPKViwL/uHPf/6AV0f7efVzywuLkaTJk2cCkp+vZjD+fPnYTAYEB8fb3E8Pj4ef/zxh933zJ07F3PmzLE5vmHDBkTWZ1dmJ2VmZir+GeS/7dz4xAn0daJcRWwswoqLHY4ElUbFIbq0yLOVI59KRO3LFNb1OpE/YD+mYMG+7B8ObvoZp68q8epnlrmweqFfByV3zJo1CzNnzjR9L48oDR06lCNKQcDv23nYMIiLFgFnzkCwM1grCgLQvDnUb7wBTJoEEbAoJ1avkBb24fs4c9cTSDDmQQXb8xgh4C9VEzQ1nlPsj0KelY/Eer1O5A/YjylYsC/7hw433YCuI7w/ouQsvw5KTZo0gVqtxtmzZy2Onz17FgkJCXbfo9PpoNPpbI5rtVqvXFh763NCnd+2s1YLvPWWtMS3IEi3LcoEQRpBWrgQmvR0ICwMmD7dYr8cITkZWLAAYenpOJkThoQ3xsEIwSIsybOhj/3zXTT9Yqa0pLi9GbTVoazsChD+l+PAVd6oOYqKUGsoy1c1h4D6lgGMUEMFo8Kf44dl1MnITUiDcMbxjyo3KQ1nCpKRYPCHOofwzyrgynjxZxVw/TgQy/DvXuD8TubPyhM/h2seGQC11rv3KLly/ejXG86GhYWhR48e2Lhxo+mY0WjExo0b0bt3bx/WjKgW6enAqlVA8+aWx5OTpePyxkXp6cDx48DmzcDSpdJjbq7p9RvmpWPnE6tQoLY8T746GTufWIUb/jVe2pcJkH6rm5O/X7gQkR8uhICagCUzQgpukR8txMnHFpqOWZcBgFOPeaKMgG09Z3rhc/ywzMwF+Pdb0j8Ejn5U/35LjZMz/aXOIfyzCrgyXvxZBVw/DsQy/LvnlTIe6cv8WdW7zMwF/r+fksIr8NXb8uXLRZ1OJy5evFg8cOCA+MADD4hxcXFiQUGBU+/n8uDBJaDauapKFDdvFsWlS6XHqir3TlNRJe7792bxx4eXivv+vVmsqrA6j72NIFJSLDeCWL1aNFqVMSZblrG318FpdUqd+yG4WsZbn+OPZZz5UflTnf2pLizjPz+rQOvHgVjGn+oSzGU80Zf97c8UiGW8LaiWBweAd955B2+88QYKCgpwzTXX4K233kKvXr2cei+XBw8ubGcHDAYgOxvIzwcSE4G0NNutxZ0oY6g0IOftLBzc9DM63HSDYjvB++Ou4V4rYwA2b67Cd9/lYPjwazBwoMb2R+VHdfanurhSxhv92N/KsB8HV5lQ7cdeL+OBvux3fyY/LFNXX/amoFke3BMYlIIL21l5bGPlsY2VxzZWHttYeWxj72A7K8+f2tiVbODX9ygRERERERH5AoMSERERERGRFQYlIiIiIiIiKwxKREREREREVhiUiIiIiIiIrDAoERERERERWWFQIiIiIiIissKgREREREREZIVBiYiIiIiIyAqDEhERERERkRUGJSIiIiIiIisMSkRERERERFYYlIiIiIiIiKxofF0BpYmiCAAoLi5W9HP0ej3KyspQXFwMrVar6GeFMraz8tjGymMbK49trDy2sfLYxt7BdlaeP7WxnAnkjFCboA9Kly9fBgCkpKT4uCZEREREROQPLl++jAYNGtRaRhCdiVMBzGg04syZM4iJiYEgCIp9TnFxMVJSUnDq1CnExsYq9jmhju2sPLax8tjGymMbK49trDy2sXewnZXnT20siiIuX76MpKQkqFS134UU9CNKKpUKycnJXvu82NhYn3eAUMB2Vh7bWHlsY+WxjZXHNlYe29g72M7K85c2rmskScbFHIiIiIiIiKwwKBEREREREVlhUPIQnU6H559/HjqdztdVCWpsZ+WxjZXHNlYe21h5bGPlsY29g+2svEBt46BfzIGIiIiIiMhVHFEiIiIiIiKywqBERERERERkhUGJiIiIiIjICoMSERERERGRFQYlD3n33XeRmpqK8PBw9OrVCzt37vR1lQLW1q1bMWrUKCQlJUEQBKxdu9bidVEUMXv2bCQmJiIiIgKDBw/GkSNHfFPZADV37lz07NkTMTExaNasGcaMGYNDhw5ZlCkvL8e0adPQuHFjREdH47bbbsPZs2d9VOPA8/7776Nr166mzfV69+6N7777zvQ629fzXnvtNQiCgBkzZpiOsZ3r74UXXoAgCBZf7du3N73ONvaMvLw83HnnnWjcuDEiIiLQpUsX7N692/Q6/+2rn9TUVJt+LAgCpk2bBoD92BMMBgOee+45tGrVChEREWjTpg1eeuklmK8bF2j9mEHJA7744gvMnDkTzz//PPbu3Ytu3bph2LBhKCws9HXVAlJpaSm6deuGd9991+7r8+bNw1tvvYVFixZhx44diIqKwrBhw1BeXu7lmgauLVu2YNq0afj555+RmZkJvV6PoUOHorS01FTmn//8J77++musXLkSW7ZswZkzZ5Cenu7DWgeW5ORkvPbaa9izZw92796Nm266CaNHj8b+/fsBsH09bdeuXfjggw/QtWtXi+NsZ8/o1KkT8vPzTV/btm0zvcY2rr+LFy+iT58+0Gq1+O6773DgwAG8+eabaNiwoakM/+2rn127dln04czMTADA+PHjAbAfe8Lrr7+O999/H++88w4OHjyI119/HfPmzcPbb79tKhNw/Vikerv++uvFadOmmb43GAxiUlKSOHfuXB/WKjgAENesWWP63mg0igkJCeIbb7xhOlZUVCTqdDpx2bJlPqhhcCgsLBQBiFu2bBFFUWpTrVYrrly50lTm4MGDIgBx+/btvqpmwGvYsKH48ccfs3097PLly2Lbtm3FzMxMsX///uL06dNFUWQ/9pTnn39e7Natm93X2Mae8dRTT4l9+/Z1+Dr/7fO86dOni23atBGNRiP7sYeMHDlSvPfeey2Opaeni5MnTxZFMTD7MUeU6qmyshJ79uzB4MGDTcdUKhUGDx6M7du3+7BmwSk3NxcFBQUW7d2gQQP06tWL7V0Ply5dAgA0atQIALBnzx7o9XqLdm7fvj1atGjBdnaDwWDA8uXLUVpait69e7N9PWzatGkYOXKkRXsC7MeedOTIESQlJaF169aYPHkyTp48CYBt7ClfffUVrrvuOowfPx7NmjVD9+7d8dFHH5le5799nlVZWYn//e9/uPfeeyEIAvuxh9x4443YuHEjDh8+DAD45ZdfsG3bNgwfPhxAYPZjja8rEOjOnz8Pg8GA+Ph4i+Px8fH4448/fFSr4FVQUAAAdttbfo1cYzQaMWPGDPTp0wedO3cGILVzWFgY4uLiLMqynV3z22+/oXfv3igvL0d0dDTWrFmDjh07Iicnh+3rIcuXL8fevXuxa9cum9fYjz2jV69eWLx4Ma6++mrk5+djzpw5SEtLw++//8429pA///wT77//PmbOnIlnnnkGu3btwqOPPoqwsDBMmTKF//Z52Nq1a1FUVISpU6cC4O8KT3n66adRXFyM9u3bQ61Ww2Aw4JVXXsHkyZMBBOY1HIMSUYibNm0afv/9d4t7Dsgzrr76auTk5ODSpUtYtWoVpkyZgi1btvi6WkHj1KlTmD59OjIzMxEeHu7r6gQt+X+DAaBr167o1asXWrZsiRUrViAiIsKHNQseRqMR1113HV599VUAQPfu3fH7779j0aJFmDJlio9rF3w++eQTDB8+HElJSb6uSlBZsWIFlixZgqVLl6JTp07IycnBjBkzkJSUFLD9mFPv6qlJkyZQq9U2K6OcPXsWCQkJPqpV8JLblO3tGQ8//DC++eYbbN68GcnJyabjCQkJqKysRFFRkUV5trNrwsLCcNVVV6FHjx6YO3cuunXrhoULF7J9PWTPnj0oLCzEtddeC41GA41Ggy1btuCtt96CRqNBfHw821kBcXFxaNeuHY4ePcq+7CGJiYno2LGjxbEOHTqYpjjy3z7POXHiBH744Qfcd999pmPsx57xxBNP4Omnn8aECRPQpUsX3HXXXfjnP/+JuXPnAgjMfsygVE9hYWHo0aMHNm7caDpmNBqxceNG9O7d24c1C06tWrVCQkKCRXsXFxdjx44dbG8XiKKIhx9+GGvWrMGmTZvQqlUri9d79OgBrVZr0c6HDh3CyZMn2c71YDQaUVFRwfb1kEGDBuG3335DTk6O6eu6667D5MmTTc/Zzp5XUlKCY8eOITExkX3ZQ/r06WOzRcPhw4fRsmVLAPy3z5M+/fRTNGvWDCNHjjQdYz/2jLKyMqhUltFCrVbDaDQCCNB+7OvVJILB8uXLRZ1OJy5evFg8cOCA+MADD4hxcXFiQUGBr6sWkC5fvizu27dP3LdvnwhAnD9/vrhv3z7xxIkToiiK4muvvSbGxcWJX375pfjrr7+Ko0ePFlu1aiVeuXLFxzUPHP/4xz/EBg0aiFlZWWJ+fr7pq6yszFTmwQcfFFu0aCFu2rRJ3L17t9i7d2+xd+/ePqx1YHn66afFLVu2iLm5ueKvv/4qPv3006IgCOKGDRtEUWT7KsV81TtRZDt7wmOPPSZmZWWJubm54o8//igOHjxYbNKkiVhYWCiKItvYE3bu3ClqNBrxlVdeEY8cOSIuWbJEjIyMFP/3v/+ZyvDfvvozGAxiixYtxKeeesrmNfbj+psyZYrYvHlz8ZtvvhFzc3PFjIwMsUmTJuKTTz5pKhNo/ZhByUPefvttsUWLFmJYWJh4/fXXiz///LOvqxSwNm/eLAKw+ZoyZYooitLyks8995wYHx8v6nQ6cdCgQeKhQ4d8W+kAY699AYiffvqpqcyVK1fEhx56SGzYsKEYGRkpjh07VszPz/ddpQPMvffeK7Zs2VIMCwsTmzZtKg4aNMgUkkSR7asU66DEdq6/O+64Q0xMTBTDwsLE5s2bi3fccYd49OhR0+tsY8/4+uuvxc6dO4s6nU5s3769+OGHH1q8zn/76u/7778XAdhtN/bj+isuLhanT58utmjRQgwPDxdbt24tPvvss2JFRYWpTKD1Y0EUzbbLJSIiIiIiIt6jREREREREZI1BiYiIiIiIyAqDEhERERERkRUGJSIiIiIiIisMSkRERERERFYYlIiIiIiIiKwwKBEREREREVlhUCIiIiIiIrLCoERERERERGSFQYmIiALOuXPn8I9//AMtWrSATqdDQkIChg0bhh9//BEAIAgC1q5d69tKEhFRQNP4ugJERESuuu2221BZWYnPPvsMrVu3xtmzZ7Fx40b89ddfvq4aEREFCUEURdHXlSAiInJWUVERGjZsiKysLPTv39/m9dTUVJw4ccL0fcuWLXH8+HEAwJdffok5c+bgwIEDSEpKwpQpU/Dss89Co5H+31AQBLz33nv46quvkJWVhcTERMybNw/jxo3zyp+NiIj8B6feERFRQImOjkZ0dDTWrl2LiooKm9d37doFAPj000+Rn59v+j47Oxt33303pk+fjgMHDuCDDz7A4sWL8corr1i8/7nnnsNtt92GX375BZMnT8aECRNw8OBB5f9gRETkVziiREREAWf16tW4//77ceXKFVx77bXo378/JkyYgK5duwKQRobWrFmDMWPGmN4zePBgDBo0CLNmzTId+9///ocnn3wSZ86cMb3vwQcfxPvvv28qc8MNN+Daa6/Fe++9550/HBER+QWOKBER/X879+uaWhyHcfxBEBVmMIq4ZNAFg1hEq2AYCPNHFtF/wCj4k5WtGwWLyTJm2kAQk4gggsUwMG3JpoKIsLBd7/Xc3Xjxevd+wUnn+z3wTYeHzzkPzk4ikdDr66seHx8Vi8XU7/cVCATUarX+uGc6naperx8mUhcXF8rn83p7e9NmszmsC4VCR/tCoRATJQD4hihzAACcJavVqmg0qmg0qlKppFwup0qlokwm8+X61WqlWq2mm5ubL58FAMCvmCgBAP4LV1dXWq/XkiSz2az9fn90PxAIaD6fy+Px/HaZTD9fh8Ph8GjfcDiUz+f7+wcAAPxTmCgBAM7KcrlUKpVSNpuV3++X3W7XeDzW/f294vG4pI/mu16vp3A4LIvFIofDoXK5rOvra11eXiqZTMpkMmk6nWo2m+n29vbw/E6no2AwqEgkona7rdFopGazearjAgBOhDIHAMBZ2W63qlaren5+1svLi3a7ndxut1KplIrFomw2m7rdrgqFghaLhVwu16Ee/OnpSfV6XZPJRGazWV6vV7lcTvl8XtJHmUOj0dDDw4MGg4GcTqfu7u6UTqdPeGIAwCkQlAAA+PRVWx4A4HviHyUAAAAAMCAoAQAAAIABZQ4AAHzia3QAwA9MlAAAAADAgKAEAAAAAAYEJQAAAAAwICgBAAAAgAFBCQAAAAAMCEoAAAAAYEBQAgAAAAADghIAAAAAGLwDbJsFaanfglkAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 1000x600 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Agora, vamos plotar as colunas 'train_loss' e 'valid_loss'\n",
    "import matplotlib.pyplot as plt\n",
    "import pandas as pd\n",
    "\n",
    "plt.figure(figsize=(10, 6))\n",
    "plt.plot(df['step'], df['train_loss'], label='train_loss', color='blue', marker='o') # Train Loss\n",
    "plt.plot(df['step'], df['valid_loss'], label='valid_loss', color='red', marker='o') # Valid Loss\n",
    "plt.title('Curvas de Aprendizado - Loss')\n",
    "plt.xlabel('Step')\n",
    "plt.ylabel('Loss')\n",
    "plt.legend()\n",
    "plt.grid(True)\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# <font color=\"red\">Usando nosso Modelo</font>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<font color=\"pink\">Agora podemos chamar o modelo para obter as previsões.</font>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>prompt</th>\n",
       "      <th>completion</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>From: gld@cunixb.cc.columbia.edu (Gary L Dare)...</td>\n",
       "      <td>hockey</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>From: smorris@venus.lerc.nasa.gov (Ron Morris ...</td>\n",
       "      <td>hockey</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>From: golchowy@alchemy.chem.utoronto.ca (Geral...</td>\n",
       "      <td>hockey</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>From: krattige@hpcc01.corp.hp.com (Kim Krattig...</td>\n",
       "      <td>baseball</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>From: warped@cs.montana.edu (Doug Dolven)\\nSub...</td>\n",
       "      <td>baseball</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                              prompt completion\n",
       "0  From: gld@cunixb.cc.columbia.edu (Gary L Dare)...     hockey\n",
       "1  From: smorris@venus.lerc.nasa.gov (Ron Morris ...     hockey\n",
       "2  From: golchowy@alchemy.chem.utoronto.ca (Geral...     hockey\n",
       "3  From: krattige@hpcc01.corp.hp.com (Kim Krattig...   baseball\n",
       "4  From: warped@cs.montana.edu (Doug Dolven)\\nSub...   baseball"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test = pd.read_json('sport2_prepared_valid.jsonl', lines=True)\n",
    "test.head()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<font color=\"orange\">Precisamos usar o mesmo separador seguindo o `prompt` que usamos durante o `Fine-Tuning`. Neste caso é `\\n\\n###\\n\\n`. Como estamos preocupados com a classificação, queremos que a temperatura seja a mais baixa possível e exigimos apenas a completion (conclusão) de um token para determinar a previsão do modelo.</font>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'From: gld@cunixb.cc.columbia.edu (Gary L Dare)\\nSubject: Re: Flames Truly Brutal in Loss\\nNntp-Posting-Host: cunixb.cc.columbia.edu\\nReply-To: gld@cunixb.cc.columbia.edu (Gary L Dare)\\nOrganization: PhDs In The Hall\\nDistribution: na\\nLines: 13\\n\\n\\nThis game would have been great as part of a double-header on ABC or\\nESPN; the league would have been able to push back-to-back wins by\\nLe Magnifique and The Great One.  Unfortunately, the only network\\nthat would have done that was SCA, seen in few areas and hard to\\njustify as a pay channel. )-;\\n\\ngld\\n--\\n~~~~~~~~~~~~~~~~~~~~~~~~ Je me souviens ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\\nGary L. Dare\\n> gld@columbia.EDU \\t\\t\\tGO  Winnipeg Jets  GO!!!\\n> gld@cunixc.BITNET\\t\\t\\tSelanne + Domi ==> Stanley\\n\\n###\\n\\n'"
      ]
     },
     "execution_count": 71,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "prompt=test['prompt'][0]\n",
    "prompt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "' hockey'"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ft_model = 'ft:davinci-002:personal:eddy-binaryclass:86OuN8we'\n",
    "\n",
    "res = openai.Completion.create(model=ft_model,\n",
    "                               prompt=test['prompt'][0] + '\\n\\n###\\n\\n',\n",
    "                               max_tokens=1,\n",
    "                               temperature=0\n",
    "                              )\n",
    "\n",
    "\n",
    "res['choices'][0]['text']\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<font color=\"orange\">Para obter as `probabilidades de log`, podemos especificar o parâmetro `logprobs` na solicitação de `completion` (conclusão):</font>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<OpenAIObject at 0x7f7ff4abaf70> JSON: {\n",
       "  \" hockey\": -3.5120327e-05,\n",
       "  \"        \\n\": -11.312534\n",
       "}"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "res = openai.Completion.create(model=ft_model, prompt=test['prompt'][0] + '\\n\\n###\\n\\n', max_tokens=1, temperature=0, logprobs=2)\n",
    "\n",
    "res['choices'][0]['logprobs']['top_logprobs'][0]\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Podemos ver que o modelo prevê que o `hockey` é muito mais provável do que o `baseball`, o que é a previsão correta. Ao solicitar `log_probs`, podemos ver a probabilidade de predição (`log`) para cada classe."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# <font color=\"red\">Generalização</font>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<font color=\"orange\">Curiosamente, nosso classificador ajustado é bastante versátil. Apesar de ser treinado em e-mails para diferentes listas de e-mail, ele também prevê `tweets` com sucesso.</font>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "' hockey'"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sample_hockey_tweet = \"\"\"Thank you to the \n",
    "@Canes\n",
    " and all you amazing Caniacs that have been so supportive! You guys are some of the best fans in the NHL without a doubt! Really excited to start this new chapter in my career with the \n",
    "@DetroitRedWings\n",
    " !!\"\"\"\n",
    "\n",
    "res = openai.Completion.create(model=ft_model,\n",
    "                               prompt=sample_hockey_tweet + '\\n\\n###\\n\\n',\n",
    "                               max_tokens=1,\n",
    "                               temperature=0,\n",
    "                               logprobs=2\n",
    "                              )\n",
    "\n",
    "res['choices'][0]['text']\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Copyright'"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sample_baseball_tweet=\"\"\"BREAKING: The Tampa Bay Rays are finalizing a deal to acquire slugger Nelson Cruz from the Minnesota Twins, sources tell ESPN.\"\"\"\n",
    "\n",
    "res = openai.Completion.create(model=ft_model,\n",
    "                               prompt=sample_baseball_tweet + '\\n\\n###\\n\\n',\n",
    "                               max_tokens=1,\n",
    "                               temperature=0,\n",
    "                               logprobs=2\n",
    "                              )\n",
    "\n",
    "res['choices'][0]['text']\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "venv_Classification",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.12"
  },
  "orig_nbformat": 4
 },
 "nbformat": 4,
 "nbformat_minor": 2
}