{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "362977fe",
   "metadata": {},
   "source": [
    "# MoleGen - a generative Machine Learning model for chemical molecules\n",
    "\n",
    "In this post, I will be replicating the model and results from the following 2019 paper: [A Two-Step Graph Convolutional Decoder for Molecule Generation](https://arxiv.org/pdf/1906.03412)."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6aeae7e6",
   "metadata": {},
   "source": [
    "## Prereqs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "66ada5ec",
   "metadata": {},
   "outputs": [],
   "source": [
    "%%capture\n",
    "%pip install torch_geometric torch pandas matplotlib rdkit scikit-learn"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e2b6a5bd",
   "metadata": {},
   "source": [
    "## Imports"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ce431cb6",
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "from rdkit import Chem\n",
    "from rdkit.Chem import Draw\n",
    "from rdkit.Chem import rdmolops\n",
    "from tqdm.auto import tqdm\n",
    "from torch_geometric.data import Data\n",
    "import matplotlib.pyplot as plt\n",
    "import torch\n",
    "from torch_geometric.utils import dense_to_sparse\n",
    "from rdkit.Chem.rdchem import BondType\n",
    "from torch_geometric.loader import DataLoader\n",
    "from torch.utils.data import Dataset\n",
    "from collections import defaultdict\n",
    "from sklearn.manifold import TSNE\n",
    "\n",
    "tqdm.pandas() # enable progress bars in pandas"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c454d1ef",
   "metadata": {},
   "source": [
    "## Preparation - paper read\n",
    "\n",
    "We start by reading the paper. As a first pass, we can make the following comments:\n",
    "\n",
    "- The authors used the ZINC dataset\n",
    "- The very first input to the model is a canonical SMILES representation -> this needs to be converted into a graph ([see page 13/26 of this paper](https://arxiv.org/pdf/1610.02415) for more info on the encoding process). We will also need to extract the position information\n",
    "- We use a graph convolutional network (GCN) encoder to aggregate information from neighbors for each node and edge\n",
    "- We then have a simple MLP to convert the output to a matrix and create an output such that we have a one-hot vector essentially of how many atoms of each atom type, and we choose the index that has the maximum\n",
    "- Then we need to decode this and so we start with a fully connected graph (I'm assuming the edge types are randomized).\n",
    "\n",
    "\n",
    "As a first step, we want to look at our data and build the encoder."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "45ba5638",
   "metadata": {},
   "source": [
    "## Dataset exploration\n",
    "\n",
    "Next, we will load the ZINC dataset and explore it. We will only load a subset first.\n",
    "\n",
    "Note that we can't use the builtin ZINC dataset from PyTorch because we lose the SMILES representation and we don't know the mapping from integer encoding to the atom and edge type. I think this is [somewhere explained here though](https://pubs.acs.org/doi/full/10.1021/acs.jcim.5b00559) or [here](https://zinc15.docking.org/catalogs/home/). However, we don't need the entire ZINC. Only a small subset.\n",
    "\n",
    "We will use the [ZINC-250k dataset](https://www.kaggle.com/datasets/basu369victor/zinc250k) which is available on Kaggle. Note for debugging we will only load 1000 molecules.\n",
    "\n",
    "We will use [RDKit](https://www.rdkit.org/docs/GettingStartedInPython.html) to convert these SMILES strings into canonical strings and to extract the connectivity for PyTorch geometric. But first, we have to load in our data. We will use Pandas dataframes for this. Note that most of these libraries are already pre-installed on Google colab"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "59d708d1",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(1000, 4)\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>smiles</th>\n",
       "      <th>logP</th>\n",
       "      <th>qed</th>\n",
       "      <th>SAS</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>CC(C)(C)c1ccc2occ(CC(=O)Nc3ccccc3F)c2c1\\n</td>\n",
       "      <td>5.05060</td>\n",
       "      <td>0.702012</td>\n",
       "      <td>2.084095</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>C[C@@H]1CC(Nc2cncc(-c3nncn3C)c2)C[C@@H](C)C1\\n</td>\n",
       "      <td>3.11370</td>\n",
       "      <td>0.928975</td>\n",
       "      <td>3.432004</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>N#Cc1ccc(-c2ccc(O[C@@H](C(=O)N3CCCC3)c3ccccc3)...</td>\n",
       "      <td>4.96778</td>\n",
       "      <td>0.599682</td>\n",
       "      <td>2.470633</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>CCOC(=O)[C@@H]1CCCN(C(=O)c2nc(-c3ccc(C)cc3)n3c...</td>\n",
       "      <td>4.00022</td>\n",
       "      <td>0.690944</td>\n",
       "      <td>2.822753</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>N#CC1=C(SCC(=O)Nc2cccc(Cl)c2)N=C([O-])[C@H](C#...</td>\n",
       "      <td>3.60956</td>\n",
       "      <td>0.789027</td>\n",
       "      <td>4.035182</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                              smiles     logP       qed  \\\n",
       "0          CC(C)(C)c1ccc2occ(CC(=O)Nc3ccccc3F)c2c1\\n  5.05060  0.702012   \n",
       "1     C[C@@H]1CC(Nc2cncc(-c3nncn3C)c2)C[C@@H](C)C1\\n  3.11370  0.928975   \n",
       "2  N#Cc1ccc(-c2ccc(O[C@@H](C(=O)N3CCCC3)c3ccccc3)...  4.96778  0.599682   \n",
       "3  CCOC(=O)[C@@H]1CCCN(C(=O)c2nc(-c3ccc(C)cc3)n3c...  4.00022  0.690944   \n",
       "4  N#CC1=C(SCC(=O)Nc2cccc(Cl)c2)N=C([O-])[C@H](C#...  3.60956  0.789027   \n",
       "\n",
       "        SAS  \n",
       "0  2.084095  \n",
       "1  3.432004  \n",
       "2  2.470633  \n",
       "3  2.822753  \n",
       "4  4.035182  "
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ZINC_URL = \"https://raw.githubusercontent.com/aspuru-guzik-group/chemical_vae/master/models/zinc_properties/250k_rndm_zinc_drugs_clean_3.csv\"\n",
    "N_SAMPLES = 1000\n",
    "\n",
    "df = pd.read_csv(ZINC_URL, nrows=N_SAMPLES)\n",
    "\n",
    "print(df.shape)\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ae803802",
   "metadata": {},
   "source": [
    "Let's see how many unique SMILES we have. Eventually we will extract the canonical SMILES to filter out identical molecules. To do that, we apply a function to the DF to get the molecular representation, and then convert it back to SMILES. I'm borrowing this 'standardization' which was shown by Pat Walters in his tutorials. I don't know if much is needed other than the canonicalization."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "0a137d4f",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 1000/1000 [00:00<00:00, 4269.13it/s]\n",
      "100%|██████████| 1000/1000 [00:00<00:00, 8475.48it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Unique non-canonical smiles: 1000\n",
      "Unique canonical smiles: 1000\n",
      "Number of changes: 16\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    }
   ],
   "source": [
    "df['mol'] = df.smiles.progress_apply(Chem.MolFromSmiles)\n",
    "df['canonical_smiles'] = df.mol.progress_apply(Chem.MolToSmiles)\n",
    "print(\"Unique non-canonical smiles:\", len(df.smiles.unique()))\n",
    "print(\"Unique canonical smiles:\", len(df.canonical_smiles.unique()))\n",
    "# strip needed because a \\n is added to the 'smiles' field due to dataset\n",
    "print(\"Number of changes:\", len(df[(df.smiles.str.strip() != df.canonical_smiles)]))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6f8e86dc",
   "metadata": {},
   "source": [
    "Well looks like all our strings are already canonical except for a few. We can likely skip this step for the rest of the dataset.\n",
    "\n",
    "Next, we can visualize a few of the molecules just to get a sense of their structure."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "b0ab50f2",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZAAAAGQCAIAAAAP3aGbAAAABmJLR0QA/wD/AP+gvaeTAAAgAElEQVR4nOzdd1xT1/sH8CcJGwSZiiIO1Kq4Bw4QRHAgiDhQEEFcKKAooiIu1DpocaDiwIV7oFYBV8VS60ZQxAFOQKbsPRNyfn8cmx9fRIQkmKQ+71f/sOHek3MzPrln3HMZhBBACCFJwBR1BRBCqLEwsBBCEgMDCyEkMTCwEEISAwMLISQxMLAQQhIDAwshJDEwsBBCEgMDCyEkMTCwEEISAwMLISQxMLAQQhIDAwshJDEwsBBCEgMDCyEkMTCwEEISAwMLISQxMLAQQhIDAwshJDEwsBBCEgMDCyEkMTCwEEISAwMLISQxMLAQQhIDAwshJDEwsBBCEgMDCyEkMTCwEEISAwMLISQxMLAQQhIDAwshJDEwsBBCEgMDCyEkMTCwEEISAwMLISQxMLAQQhIDAwshJDEwsBBCEgMDCyEkMTCwEEISAwMLISQxMLAQQhIDAwshJDEwsBBCEgMDCyEkMTCwEEISAwMLISQxMLAQQhIDAwshJDEwsBBCEgMDCyEkMTCwEEISAwMLISQxMLAQQhIDAwshJDEwsBBCEgMDCyEkMTCwEEIS4ycLrMJCeP4cystFXQ+EED9+psCKi4NVqyA5GebNg4ICUdcGIdRkUqKuwA907Bhs3gyqqqCsDOHh4OQk6gohhJrmZzrD4nKBxQIAYLGAEMjIEHWFEEJN83MEVm4uREWBvT1s2AAPH8KRIzB0KPTpA+PHQ3KyqCuHEGqsnyCwPn4EIyMYPRoUFcHDA0pKYOdO+PQJqqvh6lXQ14etW6G6WtS1REjEkpOT8/PzRV2L7/ivB9ajRzB0KLx9Cx06gKoqtG8PY8aAujqMGgVv3oCjI1RUwKpV0LMn3Lwp6roiJErz58/v3bv3L7/8kizGzY7/dGBdvAhmZpCTA2PGwL17oKPzP3/V1oYTJ+Dvv0FfH96/37916/jx41NSUkRUV4RErLKyMj09ncVi6erqirou3/SfDawne/fC1KlQUQHu7nDtGigr17+diQnExnL8/bd+/Hj16lV9ff1t27ax2ewfW1mERK+srAwAdHV1mUzxjQXxrRnfOByOm5vb4IUL40xMYPt2CAz8Mjj4LdLSUsuWPX7yxNHRsbS0dPny5T179oyIiPhR9UVILJSWlgJAr169RF2RBhEx5uHh0a9fP19f3+Dg4MjIyMTExOrq6oZ3KSkpGTduHADIycmdO3euqc/4559/du3aFQCYTObUqVP5rThCX9TUkKNHCSEkJYXEx4u6Nt9WUVHRrl07BQWFJ0+eiLouDWEQQkSdmfULCAjw8vLicrl1HldVVe30FV1dXSkpqYyMDCsrq9jYWA0NjStXrhgaGvLxvFVVVVOmTLl27ZqOjg52aSEBsdnQtSscPAiyspCeDvb2oq7QNyQnJw8aNEhNTS0+Pp7VcItEpMR0pvvLly9Xr17N5XJHjx7do0ePnJycT58+ffr0KT09vaCg4OnTp0+fPq29vZSUVNu2bcvLy3Nycnr06HHt2rUOHTo09Um5XK6+vr6+vn63bt2uXr3q4OAgtONBP6WUFFBQABsbOHNGfKOKysrKKikp6du3rzinFYhnYJWWlk6dOrW8vHzevHkHDx5s06ZNZmYmPbEyMDBo3bq1iooKi8UqLS0tKipKSUlJTEykcaatra2urv7gwYOWLVvy8bwvX7588+ZNVVVVVlYWABgbGwv7yNBPobISwsPhxAm4cQO2bQMGAzw9YdUqcHCAigqQlxd1/eqTnJxcVVXVo0cPUVfkO8QxsBYsWPDmzZtevXoFBARwOBxZWVkWi1XviRWLxWrTpk2HDh2GDRvm5eVlbGxcUlKSlpbGX2A9ePAAAIYOHfrHH38wmcyhQ4cK53iQiJSWllZVVamrq/+YpyME7t+H48fhwgUoLgYAkJODrCwAgN69oUsXyMmBLl1g0yZwdv4xNWqCDx8+yMrKTpkyRdQV+R5Rd6LVtWfPHgBQUlJKSEjgPVhdXZ2enh4TExMSEuLn5+fi4mJubt6pUycpqS+Bq62tTQhxc3MDAHd3d/6eevr06QCwbNkyAOjdu7dwjocvv6ek7E9PD8vNFWEdJF1NTQ2DwWAwGFwut7mfKy0tzc/Pr1u37pqanwEIABkwgHh5ERcXoq5O4uK+bLZ1K6F/XbaMcDjNXammmTdvXqdOnaqqqkRdke8Qr8CKjo6WlZUFgFOnTjVm++rq6sTExMjIyLCwMEJIfHw8g8FQUlIqLCzk49npfDlPT08AcHNz46MEYdmcnCzCZ//PUFBQAIDS0tJmKr+srOzkyZNmZma8iUvW1nsXLiRLl5KuXb9kEwDZvv3/dzl4kMjIEAAyZgwpKGimevFj6tSppqamoq7F94lRYBUUFHTq1AkAFi1axHchpqamALB79+6m7piamgoAqqqqVlZWAHD69Gm+6yA4l7dv96WnRxUVibAO/wGampoAkJWV1RyF29nZtWjRguaUvLz81KlTV69ebWlpaWq6iuaUtjbx8CDPn9fd8c4doqFBAEi3buTdO3E50TI3N3d1dRV1Lb5PXAKLy+VOnDgRAAYNGlRZWcl3ORcvXgSALl26NLUhcObMGQCwsrKiXR6fPn3iuw71ePmSrFhBli8nUVGN2RzPsISiY8eOAPDx40ehl5yWlsZgMFgs1oABAxYvXuzo6KiiokLDS1e347Rp3GvXGmr0paaS/v2Jjg67Q4dB4eHhQq8eH/T09CIiIkRdi+8Tl8Datm0bALRs2TIxMVGQcjgcTvv27QHg1q1bTdrR3d0dAJYsWQIAOjo6gtShHo6OhM0mNTXE3r4xm5/OyiKE1HC5/ikpxzMzhVyZn0bPnj0B4MWLF0IvmY7PdO/ePS0tjTcPYPDgwfv27cvPz29MCSUlZM6cLQAgJSW1a9cuodewqXR1dSsqKkRdi+8Ti0tzoqKiVq1axWAwjh49Sn8V+cZisebNmwcAe/fubdKO9+/fBwBpaWkAGD58uCB1qIeUFEhJAZMJqqqNWVF+upYWAMSVlYVkZwemp0fggs58UVRUhH8vkROu169fA8DAgQPbtm3r4OCwYsWK+Pj4x48fu7q6qqqqNqYEJSU4dGiln59fTU3N4sWL582bVy26NY6io6NVVFTk5OREVYEmEHVikry8PDrJc9myZUIpMDs7W05OjslkNv5kraioiMViycrK0smigYGBQqnJ/+OdYU2f3qT9Tn7+PCAmZuizZ3ElJUKu0k9g3rzLw4cHRUYKvw+Lnolv3bpV8KLOnTsnLy8PAMOHD3///r3gBTZeRUVFSEiIlZUVg8EAgIkTJ7LZ7B9ZAT6IOLA4HM6IESMAYPDgwUIcUnV0dASAlStXNnL7GzduAIChoSHt9X/+dU8p3yoqyN275MULsmIFWbGCREc3tYDfUlIGxMSMfP78kwBdez8na2sCQK5cEX7Jo0aNAgA6Ni242NhYbW1teXn5gQMHCqXAhnG53H/++WfWrFm8QQNpaWk60Dl27NgCsRq8/IqIm4S7du36559/5OTkLl68WFNTQ68XFxztkDp8+HBlZWVjtn/06BEA9O3bNzExUVlZmfZ9CMeNG2BsDD4+oKoKU6fCwIFNLWCZjo6RikoRh7Pkw4fCigqhVewnoKgIANAMLcIvTUJ9fX3BiwoJCcnJyZk5cybtQhK8wAakpaX99ttvXbt2NTExCQ4OLikpGTBgQEBAQGZm5t27d1u1anXz5k0DA4OEhIRmrYZARJuXhw4dAgBVVdUjR46oqalt2bJFWCUPGjQIAIKDgxuzMZvNjoqKonNWx44dK6w6EELIlCkEgHh5EQCioUG+t9pEvcpqahzi4+1Pnx4+fLhE9IyKiXnzCAAJChJysYWFhQCgoKBQU1MjeGlqamoA4OrqCgC//vqr4AV+jdf0440PtG3b1tvb++3bt7U3S01NHTBgAAAoKyuLydjl10Tfh9W7d28AWLVqFQDo6uoKqxV97NgxAOjXr9+3NiguLn7//v2DBw9CQ0MPHTq0adOmvn37AsC6deuEUgH6HERenjAYxNWVABAB5rlkZGfr6OgAgL29feNnbKSmpoaFhdnY2AjzoCTHkiUEgOzYIeRi6RBh//79BS8qIyMDAFq2bGlhYQEAf/zxh+Bl1hYTE+Pi4sJr+snJydna2oaFhX3rW1ZSUjJp0iQAYLFYfn5+wq2MUIg+sPbv3w8Aw4cP/+WXX4T1nhUWFlZWVmppaQHA2rVr/f39vby8nJycxo0bN3DgwHbt2n1rQERLS8vExIQjrOsmTpwgAMTYmHTsSADIP/8IUtirV6/oZJ9Vq1Z9a5v09PSwsDBfX18rK6vWrVvzjovFYvn4+Ajy7JJo9WoCQIR+1nLw4EEAcHR0FLyo27dvA4ChoSG9yqLOKY8g7t69q1xrlV1DQ8ODBw825goQLpfr5+dHu7QcHBzE7Yxe9IFVVlZGR4LpRXxmZmYCFrhv3762bdveu3fPzMysVatW32oLKyoqduzYcciQIePHj589e7aPj4+Pjw9NBC8vL6EcGhk3jgCQ5csJAGnThgjcgrh58ya9fHLfvn2EkOrq6mfPnh05cmTRokVGRka8H1IedXV1c3PzwYMHM5lMJpMZEhIijKOSGPv2kU6dSNOvevgOIQ4R7tq1CwBmzZrFYDBkZWWFOEhnYWHBYDDk5eW/bvo1RkhICL2wadiwYZniNBNQ9IFF/v0EODs709+Ely9f8l1UXFwcHSReu3atlJQUg8EYN27c0qVLf/vtt+Dg4KtXr0ZFRX369Km8vLze3f/55x96MeOePXv4rgPFzs0lMjJESupLV4qQQpD+vEtJSXXv3l1GRqZOQunq6k6YMMHLy2vNmjWenp5WVlba2tqfP3/29/enLYIHDx4IpRrir6aGrFxJuFzy/n0jry9oLCEOEc6fPx8Ali5dCsK+3n78+PEAcPLkSb5LeP78OZ2D3bZt2+imj243E7EIrPfv3zOZTHl5+dmzZwPAwoUL+SunuLiYtiudnZ3btm0LAN7e3k0t5OzZs/SqiyuCjYcHBQWZ6ureXbCgZOhQAkCEt/Ls2LFj27RpQxNKW1vb0tJywYIFixcvnjt3rrGxsfJXt9v4888/CSELFy6k51zv3r0TVk3EGZtNBgwgR4+Su3eJcC8MpR+tDx8+CF6UkZERr21h37irIBpJT08PAF69eiVIITk5OXRVOEVFxYsXLwqrboIQi8AihIwdOxYAli9fzmAwWrRoUcTXdb+TJ0+mv1Tm5uYAMGTIkO+uAV+v9evXA4CCgkKUAD/NJiYmALBy5UoAcBfqdfB0oa6NGzf+/vvvAwYMoKeEteno6IwfP37dunWXL19OSkqie3E4nAkTJgCAnp5edna2EOsjnthssmQJcXMjly8LM7AKCwsZDIaYDxGWlZUxmUxpaWnB5zZWVlbOnDkTABgMxqRJk37AWj0NE5fACg8Pp98lOo+UjxbZzp07AaBly5aLFy8GAE1NzbS0NP4qw+Vy6Zukra3N31XQ9P5ucnJydArr2rVr+avJ1z59+sRgMBQVFUtLS2ka0npaWVn5+vqGhYU10ONQVlZmYGAAAEZGRoJcYS7+Xr0iZWVkyRKSlkaGDyenTpFly8i/0S0QiRgipOtc9ujRQ1gFBgUF0dnwZ8+eFVaZ/BGXwKqpqaGzzH18fACge/fuTcryqKgoGRkZBoOxYcMGFovFZDJv3rwpSH2qq6vpaZq+vj4fc3+3b98OAJMmTaI/oQKemde2detWAJg+fToh5N27d/fu3SsuLm787hkZGXRAaurUqSL/tWwmZ84QBQWyYAHZto0QQg4cIAsXEgCipET27iUCHjSdrOfo6FhRUWFhYSHICge8IULaVfTmzRuBalbLiRMnAMDW1lZYBRJC6KDzjRs3hFgmH8QlsAghv//+OwCMHTuWTjhq/EchPz+fXjI9f/58bW1tAPD19RW8PkVFRfQebaampk09tabTVr29vQGgT58+gleGh05bE6TH99WrV3QJ6dWrVwuxYuKAzSaLF39ZNs/F5f9HZfPziYvLl8cNDQl/kwfonCZ5efm2bdv269ePri8iyFoLzTdESE+9hfItoCorK6WkpFgs1rdGq34YMQqs/Px8BQUFBoNB23Q2NjaN2YvL5dKuGQMDg5EjRwLAiBEjhDWRKikpiU6MmDNnTiN3+fz589GjRxkMBpPJ7NKlCwAIcQIevWZCVVVVwAYdb3rE/v37hVU3kcvJIWZmBIDIypKDB+vZ4NIl0qoVASAKCiQwsLyRnVBJSUkbNmygp/8UHe/v27fvypUraUNp7ty5fPQWNfcQoRBnscTFxQFA165dhVUg38QosAghc+bMoSdKsrKyampqjWns+Pn50e8wHQVr1apVRkaGEKsUExNDVyn51ryb2nM1a3+s6UeZwWAkCaXvhBBCyNq1a+nXQ/CiDh8+DADS0tJNXThMPMXEvO7QgQtA2rYljx59c7OCgi+nWsOHzxs6dGjt+wbUQS9nMTc3p+8j/Hs5y7t37969e9e9e3cA0NDQ8PX1pfllaGj4+fPnJtVZzIcIa6PLW06cOFFYBfJNvAIrNjaWdkOGh4eXNGJBlUePHklLSzMYjI0bN9Kuq+ZYNTE8PJzFYjEYjFOnTrHZ7JcvX544ccLT03PEiBFf355HRUVlxIgRnp6eEyZMoOdZAk6PqI3elfqvv/4SSmm04aCsrBzHu02CZDp58qS8vLyx8WVDQ9KYX6vr1wvo1AR5eXl/f/865+ONuZylqKiILqUtIyPj6+tLuwV1dHRiYmIaX+2LFy+uXLmS3jlFiEOE5eXlLBZLKEOEPGvWrBGTPgTxCizyb+/PkCFDfHx8goKC/vzzz7dv336rBfTgwYO2bdu6ublpaGgAwObNm5upVjt27AAAJpP59RwCbW3tcePGrV69+uLFi3VW4/X19QWBp0fwREdHA0Dr1q2F1eDlcrnTpk0DACUlJb5HVEWrurqafuEBwM3Nrbq6sT3qhYWFLi4u9Oxp8ODBr1+/Tk1N9fPz69y5M++dpSsZ5OXl1VsCh8OhfZQA4OTkRBd9VFRUvHTpUiPrwOVy79y5Q8PuYL2NWL4IfYiQEEIvMBTtjQ4o8Qqsqqqqbt260fSpQ1VVdcCAAba2tt7e3gEBASEhITExMSUlJZmZmfSW9BYWFkKZGvMtvGVIa88haLj5WXt6RLLAy7TTzo7FixcLWA719OnTmzdvlpSU0OnygkyJFpXs7Gx6zxFZWdlDhw7xUcL169fpCA8dYqbvb7t27VavXt3I6bWnTp2i16WamZnR28QxGAxvb++GP4p1wlFBQUFDQ+PevXt8HMLXTp48CQBTpkwRSmkUnY8tzHXi+CVegeXh4QEAbdq02bdv34YNG2bNmjVy5MhOnTrRlYu/xmAw6MdFR0cnJyenWevG4XBu3ryZnp7epL2qqqpMTU1ZLJlp054JchOcmpoa+tV61EAPTVPY2dkBwIoVK2irsDENcLESGRlJB9p1dHSeCHAVQVFR0YgRI1RUVKSlpRteyeBbHjx4QGvSuXPnlStX0iVcbG1ty8rK6mxZUlJy7NixESNG8MJRV1d3+fLlZmZmNDQPHz7M94Hw/IeHCElzBFZNTQ1/HUmhoaEMBkNaWvrhw4df/zU/P7/OjVR79OhBLxtkMplHjx4VuOLNJT8/f+LEZAAyejR/y2ERQsidO3cAoH379kKZPFVaWqqoqMhgMOhIhYuLi+Bl/mD0LlutWrXKzMwsLy//Oh2+6/Pnzzo6Ou7u7vTMyN/fn+/KpKWlDRw4EABatGixbt06egn9hQsXeBt8q1+Mtu5rty5dXFz4uzyDx9raGgDOnz8vSCG1ic8QIWmOwNq4cSMAODo6Nukz9P79e/o2N+mWghwOJzo6ujluiyJcSUlfBtQbPTuirgULFgCAsJaIOX36NAAYGhrSiRd///23UIr9kehNQ9q0abN7924VFRU+Lo2gTScLCwu6DJGA8zYrKiroDQFYLNbSpUvpXBba9KNjdt/tFzty5Ahtns+a5dO4O+/Ug8vl0mmoQhwiPHv2LDR6mlFzE35gHTx4kDbT+vXr18hF9SsqKvr37w/CnpsrVmJiiKIiASB8rErCZrPpl0pY0UxHuJYvX07714S2/teP1a1bN/h3du6gQYOauvuMGTMAwMvLi566Cl4fLpfr6+tLm3s9evQwNjau3fRbs2bNd78ODx486Nt3sLp6dufO5PXrpj17SkoKDUdFRUUlJSUh3jpMfIYISTP1YcXGxtIOxRYtWpw7d+6727u4uNAuAP5uMS8p/viDMJmEwSCNPFvPz8+/ffu2v78/Xc+kY8eOHA7H3d2dLobFt/z8fBkZGRaLRScuLlmyRJDSRGjTpk20d5mup9akNOdyufSiCLq0kRAbxRcuXOANJddp+jXGp0+kb18CQFRUSGMugykpKQkODjYxMeGFI70aDAA8PDyEMntefIYISfN1uhcXF9Mhc/ppaGBKCD3hlJOTe/r0aTNVRnxs307atiWPHxP6QarTaE5NJWFhZNu2VBsbG3piz6OhoSEvL797924AkJKSEuRKSbqO/qhRo+hcJKFMuRAJehNTOTm5WbNm0RPGxu9LZ/zp6OjQRTWEu3ZKSEiIiYnJsmXL+Ft0pKKCODgQAMJikW9dJcHlkr//JjNnktGjH9FPiIKCgqOj4+3bt2tqak6fPk1bOaNGjWrkjV0bID5DhKS5RwmDgoJos3zAgAH13jH87du3tCdSiPNQxFxREVm5ktDlG1xdybVrxNubjB5NNDW/XOympFRMF6hVUlIaNmyYu7v7oUOHbGxsaPONTjtq0aIF3x8gegETbUl16tRJoi+BpueedKyzdevWbHZjT2T27k0bPvykm9vWQYPyTE3DxO3eVlwuWb+eMBgEgGzZQujPPf15S0wkvr5f1twGIFpaNcbGpkeOHKlzWcjDhw95Y5evm9q8rEWshgjJD5jWEBMTQy9YUVFRqfM7Vl5e3qdPHwCws7Nr7mqIlZUryapV5MUL4upKbGy+fPIAiLo6MTcnK1aQkJDQhISE2nN5eKtH9OjRY+rUqbS/OSUlpalPnZmZSW8Z6+zsLD4dE3y7cCHCyOiIqWmxre2Wzp1fNf5WL6amBICsWEEAyLBhzVlFAYSHk169yI4dxMGBEEJcXcn8+V9SDIDo6pK1a0kD3WK1xy5DQ0P5qMDHjx/ph01dXZ3fgxCyHzEPq7CwkC6tx2AwPDw8eKO29DvTtWvXJi2Q8h+wciXJySHOzmTBAnL+PFm/noSGku+GD2/1CBMTE9qQ6devX1PnTwUEBADAhAkT1NXVQbDVqMVBeTlp2ZIAEE9PAkAaOWZTWkpkZQmLRebMIQBkw4ZmrqUAOBxy6BDx8iLXrxNXV7JjB5GTI7a2JCyMNKZbrPbYZeMvwi8vL699HSWTybSwsBDoMITnB00c5XK5AQEBdP6ngYFBcnIyvfhWUVFRiOOvkmLlSlJSQq5fJwMGNG1H3uoRDg4O9LpCCwuLJnWsDh48GP5dB7V79+5Ne3qxRFfMX7iQsFhERoY0ZvpweDgBIEOGkM6dCQB5/Lj5aymAQ4dIbCxxdiazZpGSEtLUH/fad8Gxt7dvoGVXU1MTGRnp5OREr/bn9YudOHFCfMaRf+hM93v37tGOXlVVVTqScuzYsR9ZATFx8+aXTnc+JvfxVo9Yvnw5neswb968Ru778eNHulopnebefJde/kj373+5J9GYMQSABAZ+f5dFiwjAl5WzVFUbdaoiQocOkefPyYsXREOD/0LCw8PpYv9Dhgz5+noy3pSI2vPFgoKC6jR9/P39+eiFEK4ffWlOTk4O7YuRlpbu3bu3+CT3j+Tq+iWwGh01/4O3esT69evp2ibb6Nqa33Pu3DkZGRk7Ozv62W3kLDnx160bASArVxIA0pj5WDY2hMEgy5YRADJ1avPXTzAxMYSuW3PxokDLpcbFxXXo0IH2ftKLmeo0/eiwqbe397c+GGw2W+RDNCK4lpBOd6Cv0ciRI5u6itB/gKsrOX+eXLhArK35LIGuHkHXNqH3HGzkIgF5eXl79+4FgMGDB/P53OJn06YvHVjTppFGLoyekUEiIsj06aQR0wRFbPt2QpftWriQCLgQf1ZWFl2ES05OTl9fn9f0U1RUdHR0/Ouvvxq5fEBISIgQF3Rukh8dWIGBgXTA/sSJE3TmnpaW1n9jDbnGc3UlsbHkxQsiyKptixYtosM3dK62vLx8AzMni4qK7t69u2vXLmdnZ9oYl/TxwdpSU8mqVSQrixBCGnPPkIwMYm5O2Gxy7VpDq/2Jie3byb59JDSUWFoKGliEEDabvWDBAvoj962m33etWLFCVIElBT9QXFwcvRzkwIEDDg4OY8aMmTFjRkREhIWFxZo1a9atW0dfxJ9Bz54gJQVKSvyXEBAQkJKSEhoaevnyZWdnZwaDQS9VoQoLC1+9evX0X2/evOFyuby/MplM2o3136CjAzExEBEBDg7g5we//gpFRVBWBhUVUFxczz9GjgRVVQgMhDZtQF5e1LVvhFatQEdHoE8Lj5SU1P79+1u1avX27du1a9f26NGDj0J+++03+o/c3NyqqiraMf2D/LBoLCgooBOyat8nlcPh0EYNAJiZmf0kzcPExC//qG8ubRPwbts1bNiwZ8+eXbp0ac2aNZaWlrzbrPLIyckNHDjQxcVl//79t27d+taidJJr4ULi5kby84mr65frzBv4b8cO4udHPD3Jjh0kMlLUVf8eITYJhYvNZhsbG2/ZsuVHPimDEPJjYnHKlCl//PFHnz59Hj9+TK8b4ImMjJw+fXpWVpaOjs65c+fognyoMTIzMw0MDOgqXbUfV1FR6du3b79/de/end514r9q0SJYvhz27YPiYnjyBAoKQEEB5OSgZct6/tGnD8TFwdy5YGwMgYFgairq2jcoLQ3U1UFeHj58AD09+L3NnMEAACAASURBVLd/XCxcvnx5woQJ9ISjqqrq6/V4he/H5CK9T1/Lli3rvUCHEJKamkpzSkpKytfXt1nXDv2PuXz5sry8vLy8vIWFxapVqy5cuPDhwweRj+b8YPSsfds2Mn/+9zcuLCR37hBCyI0bRBg3nEfkzJkzY8eO/QFP9CMCi3eX04ZHsthsNq95OH78eMEv2vx58LF83X8M7QKuqiLx8aKuys8nKyurdevWP+aqiWZvEhYUFPTv3z85OdnLy4vee7Jh4eHhzs7O+fn5urq658+fHzJkSLNWDyEkuJycHE1NzR/wRM0bWISQiRMnhoaGDh48+O7du3Tlhu9KSkqaNm1adHS0kpJScnIyveoNIYSatyN269atoaGhampqdI51I/fq2LHjvXv3li1b1rlzZ0wrhBBPM55h/fPPP+bm5jU1NVeuXKEL4yOEkCCaa6Iml8t1dXXlcDg+Pj6YVgghoWjGM6yPHz/u3Llz165d9E5tCCEkoB80cRQhhAT3s1y7hxD6D8DAQghJDAwshJDEwMBCCEkMDCyEkMTAwEIISQwMLISQxMDAQghJDAwshJDEwMBCCEkMDCyEkMTAwEIISQwMLISQxMDAQghJDAwshJDEwMBCCEkMDCyEkMTAwEIISQwMLISQxMDAQghJDAwshJDEwMBCCEkMDCyEkMTAwEIISQwMLISQxMDAQghJDAwshJDEwMBCCEkMDCyEkMTAwEIISQwMLISQxMDAQghJDAwshJDEwMBCCEkMDCyEkMTAwEIISQwMLISQxMDAQghJDAwshJDEwMBCCEkMDCyEkMTAwEIISQwMLISQxMDAQghJDAwshJDEwMBCCEkMDCyEkMTAwEIISQwMLISQxMDAQghJDAwshJDEwMBCCEkMDCyEkMTAwEIISQwMLISQxMDAQghJDOEEVmxsbMMPvn79uqqqqs4Gubm5qampQqmA4AoKCpKTkxu5cb3H+7WampoXL17wX6fm9Pnz58zMTFHXAqGm+X5gnT592s7OztHRMTIyEgA4HM6uXbsmT548b968ly9fAsD9+/dv3LgBAAkJCa6urhMnTvT396+pqbl48eKTJ08AgBCyYsUKJpNZXV3t7+8/adKkBQsWvHv3rqqqasOGDV8/Y1lZ2d69e9euXdvUgzl06JCVldX8+fPT0tKaum9AQEBWVhYAXL161dHR0c7O7vLly7TywcHB06ZNmzlz5v379wHg/fv3wcHB9RZy9uxZe3t7Ozu706dPAwCTyfTx8eFwOB8/fnRwcCgpKQGAkpISHx+fr/c9ePCglZWVi4tLSkpKUyv/LTdu3Jg4caK9vX1MTAwAzJ07d+q/SkpKNm/e/PUuycnJ8+fPHz9+/NGjR7/+a1FRkb+//7Zt24RVQ4SahjRo165dlpaWSUlJHz588PX1rampcXNzc3V1zcjIiI2N3bZtGyHEwsIiOzs7NTW1Q4cOf/31V05OTkBAQEJCQmpqqo2NDSHk0qVLW7ZsIYTMmDFj2bJlnz9/joqK2rNnDyHEzs7u/fv3dZ5048aN27ZtGzRoUMN1q+PZs2fz5s3Lzs4+deqUpaVlk/YtKCgwNzcnhFy8eHHIkCG08mvXrq2oqNi8efPkyZNTUlLevn27YcMGLpfr6Oj4+vXrrwvZvXv35MmT09PTMzIypkyZsnPnTkLInj17goODo6Oj1dXVlyxZQgjJyckZOHBgnX2fP3++bNmy/Pz8M2fONLXy31JYWOjo6JiZmfn8+fMOHToQQnR0dD7/ixBiY2OTmppaZ68FCxZERETk5uYaGBg8evSozl9XrFixceNGKysrodSQSkpK2rNnz6dPn4RV4N9//713794m7XLv3j1hPXsdUVFR1dXVzVR4vRo+lnfv3tF3v86De/bsSU9PF1Ydbt26FRQURAiprq5+/Phx7T8VFBS8fPmS75K/E1i//PJLUlIS73+rq6s1NDQqKip4j2RlZY0dO5YQsnnz5g0bNtTZffjw4UVFRVOnTn379m1hYWGbNm3YbHbtDc6cOePn50cI8fHxGTt2rIWFBc2CrKwsXmDduHHDzMxs7NixR44cqa6udnFxsbS0tLa2Tk9Pj46OHj16tKWl5aZNm2pXSV9fnxBy//790aNHW1hYeHl5cbncoqIie3v78ePHT506tbi4+ObNm7TYw4cP86phZmZW5/3W0dHJysqq/QoYGBgQQmgTkqKfgM6dO2dnZ9PNcnJyOnXqRAjJyMiwsLCIjo62s7MzMjJ69uwZL7AePnw4evTocePG/f7777zyL1++bGtrSwiJiIig1du/fz+Hw3Fzcxs3bpyVldWnT59iY2PHjBljZWW1du1aQsiGDRvGjBkzduzY2NjYpKQkKyur8ePHu7q61tTU0DI/fPjQrVs3eiy1D+3QoUN79uzhcrmenp4WFhaWlpa1fzwWLVp09uzZOk9NCHn58qXggfXs2TMbG5s5c+YsWrQoNDQ0KCiooqLi3LlzkydPdnZ23r17NyEkNzfX3d19wYIFLi4u8+bNS0lJofsGBwc7OTm5u7s7OTmdOXOGEOLg4EBD4fbt20FBQfn5+ba2tlwud+vWrb1796Zv34ULFyZNmuTs7BwQEMDlcpcsWeLk5DRt2rSoqKgPHz78/vvvc+bMCQgIIIScP3/+jz/+qF1bHx+fmTNnWltbf/1VbxiXy12wYEG9f6qqqpo7d+7MmTNtbW35SLSIiIjp06dbWlpev36dELJu3ToHBwc7O7vbt2/v37//yZMntTcODw+fMmXKpEmTrl+/np2dvXr1akLI48eP6VuwdOnSs2fPnjx5srKy8tixY1OmTJk5c+bBgwcJIZ8/f54/f76rq6uLi8v8+fMzMjJogQcOHJg5c6a7u7ujo+OlS5cIIVOnTqV/Cg0NPXnyZF5eHn3k0KFDUVFRb9++nTRp0uzZs+fOnVtZWenm5kYISUpKmjJlyqxZs5ydnfPz81etWtWnT5/vHvh3AktZWbn2/3769Klnz561H7lz5467uzshZP78+fTTU5uzs3N0dHTv3r05HM7Lly+HDRtWZ4OYmBgnJ6e//vpr4sSJhJDc3FyahrzA4nA4enp6eXl5XC43PT09ODjYw8ODEJKZmcnhcPr370+/Y2lpabwyd+/e7eXlRQjp0aMHfdze3v7q1asbNmygp4RpaWl1il23bh193bt06VL7Q8lms9XU1GpX+MOHDxMmTCCEXL9+fem/9u/fz+VyNTQ0am+pqalJI6Nnz57R0dG0XTZ06NCsrCwaWL169UpOTuZVnsPhWFhYaGpqvnjxgsvl0vij1Tt79qyLiwsh5PPnz2w2e8iQITTW09LSHj16NGbMGC6XW1BQUFpaOm3aNPoJ5r0g7u7uGhoaly9fJoRoaWkZGRkZGRmtWbOGEHL37l1XV9ewsDAHBwdCSHZ2dlVVFd2rpKSkZ8+eXz81EVJgjRo1Kjc3lxBSVlYWFhZ28eLF4uLikSNH0lesrKysqqrKzMzs9evXNTU1BQUFGRkZo0aN4nK5gYGB69atI4QUFhZyOBx3d/e7d+9aW1vTml+/fn3Xrl2EkJkzZxJC8vLyXFxcMjIySkpKRowYweFwaOHh4eE06zkcTmVl5ZIlS9LT0+3s7GxtbVNSUk6ePHnu3LnKysrdu3dv3bq1qKgoISGBEBIcHHz06NHKyso9e/Zs3bo1Pz//ypUr27dvp6cS+fn5W7duDQwMZLPZMTEx69atu3XrFg3Q2geelZW1adOmoKCgyspKGsEzZsxITU0NCwvbu3fvjh07OBzO9evXz549++uvvxYVFXE4nMOHD2/cuDEzM5Nus3PnTg6Hk5iYyOVy4+PjXV1dHz58uHDhQkIIl8stLy8vKSmZPXs2IeTq1avr1q17+fLlu3fvOBxOfn4+bfE4OTlVVlaOGDGiqKiIviBnz569ceNGTk4O/SzRB8vLy0eOHPn+/fuamprCwsJPnz6NGzeOEPL777/TX/eCggIOhzN79uwnT55YWFjQA7xw4cLhw4d5b8G0adMIIRMnTqQf9bKyMkLI2rVr3759O3369Ddv3tAH6btsbW393U+OVMMNRhUVlcLCwpYtW9L/VVdXz8vLq71BRUWFrKwsAGhoaOTk5NTZXU5Orry8nMPhsFisb21QUVGRkJBgYGBAy6+zQXZ2trq6upqaGgC0adMmISFh8ODBANC6dWsAyMnJ6dy5MwC0bduWbn/r1q0zZ878+eef1dXVlZWV9HHayktISHB3d6cbZ2Vlqaqq8oqtfRS5ubmtWrWipUlJSbFYrKqqKvrX2sc7ZMiQbt260Qfl5eUZDIacnFxlZaWcnBwAVFVVSUtLM5lMAOByuXSzAQMG9OvXj/Z/1dTUFBcXt2/fnld5Fot1/fr1V69eOTg43LlzR0FBQVNTs85R04qlpaX16NGD7njr1i0DAwMGg0Hfozdv3tAteS9IYGDgr7/+2q9fP3NzcxkZmXv37n394tNd6NPRyk+bNs3Ly4s+9ZAhQ3hPLSzV1dX0vVZQUKCPfPz4sVevXvQVU1BQuHjxoqWlJZvNdnFxyczM3LVrV8eOHd+9e3f+/PnIyEgPDw81NbXi4uL58+cfOXIEABYuXMhkMlNTU8eMGcN7Fvr+AkBSUlLPnj1ZLBYt/MWLF0ZGRvQ1Z7FY79+/b9OmDQD4+vquXLnSwsICALZs2TJw4EB9ff2SkhL6Rj969GjBggV+fn79+vXr2bNnYWEhTSUfHx9DQ0M/Pz8fH5+qqqqcnJy1a9eeO3fu1atX9+/fHzx4cEpKyuPHjwHAwMBg+fLlfn5+BQUFXC5XXV197ty5FRUVOjo61tbWV65cCQoKioyM3LFjx2+//cbhcI4dOyYnJ0cImTdvXnFx8bp16+jZ6N9//21ubn758mWafU+fPh02bBgAMBgMeXl5AMjKynr69OnVq1d/++239+/f9+zZEwAePnzYt29fAGjXrl18fDyTyVRWVq79Frx9+3bAgAEMBoM+ePz48alTpxYUFCxYsCA5Ofn48eNqamqfPn26du1aRESEm5sbfXnnzZt3+fJlei4JAImJidOmTaMFVlZWstlsAMjPz6cfdfpc3bp1i4mJSU9P/+WXX3gP8kKmYd/pdLe2tt6+fTv9d3l5uaKior6+/qlTp3iPaGpq0giztrY+cuRIUVERAFRVVdXU1ABATk5Oq1atWrZsWVxc3Lp1azU1tbCwMN6+vA06d+4cFxdXbwU0NTVzc3NLS0vp/9bZUlVVtfY44+7duwMCAq5du6asrCwjIyMtLZ2bmwsAcXFxXbp0qb2vhoZGXl4e7QUHAC0tLd5R7Ny5k0YMraGlpeXOnTu/Pt6oqKh9/woNDQWACRMmBAYG0i337t07YcIEAGCz2bywA4DNmzfTLxiLxVJQUPj8+TN9PCMjg9ZNTU2tqqqqZcuWpaWl9MX8+qhbtWqVmJhY75/09PR4/1tRUXHnzh0AUFRUZDKZvNzkyc7O/vrFT01NHTt2rIODg7OzMy3/+fPn9b41gqBnUrUfUVNTo4Me1OvXrw0MDPbt2xcQEKCjo6OlpSUtLU2/2/Hx8To6OhMnTtTU1JSSkqIvb2Bg4IEDBxYuXFjv09UpXF1dvfb/SktL03/o6el17dr19u3bABAfH29mZtapUyca/QcPHtTW1u7Xr198fPzIkSPp4127dqUvYHFxcX5+fo8ePfr161dUVNS9e3dlZeVhw4ZVVla2aNFCVlZWXV1dXV1dVla2vLxcT09v4MCB8vLyCgoKhw8fbtOmzcuXL3V0dHR1dbt27VpUVCQvL9+/f3/674SEhJEjR7Zu3bpr167t2rXT1dXt0qUL/WBMnDjx6tWrO3bsqHM49IgSEhJMTEyUlZUHDBhAX8/Dhw8vX74cAFq0aFFVVfX1qH2dVyk+Pn7w4MH79+8PDAzU0tLS1NSUlpYuKCjo3LlzTExMjx49LC0t6VsgJyfHZDIPHDhw4MABFxcXXglVVVVKSkoAwGAwaHJRysrKFRUV9A395kfkG75zhrVlyxYPD4+hQ4fKycnp6ekdPnw4ODjY1dU1KCiIyWSam5uvXLny9evXAGBgYODh4TFq1ChFRUUOh3PixIn27dsnJSV16dLF0NDw8ePHo0ePPnPmjLu7O22XTZo0ydPT8/Hjx8OGDRs9enR4eLilpaW8vPyqVatiY2NPnTqVmZk5atSoCxcu+Pv7W1tbq6urm5qazpkzZ9asWTY2NiwWKyAgIDAw0MnJSUNDQ19fv0uXLuvXr+/Tpw8N+IiICNoLrqCg0KZNm/Hjxw8fPnz27Nl//fUXk8k8ceLE9u3bra2tNTQ0TExMhg0bdvLkSScnJ09PzxUrVgwZMkRJSalFixb0nN/d3d3Q0FBaWrpXr1579uwpLi6urq4eO3bs2LFj67xWixYtGjFiBAC0b99+z549ABATE2NgYCAlJaWoqAgALVu2XLt2LT3JCgwMtLe319DQ6Nev38yZM319fUtLS8vLy3ft2sVgMAICAmxsbDQ0NAwNDd3d3SMiIiZMmCAlJeXv7x8YGDh37lx1dfWuXbtu3rw5NDTUwsJCUVFx2bJlW7duXbhw4d69e9XU1Hbu3Hny5MlNmzZVVlYuXbpUWVm5zo8YffEtLS1v3LhhbW0tIyOzYcOGGTNmVFRUBAcHBwcHOzg4ODg41H7qy5cvX758OT09fdSoUREREU39tPFYW1t7e3tPmDAhNTWVvjK6urqEkIMHD3bv3r28vFxVVTU7O1tbW9vHxyc3N3fZsmXV1dW//PJLQUGBiorKrVu33r17l5eXFxcXt2rVqm8NKD99+jQ9Pf3evXvjx4+XkpI6cOCAvr5+WVkZ7dBRV1cnhHTs2FFaWpr37fX29u7Tp4+FhYWJicnevXuNjY3btm3722+/ZWVlubi4fPz40djYeN++fSYmJnVOOTt16hQSEtKqVatevXq9evUqKiqKw+Ho6uomJyf379+ft3Hr1q3Dw8MVFRX79Omzc+dOc3Pzjx8/qqiofOuFMjY2PnTo0NSpU+u8d4cPH27dunV+fr6Wlta4ceMsLS07duwoJyenqanZv39/Lpc7ePBgHx8fegqTlJS0efNmX1/fFy9eDBkyJCkpaebMmaNGjVq3bt2YMWMyMjJomd27d8/LywsODtbT02Oz2aqqqllZWRoaGt7e3qWlpYsWLZKSkurYsWNubq6amlp4ePjz58/z8vIePHiwadMmegpZh7KyMs1WR0dHDw+PGTNmfPz40cnJKTExUV9ff+7cuW5ubnPnzv3w4YOTk9ODBw9ycnJu375tbm7e0EeHry6I/+Ht7X3nzp2vH7969erGjRsJIcnJydOnT/96Ay6Xa2ZmVllZKXgdBGdubk6b9N+1Z8+eU6dONbLY2bNnx8XFCVCv5sJms0eMGFFnDORHevTo0fHjx2NjY2kfFiGEdt+cPHkyJSUlLS1t1KhRxcXFtKursLCQ7jVr1qzbt28XFhay2eyqqqry8nJCyP379+kpW2ZmJu3TpB0of//9d0hISEhISGlpKYfDuXHjxokTJ+jQQW5uLu1cLy0tPXfuXHh4+KNHj2gnV3x8PO1dunPnzsWLFwsKCkL+RYe36OOFhYUPHz6k2+fm5nI4nKtXr169erW6ujorKyskJCQmJiYrK8vT07P2UVdXV1+5cuXmzZtsNjshIeHEiRPv3r2jh0AISUpKSk1Npf/Oy8ujPZVRUVHnz5/Pzc2tvU1FRcUff/wRGhpK38GioqILFy5cuHChsLDwwYMH+/btI4S8e/fu7NmzHz9+vHv3Lq3/n3/+WVNT4+TkRCtz//7948ePv3z5kvZh0eqFh4efOnUqPT09MTFx3LhxpaWleXl5td8Ce3v7+/fv0w6syspK2ul89+5d+tfU1NTExETeW+Dh4UF3f/bs2fHjx+mIIW0IE0JevHhx/PjxBw8eEEKuXr1KK9nwx0YIgZWTk0M7cevw9vbmHaSXl1dpaWmdDZ4/f75//37BKyAU169fDwsLa8yWlZWVixcvbsyW1dXVixYtEqxezeXhw4cnTpwQdS0IIeT+/ftz5sx5+/ZtncejoqLs7e0XLFjg5ubGm/RQWlrq5eU1Z84cNze3b9X/jz/+mDx5cuMrUFNTU3uUWbj8/Px+8LSG3377rYGTgEePHt26davOg7dv33ZxcaFBU9u9e/fs7OxcXV3d3d0zMzPpg8XFxYsXL6Zvwblz5+p9lnPnztnZ2RFCEhMTT548WftPubm5TZ10UhuD/G9XAkIIiS28lhAhJDEwsBBCEgMDCyEkMTCwEEISAwMLISQxMLAQQhIDAwshJDEwsBBCEgMDCyEkMTCwEEISAwMLISQxMLAQQhIDAwshJDEwsBBCEgMDCyEkMTCwEEISAwMLISQxMLAQQhIDAwshJDEwsBBCEgMDCyEkMTCwEEISAwMLISQxMLAQQhIDAwshJDEwsBBCEgMDCyEkMTCwEEISAwMLISQxMLAQQhIDAwshJDEwsBBCEgMDCyEkMTCwEEISAwMLISQxMLAQQhIDAwshJDEwsBBCEgMDCyEkMTCwEEISAwMLISQxMLAQQhIDAwuhugghmZmZqampoq4IqgsDC6G6Ll261KZNmyVLloi6IqguDCyE6tLR0QGAtLQ0UVcE1YWBhVBd7dq1AwBsEoohBiFE1HVASLzU1NTIyclxudyKigoZGRlRVwf9PzzDQqguFoulra3N5XIzMjJEXRf0PzCwEKoHbRViN5a4wcBCqB603x27scQNBhZC9cAzLPGEgYVQPXBmg3jCwEKoHjizQTxhYCFUD+zDEk8YWAjVA/uwxBNOHEWoHlwuV05OjsPhVFRUyMrKiro66As8w0KoHkwmU1tbmxCCc0fFCgYWQvXT0tICgCdPnoi6Iuj/YWAhVI8TJ068ePGiXbt206dPHzVqVHh4OHaeiAMMLIT+R0FBwaRJk2bOnFldXd2yZUtpaenbt29bW1v379//+PHjVVVVoq7gTw073RH6f0+ePLG3t09MTFRWVg4KCjI1NV2/fr2ysvLp06fT09MBQEtLa9asWYsWLWrbtq2oK/tTIgghQrhcbkBAgLS0NAAMGjTow4cPhJB169YBAJPJHDdu3Nq1awcPHky/NTIyMra2to8fPxZ1rX86GFgIkezsbAsLCwBgMBgeHh5VVVX08efPnzs5OfGWxOrfv//q1asnT54sJSVFHzE0NAwJCWGz2aKt/88DAwv9Pzab/fjxY39/fz09PX19/devX4u6Rj/CX3/9pa2tDQCamprXrl37eoOsrCw/Pz9eG1BLS2v+/Plz5sxRVlamj+jp6bm5ueXl5f34yv9sMLB+dmw2OyYmxs/Pz8rKqmXLlry+AiaT2aNHj/z8fFFXsBmx2WxfX18mkwkApqam6enpDWxcVVUVEhJiYGDAaxVOnDhx5cqV3bp1o6dmJiYmP6riPy8MrJ9RaWlpRETE2rVrjY2N5eTkavdpduvWzdnZed26dXp6egBgYGBQXFws6vo2i5SUFCMjIwBgsVi+vr4cDqeRO0ZGRlpbW9OYAwBHR0d6fx09Pb1mrTAihOAooUD+/vvv169fz507t87XXgx9/vz5+vXrb968uXfvXkxMDIfDoY8zmcyePXsOHTpUS0uroKAgNjY2Ojr62LFjxsbGxsbGiYmJw4YNu3XrlqKiomjrL1wbN24MCAgoKCho167dmTNnaHI1SWJi4sGDBw8ePLh69Wo7OzsdHZ3WrVtnZmY2R20RDwZW03C53ISEhAcPHty/f//vv/+mF8fq6+vHxsbSASbxVFxcrKWlxZtDxGKxfvnll/79+2tqapaUlERFRb1+/ZrL5dK/SktLb9y4ceXKlSkpKcbGxp8+fRo1alRYWJj4h3IjRUREjB49GgCsra2PHj2qrq7Od1FFRUVSUlJycnLy8vI1NTV404rmhoH1fRwO5+nTp3fv3r179+79+/cLCwt5f9LU1CwoKOBwOPb29idPnmSxWCKsZwMuXbo0ZcoUGRmZhQsXslisnJycR48evX37lreBnJycgYGBiYmJsbHx0KFDeedT79+/NzY2/vz584QJEy5evMgbHZNooaGhNjY26urqb9++TU9Pb9++vYqKioBltm/fPiUlJSkpqUOHDsKoI6rff+Hz1xw4HE5cXNzt27fv379fJ6S0tbWHDh3aoUOHysrKQYMG9e3bd+TIkWfPnpWRkTl69Civa0Os/PPPPwCwYsWKkSNHjhw5kj6oqKjYt29fIyMjc3NzIyOjek+gunTpcuvWLVNT09DQUHt7+3PnzoltKDfesGHD6D9mzJhx8+bNa9eujRs3TsAydXR0UlJSUlNTMbCaFQZWXadPn/b29s7Ly6usrOQ92K1bt2HDhtFW1cuXL69fv07/amJicufOnWvXro0ePfr48eMtWrTYs2eP6Or+TTdu3AAAS0vLPn36TJgwwcjIaPjw4QMGDGjMGVOvXr1u375tamp68eLFuXPnHj16lMFgNH+Vm5GGhoa8vHxeXh6dzSCUVfpw/awfAwOrLjc3t+LiYgDo1KmTkZGRqqpqWVnZ8+fPjx8/XlNTQ7dhsVj9+/c3NjY2MzMDgKFDh16+fHn8+PGBgYEyMjLbt28X5QF85c2bNx8+fNDQ0Bg0aBCLxbpy5UpTS+jbt++1a9fGjBlz7NixFi1a7N69uznq+cMwGIy2bdt++PChRYsWIKSUwSWVfwwMrP+RkJBQXFysrKz87NkzGxubkydP8vr4pKWlBw8ePHz4cGNjYyMjI96kQcrc3Pz8+fNTpkzZsWOHurr6qlWrRFH9+l27dg0ALCwsBGnNDRs2jIbynj17lJWVN23aJLwKioCOjs6HDx/oynxCSRm8acWPgYH1P+h328bGRk9Pr7q6WkpKqnfv3ubm5oaGhsbGxg13zVpbW589e9bOzm716tUyMjLLli1r2nOXlYGPkptbsQAAIABJREFUD6ipQV4e+PmB8KYR0IOytLQUsBxeKG/evFlRUdHHx0cYtRMNekJECSVlcA34H0S008DEzYgRIwDg/PnzhJAPHz7wrilrvGPHjjGZTAaDceDAgSbsxmaT/fvJ/fuEEHL3LmnSvg0qKiqSkZFhsVjCunDkzJkz9EzNyclJKAWKBE3bxYsXA0DXrl0FLzAqKgoABg4cKHhRqAHiOKQlKsXFxQ8ePJCSkqKTdPT09PiYUzNz5szdu3cTQtzc3M6cOdPQpiUlcPs2rF8Po0ZB//6QlQU6OgAA7dpBVhafx/CVW7duVVdXGxoaqqmpCV5aRUWFvb395s2bGQzG6dOna49LSBZ6QlRaWgp4hiVRsEn4//788082m21iYlL7kjo+uLu7V1dXL1261MnJSVpa2tbWlven7Ozse/fu9YmP73zpErx8Cf/O1QRpaejbF/74Azw94eJFMDYWpAK1Cas9SJmamlZWVs6cOZMQ0rVrV8mdSkqbhJ8/f27ZsmVhYWFeXp4g00cBoHXr1jIyMtnZ2VVVVXjTiuYjkYFVUVHx119/DRw4sHXr1kIsVojfbU9Pz/z8/E2bNs2YMYN+gu/fv//gwYNnz54RQg4bGXWOiwMpKejXD8zNwdAQhg+Hli0hPBx8fWHQIBgxQvA6AACXy7158yYI6aCys7Ojo6NlZGSio6MBYPbs2YKXKSq8Qb127doVFhampaUJGFj0phWfPn3KyMjo2LGjkKqJviLaFmnj1V5UgI7QqaqqZmVlCav8mpqaVq1aAYAQ11Sh18QqKSnxXm1FRUVzc/Nj/v7kzh1SUSGsJ/oWegMFXV1doZQWHBwMAGPHjqV3Z4iPjxdKsSKRk5MDAGpqanQZLLpkO38KCgpu3rxJCKEXJP7zzz/CqyaqS6zPsEpLSx88eHDv3r1//vknOjq69nLaUlJSBQUFY8aMiYyMVFVVFfy5YmJisrKydHV1e/ToIXhpERERb968Wbx48eHDh9lstrm5+ahRo4yNjQcOHNi0q1vi4qC6GgYN4q8a9JzRysqKv93rLU1fX//mzZsdO3bs3r27UIoVCQ0NDQUFhfz8fHqSznc3VnR0tJ2dXXp6+qNHjyoqKgDg7t27xsJr0aM6xC6w6LW49JqY6Ojo6upq+jhdnsnIyMjQ0HDkyJE5OTkzZsx4/vy5hYVFREQEnQEoCPptHD9+vKAHAAAABw8evHjxYkpKSmlpaceOHSMiIvgp5ckTsLaG/fsFDCyhtAfZbPbt27cBgP5sCKtTTITatm37/v17vueOcrlcf3//NWvWcDicwYMHX758OTY2thmqKTEKCwtfv35taGjYvE8j6lO8LxITE83MzLp37177WjwpKakhQ4YsX748PDy8oKCAt3F0dLS6uvrcuXM7deoEAIaGhqWlpQJWoH///gBw/fp1AcshhNC7rQDAwoULAWDRokV8FvToEbl2jRCSkpLCx95ZWVlMJlNeXr6srCwxMVHAZa0iIyMBQF9fv1+/fgBw48YNQUoTB6ampgCwcePGkSNHHjx4sEn71l5Sec6cOSYmJgDAZDJtbGx+nuWSORzOq1evjh8/7uHhMWDAACaTKSUlJfg3sWHiElj07achNWDAAG9v77CwsMLCwno3vnr1Kl3LxcvLq3379gAwatSoyspKvp89IyODwWDQ7zbfhfDU+W7TDg6+xcXFaWlp0XsiNEZ6enpISIiLi0uHDh1atGjxyy+/xMfHt2nTxsTEpLy8nO9qeHl5AYCbmxt9oQQpSkzQj9ykSZNqamqatGNkZGSbNm0AQENDY/PmzbRRqaWl9R8I8e/KysoKDQ1dtWqVqalp7c5ZAJCTkzM0NExMTGzWCohLYPXu3ZsGUEXjuqIvXbpEO4OWL19OPzGC/LgdPnwYAMaPH8/f7nXU/m4rKio28oi+pbq6+unTpw1swOVyX716tW/fPjs7O/pF4lFQUAAAT09PwWOdLgS8fPlyIb5QouXk5ERfpU6dOvn5+TVmMejaSyqbmJgsXbqU/tvMzCwjI+MH1PnHKy8vj4mJCQoKcnR07NGjR53r3rW1ta2srPz8/O7duyfg57yRxCKw0tPT6Xfb09PTw8MjKSmpMXudOHGCzilfs2YNnRU5Y8aMpv5aUhMnTgSAps1N/7ba321ra2uhlFlYWDh58mTe/9Kz8aCgIFtbW01NzdqfIU1NTd5n6MKFC0KJ9cTERABQUVGxtrYW4gslWkVFRY6OjrRXAQCUlZWXLFny8ePHb21fe0nlJUuWDB06lDYIfH19+fvUib9Hjx7VWZZSSUnJ0NDQw8MjJCTk8+fPP75KYhFYQUFBADBhwgSaO+/fv2/kjkeOHGEwGAwGY/Xq1bT3dNasWVwut/FPnZmZeebMGToBMjk5+dKlS66urk0qoQ7ed5v23wcFBfFdVB0lJSXv3r2bOnXquHHj6lx63bZt2+nTpx84cIAuHFp7r9qxTqcaTZkypfHrl1N0eQZbW1v6IicnJwvroESupqYmLCzM3NycnjswmUxzc/OwsLA6L+OVK1foh1NHR+fXX3+lA9O6urr36dVU/1H0kgZ5eflZs2YFBQXFxcU19ZMjdGIRWPR3m56SdOvWrUn77ty5k/7obdy4ka6T6eHh0fAuGRkZISEhtKeQfkzV1dWlpKQOHTpEg+C7JTSgznf706dPfBf1tZ49e9Y+G7e1tQ0KCnr16lXDe/Fifd26dXQ0wNnZuUmhPGbMGABYsWIFAPTu3VuwgxBTsbGxLi4u8vLy9OXt27dvUFAQ7arjcrn03MrS0tLFxYVuYGNj85+/r9egQYMAIDQ0VNQV+X+iD6zKykolJSUGg+Hm5ka7sZpagq+vLwBIS0tv2rSJniutXbu2zjbv3r07fPiwk5NTnQUhlZSURo8ebW5uDgAyMjKbN2+mJXh6evJ3OLW/23379uWvkHpVVlYqKCgwGIz169enpaU1ad+AgABerNO+0saPXZaWlsrJyTGZzHnz5gGAj49P0+suMb6+BaG3t3daWlpKSsqaNWtoT6ucnFxAQICoa9rs0tLSGAyGkpKSWA2wiD6w6LUj/fr109fXB4DIyEg+CvH29gYAeXn5X3/9lfbabN269ePHj7SzkHY587Ro0cLc3Jz28vDWY1i5ciXNLF9fX9pu37x5c1OrUee7vXr1aj6O5VvoqqH9+/fnb/f169fTWP/1119pKK9Zs6YxO+bn569du9bZ2blLly4A8N9uBFGVlZVHjx7t06cP/cDIysoaGxvTk6/u3bvHxcWJuoI/Am0rTJkyRdQV+R+iD6xFixbRH3za8cnHii6EEC6X6+rqCgAKCgrr1q1jMpl1LkBt1arVlClTdu/eHRcX960u0qVLl9ISeKnn7+/fpGrQxTyHDBnSuXNnAHj48CEfx/ItdFbX1yePjUdjXVZWlhfKW7ZsacyO1dXVu3btAgBVVdWfZ54RIeTevXu2trZSUlItW7ZkMpmOjo4lJSWirtT/Ki8ne/aQLVsIX5P1GkDnqZ0+fVq4xQpI9IFFv9t0uTtB4pzL5c6ZM0dJSSkyMtLMzExeXl5VVdXW1jYgICAmJqYxXTZcLpeeGamoqNDR66Yua0V3p6ssqampCbeHkt7Z9PHjx3yXwOVyabubxjpd1mrPnj31blxdXc27eJO3cmGvXr34fnbJRS/JVFFREXVF6uPhQd68IUVFZNo0IrzPW1lOTntVVRkZmdoTtsWBiAMrISEBADQ0NOgsvuDgYEFK43A4CQkJ5N/vdlRUFB8l2NnZ0SqtXr2aDhs1/keGdlJ6enoCwIwZM5r67A2Ij48HAE1NTQFH0Llc7ty5c3mhLCcnd/nyZd5fi4uLb968uWrVKiMjo9qnqAwGo3v37kOGDImNjRX4UCQPl8ul7cHmnsbNj1mzvvxj/XrSuPlAjRIcTGRl0+bMEVqBQiLiwNq2bRsATJ8+XUFBgclkZmZmCl7m69evBfluV1dX0xkJWlpatGOLxWKFhIQ0Zl8ul/vs2TN6H62zZ8/y8ezf4u/vD0Ja5LN2KEdGRhYXF0dERHh7exsaGtZesJBevOni4hISEpKdnS3480o02g54+/atqCvyFScnQlsPbm6kqEhoxY4fTwCI8CblCIuIA4u2k2nfyqBBg4RS5u+//w4AdJE5/lRVVdEzvnbt2tGeIxkZmWvXrjWwC4fDiYmJ2bFjx7j/a+/c46HK/z/+HuMycosilySEUO6XMFFRu6ot2q3NbjbdS1Fr25VS6bay220ppRuirVRYl5hvKSS5REm6CF2Qu3FnzJjP74/Tzs/XbsrMiPH9PB/9oXPO5z3vOXPmNZ/zPu/P+z13LpFDUF9fz7UD/6R37Wbe6e7uJko4kMnk3os3RURErKysfvnll4SEhA+ti/rfhLhQb926NdSO/IPUVOThgby90fHjCCF0+zZycUE8xhlbWxGFgoSE0PBL3x9KweKUGydKwfn5+fHFLLES9RPnRB+ivb2dKBIyadIkIjKlra3dJ95MlOg6duzY4sWLe5e4ERISolKpvL2J/6K5uVlERISoqMMvm52dnUSGB5lM/ujiTYyrqysAhIWFDbUj/VJQgJSVEQBavhzxkPyMoqIQAOLrNcwvhlKwrl69CgC2trbENyc3N5d3m01NTfz6bjc3N5uZmQGAjo6Ol5cXsWCovb397t27vesIcuBkcvISF/9XoqKiAMDOzo6/Ztva2hISEpr5eB8xciH6tu3bt2+oHfkwQUGITEZ+fkhSEgGgTZu4N+XiggDQ4cP8c45vDFk9rBcvXhBPymfNmpWSktLd3U0UeOERoi77jBkzeKzLDgDS0tI3b96cNWvWw4cPOzo6Wltbnz592rtEF4lE0tfXt7OzI5oV9ll4zEf4W5edg4SExAgoa/V5EIC2g0JC0NMDBw7Anj2wZw8cPw4TJsDPPw/YTk8P3LoFAODkxHcf+QAfxY/NZvcf5+Zkck6YMIF49TFjxhCBZF6Kw/SGWII/0PypfqipqVFVVeUsUieTyZxQdF1dHb9epR8Go3YzZqDEx8cDwNy5c4fakX7ZsQMBIHFx5OeHhIQQiVQ70Cc/bDZKS0NJSSg2dnBc5BVeBatP2YA+eepMJjMnJ+fw4cMLFizo02ZKUVHRzs6OSF/cs2cPj24QcL7b/C03np2dTaVSnZycEhMTP/8NVFZWFvCvLjuGO4hqosN9HSWbjdasQQBIRgZ5e7+wsxslJhYdHT0AC15eKDYWxcQgL69B85InuBGs7m6UmYmCgx/NmzevTzPkgwcPfigUDX9HeXpnckZHRxM55QEBAby/GeK7ra6uzrup4UBHR0dqairxDGHjxo1D7c7/NPX19QAgKys71I58DBYLOTsjgPSFCz09PQFAXFz87t27Hxn15g26fBm9fIlWrny/ZeVKNCxr5nyqYDGZ6MEDdPAgmj8fycggAKSh8YwjQ4sWLfrxxx/d3d1NTU37NFnQ1tZetWpVeHj4h2qScOqfnDx5ksc3s3PnTgDYxEu4cahpa2sjgvoODg69u/7xpXYzhheIaojDbmnOP+nsTF61CgAmT57MSRJ+9OhR70MYbW0oLQ0FBCBnZ6SkhAAQAPr9d+Tm9v4INzeenjMOGp8adLe2htzc//+vri5YWeksWbKrpaX+0aNH8fHxTCYTAERFRXt6ejQ0NBwcHGxsbOzs7PosPP4nrq6uTCZz9erV7u7uIiIiq1at+kSX/skgBacHndZWyM6GW7cgIyOXQpmZkkJsJpPJU6ZMefLkiZCQEPFtwQwhRNOKyspKHR2dofalXygUqyNHjPPzHz58KCUl5ezsHBMTM3fu3Ojo6Ldv32ZkZOTl5T0pLKzt7BRlMt8PGTMGpk0DdXVgMODkSQAALS347+KiwwQSQqif3bdugYMDNDTAL79AVhZYWICCArS1QUEB5OSAmdmS+/evAgCZTDYyMjIzMyMePBGp3gPijz/+2LJlC5lMvnjx4rfffvspQ3p6egoKCogmYL///vuoUaNUVFQoFEpDQwOnqtGwpqoKfv0V0tKgqIjTArrZxORLUVFTU1NJScmmpqYHDx4QvVfl5OTu3LlDlDfBDAn29va3b9++efMmUYxomFNXV0elUouLi2fMmFFTU0OsgeMgLCz8yNFRX1UVpk0DS0vQ1v7/faWlQCLB34VYhxsfmWEtWwYxMQAA6uqQnQ3h4cDRN1FR0NZeamamKCkpSafTc3NzidYj8vLyXAjW5s2b6XT6nj17XF1dJSQkPtRKj8ViFRQUEF2UU1JSGhsbie1EfjlCyMHBYbirlb8/MBjQ1ATr10NwMLDZ71tAm5sDhSJdX9+dn3/ixAnO4aNGjZKSkqqpqXFwcEhLSxPoboACDZHZUF5ePtSOfBLy8vI3btywsbFJTU21tbV99uyZoqKimZmZqakplUq1trb+4JxdU/PzejowPiJYixbBqVOwejXIysLTpyAsDAYGYGoKFArU18Pr1zppaV9zDpaQkLCysupTIe/T8fPz6+rqCggIWLJkSWJiIrEYAgA6OztzcnLS0tLS09OzsrLa29s5QzQ0NKytrZWVlZ89e3by5EkAsLCw4O7VPxN5eSAuDj4+UFMDAQGwZw80N0NFBWRkwKlTAEACsDE0LJaUnDZtmo2NDZVKnT59OolEWrRoUWJi4uzZs9PT0zWG66/fyIZYw3Tt2rVly5b1qXQ+PNHU1CQSCV1cXCIiIji5RIJN/yGuTZtQZiZycUHh4cjfH3377fvUf+KfkFCPmpr6/PnzAwIC7t+/z3ulpN5lrdLT0xFCkZGRfSpb6erqrly50sfHx8vLy8HBoU9RgT7BxWFHTAyKi3v/t5sb8vD4/7M5dixyckJHj9Y+fPjPujTt7e3E48IZMzwGWG0Uwwfi4uIoFAqRjaytrc3jwq/PyQhbbvWRGJaHBwQFwZo1MH06ZGZCSAgAgLQ0WFiAgwPY2IClJfD3xwYhtHbt2rNnz8rIyKSkpPT09FhaWmpoaFCp1NGjR3d0dOTn5xNF+IjjifCZra2toaGhra2turo6P73hO69fQ2AgHDkC2dmQmgoGBnDhAkyfDnZ2oKfXf5iztbV1xYpjt2/7jhtHSksDBYXP5vT/OhEREatWrWIymfb29m/fvn358iUAzJkz5/Dhw72r7A9Djh071tra+s0334ycSEL/ekbkcnd0oNZWlJGBTp5ERUWD/riTU/9ETk5uy5YtK1eu1NfX790QTVRU1MbGxsfH58aNG4K3FC45Ge3ciY4cQQOvrdrUhExMEACaOhWN9AYIw4Vjx44RN4Pe3t5sNtvT0/Pnn38mEgzJZLKPz6HPst6BS7Kzs7ds2bJ///6hdoRvDH3F0X+FwWDMmTOnt0iNGjXKxsaGKCogeCLFP2prkZ4eAkAWFoi35vOYj8BmI1/fIwAgJCRE1GW9ePEioVMrVqzw8PAQExMzNEyXlES7dyM+LS3DfIRhKlgIITqdTqVSNTQ0/P397927x12t9xFJRQXS0EAAyNoaDbQEZnt7+6e3ffxfhsVCq1ahiRO7lZQMObUYGxoavL29iTKHkpKSGzd6ffllJxGB1NIatsvvRhTDV7Aw/fDmDVJTQ/Pno3v3EEKotRWVl/c9pr4e5eW9unz58oEDB1atWjVjxowJEyYQiwp6l0UewdTX13d3d3MxsL0dzZuHAJCkJLp1q28H9hcvXixevBgAxMXFLSxeb936fs4LgGbNQikp7wMpz57x/g544vTp0/PmzRthzROxYAkqb94gJhOpqqKsLPT4MTp0CJ06hX75BX3zDTIxeb98ys7uaJ+QJXGX7ejoONTuDzr79u0TERERFxcnWro9ePDgE0tmNzYiKhUBIDk51E/bo+Tk5CVLThI6NWMG8vVFCgqIQkGbN79fkLdhA5/eCbc0NzdfvXqVlzbmw5CPPCXEDHM2bICuLti8GZKTwcfnv3ZJS8Ps2WkIBWpqampoaGhoaGhqaoqKimpra3d3dxcWFurp6Q2R14NLZ2enp6fn2bNniVxiznY5OTlbW9uvvtplbm48Zcq/P5KtqgJHRygoADU1oNGg/0U4LBacPg27d0N9PZDJ8N138NVXUFQEY8fChAmQnAzBwfx+b5ihVkwMT2zahDIy0Lp16PhxtGYN2r8f/fknyspC/XSNWLduHQC4cZa5jiyKi4uNjIwAgEKh/P7771VVVVFRUWvXruWos7l5NZH0Nn8+OngQPXiA2Gx07dr7p67h4UhJCenpDaDLH52OvL2RmBgaNw41N6Pdu1FdHVq5Eg3tCWaxWPztMjdMwIIl2BCVKVavRidOfOqQsrIyYWFhERGRV3zsCjU8iI2NJXI7tbS0/plC/OrVq9DQ0LVrGaqq/5+uS8TLly9HP/+MEEIbNqCiIm5SRoqL0c2bCKH3gvX0KdLV5f0NcU9aWhovPXeHLfiWULA5exZ0dUFHBxACeflPHfXDDz9ERES4u7v3XrQo0LBYLF9f399++w0h5OTkFBoa2n+N7Hfv4N49uHULaDSYMgXU1UFLC8zMIDKS1/u427fh3TuYMQNERYc4ubenp4folTuiGGrFxPDEmDEIAFVWDmzU06dPhYSExMTEKgc6cljy9u1bKysrABAWFj548OBAh7e0oE2bEJOJ3NzQunW8OrNpEwJA/v682sH8K0IfVTTMsKW8HBoaYOxYGGj7C11dXScnJwaDcfRo38eIAkdiYqKRkdH9+/dVVVXT09OJHpcDQkoKAEBYGDZsgKYmXv158AAAwNycVztc0NTURKPR/Pz8NDU1nZycWCzWEDgx2Ay1YmK4Jz4eASB7e27GPnz4kEQiSUhIfJ5WGoMBi8XavXs3sW5m/vz5vCccFRaiZcvQgGqg96G7G4mLIxIJ8a+BZH8QHRXCw8M9PT1NTU05PXGJVr4jsqw2FiwBZv9+BMB9u4Avv/wSAHbt2sVXpz4TRIEwABAWFt69e/cn5lj1T3AwAkC8NCDPz0cASFubd18+SE1NTVxc3Pbt22fOnClFTA7/hkKh2NjYeHl5rVy5kqivfeDAgUF0ZSjAgiXALFmCABDX3YgzMzMBQEZGRuAqkMTExBD155SVlYkyRHyhowPJyyMAxLXJkBAEgL77jl8evae5udnDw2PBggWa/6iup6Gh8d133wUGBmZnZ/dO64+NjSWTySQS6cyZM3z2ZkjBgiXAGBv3AKD8fO4t2NraAgAXgeqhhSgVPWnSpKqqKv5a3r0bAaCFC7kcvnVrs4gIOnKErz4hdPjwYQkJCUKhJCQkbGxsPD09o6Kiqqur+xl16tQpACCTyVevXuWzQ0MHFixBpaOjQ1hYRENjdmcn9/mBNBoNABQUFNrb2/no22Bjb28PAJcvX+a75dra90Eo7jpbmpiYUCijMzL4XEVy7ty5ADB79uyCgoIBpYP6+fkBgKio6H/+8x/+ujRUYMESVHJycgBg6tSpPNohEgKOHj3KF68+D0T06iaRqclv1q5FAGjTpr5rnj9KZ2eniIgImUxuG2gNjX5pbW2lUChkMrmmpoaL4Vu2bAEAaWnpfF6m4sMGnNYgqBQUFAAA7310iL5qvr6+3d3dfHDrs0C4SpR54TteXsjObuPFiyp1dXUDGvjo0SMmk6mnp8e5feMLCQkJXV1dVCpVgas81MOHD3/77bctLS1ffPFFcXExHx0bEj61LyFmuPH48WMAMDQ05MVITU1NYGAgAIiLizc3N8t/erL8kDKogqWjQ5KRqaDTG0+cOEHcUvWGwWBUVlZWVFS8ffu2oqKisrLy7du37969y8rKys3NBQBzfqdgxcTEAICzszN3w4WEhCIiIogULUdHx3v37ikqKvLVwc/LUE/xMFxCxMuTk5O5tvDu3Tt9fX0AmDx5cmFhIR99G2xMTEwAIC8vb5Dsp6WlAcCYMWPa2toqKys9PDwWLlxoamraz1e9oqLihx9+AIDg4GA+etLV1SUtLQ0AZWVlvNhpb28n7v2nTp1K/zxJYoMDnmEJJAihwsJCAJg8eTJ3Fl6/fu3g4FBaWmpsbEyj0QRlbkUwqDMsALC1tZ02bVpWVlZERMT8+fODgoI4u0RFRZWVlcePH6+qqjp69GgKhSIsLNzT0/Pbb7/99ddfAECIKb9ITW0xNPxWRKScx+4qo0aNiouLmz59emFhobOzc1JSEpGoJXgMtWJiuOH169fEutYJEyaEhIQMNG3y+fPnRFtQMzOz+vr6QXJy8NDW1gaAFy9eDN5LXLlyBQC0tLRqamp8fHz8/Px8fHw2btzo6urq4OCgoaHxr60JKRTKBr4W7lu5EgGgffv4Y628vJzoTmhra9slmFXosWAJKgEBAZyOqqampp/+yOzJkydKSkrEVdsimH0siGa9g1oeh8Viqaio9NMwlUQiKSkpmZubOzs7e3p6/v7773v37iXSWffu3csnH94nshYV8cUeQggVFhZSKBQSieTn58c3o58RLFgCTE9PT1RUFKfVtoODw0fDOrm5uWPGjAEAR0fHjo6Oz+Mn31FWVgaAQS010dXVNW7cOAkJCXl5eVNT04ULF3p4eAQEBERGRt69e/fVq1f/2hUlLi5OWFgYAP744w/efbhz5325Lv6ipaUFAL6+vny2+1nAgiXwMBiMY8eOEZ3ySCTS4sWLP9QXJz09nYjgfvXVV52dA84zGj6MHTsWAAZ12TYRt5o6depAb7cvXLhAIpGEhIR47w7t6YkA0LZtPJr5L6qrq4WEhCgUioBOrrFgjRAaGxu9vb3FxcUBQEREZO3atX2WrSQlJRF7XVxcmEzmUPnJFwjZHbz2lJ2dnSoqKgDw119/cTH8wIEDwHN+OZuNJkxAACg7m2sb/8LJkycBYCHXi4+GGixYI4ry8vK1a9cS8XgJCQlvb2/iWx0fH088FVq7di1fChsMLcR7GbxJ4uHDh4nIINctZ7y8vABAWlqa69yLpibk7Iy0tPjcaH3AnPt1AAAaeklEQVTOnDkAEMb1ivmhBgvWCKSwsHD+/PlEYEtRUXHBggVE8Hjjxo0jo+kTUfhpkJS3ra1t3LhxAHDjxg2ujbDZbDc3NwCQl5d//vz5QId7eKDKStTTgwIDuXbhX6DT6aKiomQyWXCLoGHBGrHcv3+fSC4lehGOmJYETCYTAISFhQfJvr+/PwBYWVnxaKe7u9vR0REANDQ03r17N6CxP/yANm1CLNb7JiP8IiIiAgDsuSv5ODzAawlHLNOmTUtNTT169KiKisry5cv37t071B7xh5aWFgBgs9nl5eV8N97W1nbkyBEA2LdvH4+mRERErl27ZmNjU1ZW9sUXX9Dp9H4OfvcO4uNh2zagUsHLC6SlwcQEkpN5dKEvPK7yGRYMtWJiBp2RcRvIobOzk3gkqq6uzvcVRYSsU6lUfhmsr6/X1dUFACsrq941fNrb29PT03///fCiRWwlpf9qO2ZsjDZtQmw2WrGCnzOsjo4OCQkJEon09tN7Lg4/sGBhBI9nz55RqVQAkJSU5O5B3r/S1NRESGFqaiq/bCKEKioqiGeO1tbW586dc3d3NzExIdK1AEBD4xkAkpZGDg5o924UF4fq61FQEEIIFRSgoCAUGIgGHgT7F4jplaWlJR9sDR1YsDACSVdXF7HYmEwm86tiqq+vLwA4ODjwxVpvXFxc4O9gIoGwsLCxsbG7u3tUVEk/S4xCQxEAUlNDFRW8+kCcLoGrLtsHLFgYQYXNZh88eJB4Yrhq1ap/TT3/dOrr64n0Lj4WiSdgs9nEyk2iTpaBgUFaWtonlnjt6EBUKgJA+vrc9KPm0N3dLScnBwBcPLIcVmDBwgg2165dI1bw2djY1NbWcm2HaGg4d+5cPvpGcP/+fQBQU1Ozs7MDgEuXLg1oeFMTMjREAMjSEnFdyvTmzZsAoK+vz+X4YQMWLIzA8+jRI6IIgaamZhFXC4UrKiqIllnZ/M0rRwgh9MsvvwDAunXrhIWFxcTEuEjQLy9/n/W+cCGbu1UK7u7uILDrB3uD0xowAo+hoWFWVpa5uXlpaem0adMSEhI+ZVRLS0tGRsYff/zxww8/6OnptbW12draWlhY8N09ok6WtLQ0i8Wyt7cnbjwHxPjxkJICqqqopuawm5sbm80e0HA6nR4VFQWCntBAMNSKicHwh87Ozu+//x4+HIZvaWlJTU09dOjQ0qVLJ02a1DsEDgCioqK8pLZ/CKLO4tixY4kkUl66BObkFEhKSgLA1q1bP3pwZWVlVFSUp6enjY0NUelQUlJyBCS4YMHCjBx6h+HXrFnT3Nz85MmTkJAQV1dXPT09Tid3AhERET09PVdX12PHjt2/f5/HmP0HXdq/v2Xq1PSff/5SS4tMJvffSfCjpKSkiImJAcBvv/3WZ1dTU9N//vOfvXv3zp07l6ggxEFYWFheXn5kdFQlIYQGdQaHwXxmLl++vHLlys7OThLpvy5vUVFRAwMDs7/R19fnJEMNIiYm8PAhbNsGBw+2fP219LVrPNq7fPny999/jxA6ffr0rFmzMjIy8vLy7t279/Dhw963ikpKSqampqamplQq1dramnguMQLANd0xI42lS5eOGTPG2dm5q6tLR0fH9G/MzMw+dyHz16/h0SOQkoKSEgCQplJ5N7l06dK6ujpPT8/169f39PRwtouJiZmYmEybNs3S0tLKyop4CjHywDMszMikqalJSEiIiwg3PzlyBH76CZYsgeRkaGmBsjLgrZcEh82bNwcGBsrKytrY2FCpVBsbmyGQ46EACxYGM2hMnw4ZGeDtDQEBYGICeXl8tF1TU0OUwfmfAqc1YDCDQ20t3L8PYmJQVQUAwO+Ugv9BtQIcw8JgBov2dliyBADel4lZtGho3RkZYMHCYAYHaWmwt4fRo8HVFe7eBT29oXZoJIBvCTGYQYDFgk2bYOZMkJaGzEz49dehdmiEgAULgxkEysrA0BA0NGD2bKisHGpvRg5YsDCYQUBaGjg1kQe49A/TDziGhcEMAoqKQKGAvz9UV8O33w61NyMHnIeFwQwabW0gJgYiIkPtx8gBCxYGgxEYcAwLg8EIDFiwMBiMwIAFC4PBCAxYsDAYjMCABQuDwQgMWLAwGIzAgAULg8EIDFiwMBiMwIAFC4PBCAxYsDAYjMCABQuDwQgMWLAwGIzAgAULg8EIDFiwMBiMwIAFC4PBCAxYsDAYjMCABQuDwQgMWLAwGIzAgAULg8EIDFiwMBiMwIAFC4PBCAxYsDAYjMCABQuDwQgMWLAwGIzAgAULg8EIDFiwMBiMwIAFC4PBCAxYsDAYjMCABQuDwQgMWLAwGIzAgAULg8EIDFiwMBiMwIAFC4PBCAy8ChZCiItRdDq9rKyMF4N5eXm9j29oaAgODm5oaODaJQBoamoKDg6ura1lMBhPnjzpbaqpqam0tPRDrubn5/+rwd6ecPc3521+yOyAIAa2tbW9ePGi/2NYLFZBQQF3r4LBDB5kPz+/fnbLy8svX768srIyPDzc2tr6ypUrwsLCQUFBM2fOPH/+/Llz52g02sSJE+Xl5Ynjw8LCVqxYsW7dOhKJ1I/ZY8eOUalUcXHxPts9PT1TU1NpNNqcOXP6GZ6YmEgmkwsKCo4fP3737t28vDx5efna2loLC4sNGzZkZWUlJyc7ODg8f/58165d+fn5iYmJhYWFVlZWTCZz3759ycnJ6enp0dHRJiYm+/btMzMzExcXX79+vZOTU0lJiZCQUHJysq6u7pUrVyIiIhITE7W1tSMjI62trUeNGkU4gBDy8vK6f//+pUuXxo8f397e/vTp0+Dg4KKiImtr67Vr19rZ2Xl7eyckJLx7905dXX39+vW5ubm5ublGRkZr1qzJzc3NyMiYNm3a2rVrc3Jybt++bWdnt27duqysLBqNNmXKFF9f3+joaAaD0dXVVVNTo6amRrxuSUnJvn37/vzzT2lpaQ0NjX4/2b4EBgYmJSWFhIS4uLgEBgbOnDnzn8ccOnQoKSkpODj466+/DgwMdHBw+Ocx0dHRq1evnjt3rpSU1IAc6E1lZWVQUNCTJ0+MjY1zcnLOnz/f3Nyso6Nz/fr1mJgYNps9ceLEkpKS4ODgkpISIyOj48ePW1pacoaXlZWdPHny8ePHBgYGoaGh+vr6mzdvVlJSamxsPH369Nu3bw0MDBgMRkREBI1GKysrmzJlipCQUHt7e1hYGI1Gq66u1tPTS0hI0NHRAYD4+Pjm5ubffvvNzs4uNDR07Nixo0ePTkxMvHz5soiIiJKS0unTp2k0mrq6OpPJDAwMfPDggZKSUlJS0tSpUxMTE7W0tIKDg83MzK5cuRITEzN69GgASEpK0tfX5zh87dq1mJgYhJCqqmpISIi5uTlnV2hoaFJSkrGxsaioaJ+zxGKxTp8+bWZm1mf79evXr1+/PnnyZElJyQ+d4ZSUFAaDUVpaGhoa2tTU9Pz5cwkJCRkZmT6HlZSUnDp1qrW1tbu7u6ioSF1dnbMrPj7+ypUrmpqa0tLSxJaXL1+eOnWqs7Nz0qRJqampFy5cYDAYGhoa4eHhCQkJ48aNExMTI76SBgYG5eXlnI/vzp07ERERLBZLVlY2ODj42bNnurq64eHhJiYm9+7dCwsLa2trU1RUPHPmzMSJEyUkJPq5coT72QcAM2bM2L9//6ZNmwDg4cOHoaGh06dPv3fv3v3795OSkq5evUocFhER8erVKwMDAzc3t6ysLACoqak5cuSImJjY1q1bAwICAMDDwyMsLKyrq2vBggUNDQ1ycnKcV/H392cwGE5OTkePHiWTyYsXL+7p6dmzZw+bzV69enVMTExra6u9vf2bN29KSkqmTJmSmZl56NCh77777s8//wQAT0/PhoYGCoVy69YtS0vLVatWAQCDwfD19Q0PD79586acnNzjx49TU1PT0tJmzZolKyv78OHDWbNm7d69W1pauqWlhUwmMxgMAKBQKMQHs3r16tTU1MuXLxMe1tbWiouLZ2RkAICMjAydTtfW1t6wYUNXV9f69evDwsK2bt0KAG/fvn3z5g0ASEtLBwQEFBQUxMfHh4WFrV+/3srKCgBOnz79/fffE3J86dKlefPmff311wCQkJBgY2Pj5uZGvFxAQEB6enpRUdHmzZu9vb1VVVVPnjwpISGxY8eOX3/99fr169XV1QUFBZcuXVJQUHBxcTl27BiFQtm6dau/vz+JRNqyZcuZM2e6u7udnZ1zcnKqq6stLCw2bdpUW1v7888/s9lsAOjo6AgJCWltbXVwcHj9+jVxYn/88Ucymbxly5ba2lplZeU3b97QaLSamhoLCwsmk5mbmztp0iRXV9fCwkLCCNds3779+PHjjY2NtbW158+fDwkJKS4uvnnzZktLy44dO3x8fNTU1Hbt2nX+/PnKysp3796RyeTew/fu3RsSEvL69Wt/f/8ffvghKipKQUFh6tSpbm5uYWFhr1+/ZjKZ7u7uP/30k7q6+suXL3fs2OHn5+fu7r5nz55x48bl5+f7+/vX1NTMnz+fRCLdvHmTUPPq6moGg/HmzZuGhob8/Pxdu3a9ePEiMDDQwsLC3Ny8qqpq+/btxLX65s2bI0eOmJiY3Llzx8HBgU6n02i0lpYWX1/fly9fXrhwYePGjRxvaTRaW1vbjh07tm3bpqamVltb29PTk5ubW1paOmfOHHNz88bGxsjISCMjo5KSEjMzMy0trZiYGDk5OTqdbm1tzbFTUFDw8OFDJycnTU1NHR2d0NBQd3f36OhoCwsLBQWFuLg4Ozs7CoVCo9Fmz56dnp6+efPmo0ePEud24sSJhw8f/umnn65evaqtra2vr3/t2jUjI6Pu7m4fHx9XV9fIyMioqChZWdmWlpampqZ58+YpKysbGBicOnXKx8eHOLi9vd3Hx2fZsmW6urqxsbHHjh178eLFpUuXJCUlfXx8Xr16tXPnTi8vL0lJSQaDwfn4SktLExMTDx8+/OLFi4qKCnd3923btjk5OZWXl9fU1Fy8ePHEiRPFxcXV1dUtLS01NTUKCgr9XDkfESx5eXkdHZ07d+4AgLKysra29uzZs5lMpr6+fm91T0lJCQsL632rcvbs2TVr1mhqasbGxlpYWCxYsODly5cIIT8/PyaTiRCqqqo6ceIEADg6OpLJZD8/P4QQiUTKyMgwNjZOS0vT0tJydXWtqqqqq6v79ddfEUJnz54NDQ1FCKWlpQGAsPB755WUlJqbmwGgvr5eWVmZ2Hj37t2vvvoqKCho3rx5V69eXbhwYWZmZnFx8ffff3/lypWenp7Ro0d3dHRIS0uHhYWJi4s/ffqU4zyTyezo6OD8HPX09PT09ACAkJAQAJBIpLq6OuKFKBQKsYvJZAoJCW3btm3Xrl3E7LK2tvbq1asmJiaPHz/meFVfX8+5BOvr6zk/nr09B4CKioobN27Y29sLCQkxGIygoKAdO3bIysqSSKTi4uLMzMzvvvvuxIkTJ06cEBYW3rt374YNG9TU1K5du0alUh0dHZ8+fSoqKrp9+3YGgxEcHBwSEkKc2+joaCkpKSEhITU1tczMTM6JPXPmDHFiSSRSVVVVa2urioqKhoZGUVFRXl4eMXzFihV9PmJeoFAoUlJSUlJSWVlZVlZWJBJJR0cnOjp62bJlADBjxoysrCxlZWUKhaKpqXnr1i1dXd2UlBTiM1q8eLGioqKYmJiOjk5dXZ2GhkZoaKiIiEh1dbWurq6IiIiWltbt27ft7e2Tk5Pb2toMDQ2bm5v/+uuvZcuWRUZGtrS0LFy4sLy8XFRU1N/fHwAaGxsJrzQ0NMaPHw8Ajx49In5UdHR0jhw54uXlBQATJ04kkUhjxowhPq+1a9cePXpUQkKitLRUTU3t0aNHLi4uJBJJW1u7ublZTEwsKCgIADQ1NR89erR8+XLifT1+/Hj8+PE5OTlXr17dsmULhUKZMmVKcHCwoaHhsWPHAgMD/fz8zMzMlJWVlZSU0tLSFixYcPr06aamJkdHx8DAwAMHDjCZTHl5+Y0bN+7cuXP79u07duxobW3ds2fP7t276XS6r6/v4cOH6XQ6i8UqLi7mnFsA6OjoOHjw4DfffEMmk3ft2uXp6dnV1WVoaHj06NFx48YBAIvFSk5Otra2zszMtLS0VFVV3bhxo6enZ++DOzo6JCQknjx5QszQdXR0zp496+vrSyaTJ02a1NnZOWHCBABobW3lfHxxcXGzZs0iDgaA2NjYyspKWVlZGRmZ7OxsKpXK8VBRUfGjV87HY1gbNmyIjY0FgJcvX1ZVVcXExIiKiiKEWltbGxoaOjo6Ojs7WSwWQqj3r66UlFRjYyObzZaSkqLT6SQSiUKhELJCfO2VlJT279+/f/9+VVXVpqYmAGCz2bGxsdnZ2du3b5eUlCQuIxERkdbWVmIvoXRsNpuwICIiUlVVxWAwCgoKiOvM0tLyr7/+QgjV19e3t7dLS0uXlZW9ffv28ePHt2/fNjY2JpPJz58/l5GRSU9Pz8nJIea6Hh4ev/zyi6Gh4f+fFCEhKSmp2tpaOp3e1tbGZDIBoL29PSMjIyMjo7Cw0NzcPDk5GSGUn59P3K8RLsnKytrY2Dx48KC5uZn4kUlKSrK2tr5+/Tr8rVacv62srKKjo4m/e3teW1srJia2Y8eO+Ph4ACCTyZyT+ebNGxUVFQ8Pjxs3boiJibW1tRFnmHOqiZMmLi5Op9MBgEQidXZ2EmcvLy/P3d29u7u7sbGxqalJSkrqnyf2yZMnhw4dIr5pdDpdQUGhq6uLOAYhxGKxeJxYcejq6mKxWACgpqZWVFREbNTW1n78+DEAFBYWGhoa1tbWEvpIIpF6v66kpGR9fT1x3nrflsrLy7969Yr4u7y8XE1N7d27d8TPAJvNLi8vnzBhQlNTk6ysLEd2fXx8tm/f3nuyT6ClpfXo0SPibxUVFU7Ij8lkEjNxABATE3N2dk5ISCDcmzRpEmdIH7S1tYmAYGFhoba2NkKora1NU1NzwoQJUlJSV69elZGRsbGxUVZWVlBQEBcXb2ho0NXV1dXVJZFICKFVq1Z5eXkpKyurqKgoKiqOHTtWQkIiIiLi0qVLAKCoqKitrd3R0TF27FgtLa1Ro0bJyclpamr2ObcELS0tmpqakydPZrFY48eP19HRodPpP/74Y0dHB+cM6+joKCkpdXZ2ioiIXLx4MSoqinNwY2Pjvn37Dhw4oKmpyQlxamlpcf4mk8ktLS3EZ8T5+CZNmsQ5gE6nOzk52djYFBcXI4TU1NQKCws/dJH8Kx+JYTGZTD09vSlTpsjLy1OpVDk5OSMjoy+//HL06NHW1tYnT55MS0ubMmXK5MmTIyMjCVGn0+lPnz5dvXp1dHR0VlbWkiVL8vLybt26ZWFhwWKx4uPjx44d+/DhQ06UirjDio+Pl5GRodFooqKi9+7d+/bbb58/f06j0fT09CQlJWNjY6WkpMzMzCIiIrq7u5ubm3V1db/88suQkJC0tDQPDw+EUF1d3cyZM8XFxc+cOfPy5cv58+cHBgbu2rWrsbFx6dKl0tLSdnZ2NBptwYIFnZ2dmzZtevXq1Zo1axBC2traIiIinZ2d+vr6z58/l5aWLikpsbW1pVKpwcHBxA15VlaWk5MTlUq1sbGZOnWqjIyMjIzMmTNn6urqfvrpp6ampuLiYktLSx0dHWNjYxaLZWVldenSJRqNtmXLFisrq1evXl28eLG1tZW4Hb5w4UJ9ff28efO6u7uJabOjo6OoqOjZs2dLS0upVGpoaGhaWtrWrVvpdHp1dbW7u3tkZGReXt60adNOnjxZUFDw448/UqlU4p2uWLHi2rVrWVlZLi4uRERs2rRpXV1dCQkJioqKWlpaly9fJpPJnZ2dp0+ftrGxMTc3v379upubG0Koz4nNzs4WFhbOzc2dOHEijUZbtGiRvLw8Ebi0t7cnoiH19fVlZWVPnjyZPn36gC613mhoaAQHB6ekpHzxxRft7e3Xr18npiHJycm3b99WUVGxt7dXUFA4c+ZMWlravHnzUlNTly5damlpaWlpKSoqKi8vHxoa+uDBgy1btpSXlzOZzMbGRgcHh1GjRl24cCEzM3PmzJmJiYnW1tajR4+uqqr6+uuvR48enZ2draOjo6WlVVRU5Orq2t3dbWxsDABVVVXEpH7SpEmxsbGvX79eunRpfn5+UlLSu3fvli9ffu7cuZSUFElJSQcHh8DAwNu3b+vr64uLi9vb29fW1hKytW7duhs3bty6dau1tbWtrc3IyGj69OmWlpZaWlqTJ09OSkq6c+eOiorKrFmz4uLiVqxYkZOT8/jxY4RQVFSUsLAwi8WiUChGRkY1NTVLliwJDw+vqqpSVlbu7u5WUVEREhIaNWrU69evc3Nz5eTkIiIiUlJSVqxYYWBgEBkZ2dPTY2pqSvz4qaurJyUliYiIvHjxYsGCBc3NzcS5NTAwyM/PX7ly5blz51paWhwcHMLDwxkMRllZWVRUlLa2tqmpaVpa2tSpU7W0tNrb21VUVIgIoKurq6WlZWhoKIPBiI+PZ7PZRUVFjo6OFRUVcXFxxcXFbm5uV65cSUlJYbPZzs7OxPnR0tLS0tIiPr7FixeXlZXFxcWVlJQQX0+E0IIFC2g0mouLS319fUxMzJMnT+Tk5JKTk0tKSmbMmNHfpYOGgrS0tLy8PK6H19fXR0RE9N7y4sWLjRs3EqLO4ebNm+vXr9+5c2d4eDixpaGhwdPT08fHZ+fOnQ0NDX3MNjY2enh4FBYWPnv2LCkpqfeujIyMnJycD/lDCBDXb6cfzpw509rayl+bLS0tZ8+e7f8YFosVFBTE39flheDg4A/tOnfuHIPB8Pb27nNFnT9/fufOnQEBAYWFhcSWwMDA3bt3Hzx4sLS0tI+R7OzsHTt2cO3eqVOnenp6OP8logH/eiQRR/9EswM6uA+pqalFRUWc/xI3Yv0cX1hYePfuXe5eiws6OjqIKMRAISE+RSUwGAxmsMGJoxgMRmDAgoXBYAQGLFgYDEZgwIKFwWAEBixYGAxGYMCChcFgBAYsWBgMRmDAgoXBYAQGLFgYDEZg+D9tFSNT7qtOgwAAAdR6VFh0cmRraXRQS0wgcmRraXQgMjAyNS4wMy4zAAB4nHu/b+09BiAQAGImBgiQAGIpIG5gZGNIANKMzGwMCkCaBcGF0g4aQJqZhc0hA0QzMyIxIDIcEJoJQwJiBBMTGwNUIYRmYmcAK2RixGc2Lts4wc7E1MrNwMjAyAR0MAMjCwMLawYTK1sCG3sGEztHAgdnAidXBhMXNwM3DwMPrwYTD58CH78Cv0AGk4BggqBQBpOQcIKwSAaTiCgDl1iCmHgGkzhLghh7ggh/ghMz0Hg2FnExdjZWVg52MS5ONgFBIWERfnEtRqAzGGDB2eF26cC2X4b7QZxVTGsO3Hx+Bsw+u6/1gHamwgEQ2+vviQMrfhuB2dwc/Qe6E8/vA7GN1y08kK1SZQ9iT9nccuCaALcDiD1DVetAp9s/sHiJ2OH9nJ9sweIt/0r3ffflAbPfHhPZv8XFGqzG+8UZuzbPc2Az40++txe1+LYXxJ4YK+4gxbAMYs5mMwfbGQvAbuut7XLYeakCzH7/ZKUDz3k2sNsOcdx0sMh/CRY3PvrCoUDvAtjMi7EnHeyDrMDm/LOc5fD3xWM7ENt8aanDb71TYHGPmywHHm+XArPbztkd4EoXB5sjBgDRL30XU+eIBAAAAkp6VFh0TU9MIHJka2l0IDIwMjUuMDMuMwAAeJx9VVtu20AM/Pcp9gJe8M3lZ2KnRVHEBtq0d+h/74+SUhJtkEVlL6GVR9SQM5RPrY4f1+9//rb3g66nU2vwn29EtN8MAKfnVift8enrt1u7vDw8vl253H/dXn42kkaW9+TnI/bh5f78dgXbpZ2tqyMitDN0CyWmBh2247iXCqmdXZm0nbEbwMizz0gupHT0UMF2pq5ggrFAyv50GhD5eyI9zJLHZ6TuOcXCk13yFFDI7J+RtvMEJcViF561jQXQX2lmTsJGyZcCeAEcBaRuOlS0YY9hSquM0e7VGCVB4swY5iNWwCzxUkWQAFHUs4NIVx1C3JE6EK0a44yKvkKWQNUXoF3JzB2w0gdLIOzDOY9CIiUNWSElK6LOpjAsb6Gw4cuCtN0SmA3ybGH2gFRMbIUsfaQz+Uh3nksfDFvSLIHScOUNKwEQjMey9FLIembkslGWRuywREYivZML+KuNzGBVOsGWk4Icahw8CS/7TrgVNDDUS0ERRV4CqX1p3AcPgshmJlyXmhPvhgNXt3p2MmRbtZ1kR8ZAVt18Eii46vvT7fph8PdXweP9dj1eBfWhY95z0/gY6trKMbm11WM8JZcdMyi5/Jg0yTWOeZJccUyN5MJ5OKQC4jQEuAWazI5b4MnUtU3LTubF7YpOLsUt2ORGqYA+uU4q4JjcJRUwJhdJBZrdIhUIJ1tgBaJJf6kqiCehpQLJJKhUeKdMukPe2VAxruQ2KzvrWPu3v4M8P/0DPhIvngCTW9EAAAEvelRYdFNNSUxFUyByZGtpdCAyMDI1LjAzLjMAAHicLZBLasRADESvkuUMtIX+H0xWhiyTQ/QBcoE5fCRPTIOhKL1S6boe17Pfpr03/+79uK7H58/ze8ueT76emzd9vB6HgwURrQPBy1h4nYeBhImsg8ARU1pSoCjTdTAYuo7LgROLR4pyp9ulXiEDUzRUumFobLQQKjop37D28eKGcrXA4JZNJ6h0mzwCY6WxlMdYEFiR291Dxey3ZEnkTQ4ho1jnxCLTO58LbZ0EGSL3SsQN0HUyiBtWp3F55ggdHlOXTb2HFIQj76EKqpH6Jv9lCV16I4fIOVdP9T+6agCHYrzT3VHHxMUxGzZvNlRIKstWVI2kFYGU7J27cV/Y72tgWEyvZojnLVWS+KCtSLv08/UH2RZjUPdT9YkAAAG0elRYdHJka2l0UEtMMSByZGtpdCAyMDI1LjAzLjMAAHice79v7T0GIBAAYiYGCBAFYnEgbmBkY0gA0ozMbAomIHlGFogAExOEZmZkZ8gA0UyMbA4aIAYLmwNYACgDEWCGCyBUQGiYAgyFcAGY5XATYJaDXcMIcw0jM0yCm5GBhYmRiYGJGWg+AwurAitbBhMbewI7RwYTB2cCJ5cCF3cGEzdPAg9vBhMvXwIfPwOnQAYTsyCDoBCDkDALo5AIgwgjgwBrAh9XghMz0Ew2RhEhQWYmNjZ2Dk4BVlZuLj5eHnE2RqClDLDAslp2xmGqKdsBEOf01OUO1tmT9oPYoe6lDuoXroLZFWraDnkLzMHsnJZF9sqm88FsNe0ntuevTNkHYr9X3W63Qn+GPYhd88Fn/xlxTgcQO1GB98CaU+fA4kcfyxyo0+O2A7HZT5QdKPpzHKx3tW7Hgb18r8FmNinuOPB4xmcwe++vEwdqHf+D1cwvWHzgS+R9sF7NyUsP3Fz6HWzmxdal+x1MAsHq77y0cvDUdAaL86d3OUz8uQTMPugxycFM3xDsntry9Q6tz1aDzREDABcRa0hbWQgEAAACHHpUWHRNT0wxIHJka2l0IDIwMjUuMDMuMwAAeJx9VEtuGzEM3fsUuoAFfiVy0UVsp0FRxAZat3fovvdHSRmOFEDojEmMNU8kxfc4h5LXj8v3P3/Lx0WXw6EU+M/P3ctvBoDDe8mHcnp9+3Yt5/vL6blyvv263n8WwkIce+L+jH25396fK1jOpVV2JcdypAregWJHhXHNrRRArcQdWMsRKwqb9w2QA8jVmKR7AlsHQd0AJYAUr4WizCPUjp2cNkAt14KVmrfWMiJFavANsEVEqIgoCBmRHJq1DbAPIGvviBEazVxlg7PIHHEMQEWiWBSCvju0R8DoHoEiZMSGqm3XxihsIEVRIA+LrL5tOCY1x+ikqqtnGeygbjsoZZ1SSVX7YMe4se0KRU6o1q4CDgl1YATeQZOgY6tk0MCyAHFT2UbVR1SELqNC4Ua8DZocJTJojwSRniX27JBJUrLtjNIGDSEQ2NGENqQUfVTK9z2S72MmT1KZFAxSU2bWdgIhGEDtyBZjFBFNYXdywjEXYsycgThEJbuDv14vnwbvMYqn2/UyRzEy5UB+ffvCc+xykedwYZjMEcIwnYOCYW2Og4T1KXoJs6ltCfOpYAnDVag4HC6ClHRIi+4kHfIiL0mHsqhI0qEuYsHh2iIKGbn6Qv7IZgvFOJwvXGK6pCo6hgt1j3VcOMLxNfxYGHX32SkaZety2KRrJSf/P7+x8Xz4B0UBEMMOMTi9AAABLXpUWHRTTUlMRVMxIHJka2l0IDIwMjUuMDMuMwAAeJwlULtuAzEM+5WOF8A29LSl3BLAHTp161R0uv3WLPn40pdNoEiJ5Pydj8fXH8+5fR9ynMex1UPP8zh13g65zfd+m7fJH6+tN02XLFUa5SApuzfRQVYqNzbNsmsLFYsFdCy47ILBREqlNnhIQsVNevaxSGIaIFFjZrBBkqQefUHqYzAXnI5ILzuWQeRmRfBNKABJE/LF6ezeL8CcreCg+vpV4cg9+zqtSZ5LZU3c/TIQ2pfv6m24US4oSWGp9iZB/fJkGW7jYjENx3XTLqpvBPGWg1QbyyVCpbIvXSAw2kEHCZsC3YDM8M+aiiMByBExFuCDNRBt9EAiVGuhupIoijAtt/L8vHOj8vy5cyD36x99flzngGR1GAAAAht6VFh0cmRraXRQS0wyIHJka2l0IDIwMjUuMDMuMwAAeJx7v2/tPQYgEABiJgYIkAViBSBuYGRn0ACJM7NBaBY2BxDNDKQzQDQzIxIDJkNIBQfYLGYmNgUTIM3IyAIxHCEBsZQZaGkCyFImnDQeqyhicDMwajAzMikwMWcwMbMksLBmMLGyKbCxZzCxcyRwcGYwcXIpcHOxMHHzMPDwajDx8Cnw8TPwCzAICDIICjFwCzMIi2QwiYgmiIplMImJJ4hLAHVIJkhKAY2RTpCWyWCSYUqQYksQ4mOQEE5wYgFaysYkI83KwszGzsEpKcXGys8nJCjAJiIqJi4hLJ7GCAwlBljUeFkrHvQSaN8P4ui/5DwolWUMZmvoPT1gn75xH4j9de31A6+edtuD2Ps5Vx5Qd9wJZu952n5gVdFOOxCbp9nowPl8c7A4d9mD/UGb9oD1npoyfd+i1Rl7QOzjdxjskvK2gdUUrjllP//kYzDbfp66g9spQzD75lweB92tdmA3bA21tb+8tQfMfvDUz0HI/hWYrcU63eH84gtgNu/51Q65N+0OgNjvRDsddk1oALPb19g67J8QDGaHFjc5JDi0gc2PnL7Yob5ZZC+I3W99yaH1ziKw+w/tvu9w69sDsJp90/c5rH5r6gBid3zud+B1kAezX7zI2J+/UgHMXtYkfuDib04w+33q1AP8jzvB7lHlOnLAx247mC0GAMnIlvUnFeKAAAACr3pUWHRNT0wyIHJka2l0IDIwMjUuMDMuMwAAeJx9VUtuWzEM3PsUukAE8U8uukjsNCiK2kCb9g7d9/4oKTd5CiD02aIleUyJ5Ax9avV8v3z9/ae9P3g5nVob/3lHRPtFY4zTt1aT9vT88uXazq+PT28759vP6+uPhtEI8zf5+oh9fL19e9uBdm0PMPowDZOc9iECLi235nP8GNu5PXhXCM7Vw+g2hpNvkFRI66DgzoUkVg7YILmQ2o0i8sw8PDCEN0ApoHR0ZM5bd2ajsfOoBeSOrBbliBRx0AZoBcRuHsgVhIGKjA3QCwjdhHTgjEZdeBd3FHJ0DB1iNRvAuL0lZJVyG1CcscJBycB2QGi3/F6cyC0nZiHbTEKVB6s6g8qRBnPs4gaaQAgDmsWxBArukJxnjyqOeExqRJYndkhJGlFHQvNRSE+GbpMJVR/OZAfdkZoe9z6rQJLJjoFSlQrHgdvQffpEp0Cr4g8wVdshY8YeSlnr9kCdslCwuyeO6TNlIRZFTE4qb4EwrwmAWNdMQDJOduzAqpB2SaFZeSLw4bvIkSYwAlTrbJPk3q6UyPPskCQmZVzJJaVdhlBmNBzO5gnMYNB3dEO9UzhG+OBECiSdti7/6YfUiKMIJRKx7Rt+l6SqazKyeGSGvGMcxr0fACpmZ0koE8g2Sc/Xy4dGdm9tT7fr5WhtlAOP9pWLRkeP4hpHI6qXHO0mF02PpgI57GgdnMOPBsE54ugCnANWrcM0sIgaSrD5ef788okWDd/3aRFrLVOKiyhh7sgiPphGF5HBNLaICabxRTQwTSzimKfjKgIug7CwncsgLqzmMkgLfbkM8sJTLoOyEJIrJ6gL87gM2sIwzqyjL0TiMhgLX3j+z71Xdd7YjlLRTHUs6aJ5YVnCLCqtxKn12/9pzk9/AZ8tazXusarPAAABcXpUWHRTTUlMRVMyIHJka2l0IDIwMjUuMDMuMwAAeJwlkU2KG1EMhK8SyMaGttD/T8zAgGeR1cwBQlZ9iGzm8CnZD5p+fDxJVaXPn49TzvO83E7d39efx/v777+Xx+Xt6/ppDxy7nnbuweVUfPLj+3ITJq46bkIcIX3cb00p43bcmIq5bVmRpHQvM08fAUsqm5kDlaMTIEHa6g7iXgbgpJ41B6pSlRcpVY8GUEnGVglVWD5bZ4fvOCad5KiFLK4sGMgkGu267TXUj7tQtNkqqJqY464rHWOYctzHlsiUvMyAhG4f6I6ep+fZR0ZqWr2gjRmiHOrGniTxICBtWA+on1Zc8EDbUAuLLJW1kyYt4cPIIF32DVdGb0De7dtGRFWfrhBINBolRQ4WAPfSDFUgM4KOmwuS2bKJxFAluF85Tj6NoJTQV1+BDU+zA4Ww+TNoy7IAkQisabeR2fnyXbUBYoeiiRuQmyDC6/Hv45cICX//ByVOeufLA3RNAAACKnpUWHRyZGtpdFBLTDMgcmRraXQgMjAyNS4wMy4zAAB4nHu/b+09BiAQAGImBgiQA2JFIG5gZGNIANKMzBCaiYmDQQNIMzOxQWgWOF/BBCTPyAJTiE6zQxQyY2h0gPDZHaDyUD6CzgCLMyIxoDIwp2GowDALh5tw0dwMjAyMTAxMzArMLBpMrMwsTKxsDGzsDOwcDBycDJxcClzcGkxcPAo8vAm8fBlMfPwK/AIZTAKCCYJCGUxCwgxCIgkiokBxsQQx8QRxCQYJSQZJKQYpaQZpGQZOWQZZVgZxngwmUf4EGTEGJxaglWysspwc7GysvDziYnxsAoJCIqL87BLiYjLSUpLiRYxAXzLAYqZs43uHhd90HECcO+cOOGxOkQCzj7lucqieVWwPYuskNzscDVIEswOE3Rw2NN4Ds7W3ZDr0LPXZD2LPNJ7qIKL9BcxuXtXmMG9y6AEQ+2K/vkOPahaYPWXnXftUS2Uwe/2eDrug0+Zgdt5K+32bl3SC2cdy7faH73sINqdO6NG+a7un7QOx67Pu7w9698QOxDbZ9WX/He4fYDdkupjsT/liAnbzpcdx+130usDs82v4D0hnzwWzV4ZIHdCVPwtmn/sRc+DB5wYw+0tA0IE59UpgdtpCswNfQwTA9k5l5j/Aw/cdzNZdZ3WgjTUQ7LY98T0HDumngNkPDTYesJ6uBmYbCW448MG6F6w+8Gbnga8TPcBu/v6O22FhWT9YXAwAwQCXN2ix9kcAAALIelRYdE1PTDMgcmRraXQgMjAyNS4wMy4zAAB4nH1Vy27cMAy871foB1bgUxIPPSTZNCiKbIA27T/03v9Hh9okVgCh9pqQ5TFNDofcU8njx+X7n7/l45DL6VQK/ecXEeW3EtHpueSi3D8+fbuWh9e7+/edh5df19efBStVvIPzM/bu9eX5fYfLQ+nVgoy1SO0k7nip0jyONwW4VvFUnIHTwWxtg9PyUrx6Z+KOx+HsfYcz+LPK5MPzcVPiFhucw59WCguJwrXrUN3F1+BPa3NuwuVMtUcY8QbY54dbE7coZ65B3ZtvgGMChYaylbNW1c66AwaAYM6Uwdw5gxhusgEylevMQURaOUt1J+1jh8yyUJXWCbEBOTr3sfs4CwhCvpwIKmerYFJ9C1U4TWpMR9PMvTdNujZQQ6SAmrFbz5WEGW2T8vQKX+amWXIz8LR12m7IYA4VEBFIvm199rdIoUqoAulLx2qHHDfkAFHNUC/lwbalNMsEJsUgJQEyMgrbSZ1uSIPEEUnDgmjLqGSZUHFzaaPBJ0l0VGGDlBtSRuPIBnIN0p3gRZN6pOzaaErZm/bYJS/2lpKah01qUfwtT+I3aHAbg2ckzWNLvswygcnRMilA0R/QwQ4664R21y5gCv4bGBhbAsYblAQjLGNlUo1tAPEWAPWO7khxh4/tdMhhBzJ5NGs0naJfaVf+x+vl09y7TcL7l+vlmIR5yjHwcFP0mGuMy47xBQkVP6YUYwKhHF+fvugxknKzH4OHcY1jvDCuOIYI57XOCp6Gl5kgaViW3ue5o0uLWxq2pZMtDfvSsDxNWxrT0nBfGtDS8FgazdJwLA3FaWRtHJs7vDSIpRFZGiGDgcwXwVsasUXXnEZ8kS+nkbaolNNIX8TIaWQsmuM0Eou0Jqu6KojTfDjWGbItpGqGjLgPwjTVAF9HEimvVUx5//6XjPXpH7DldwBE7uWtAAABfXpUWHRTTUlMRVMzIHJka2l0IDIwMjUuMDMuMwAAeJwtkTuO3TAMRbeS0gZkgX+KMQIMoBSpMgsIUrmfNs0sfq704sbGge+H5Jzv8/jxfv6Zb2+//vKc8/exwSOPzvV86HNcjz7Pc8zzefT8kHPyt88juxUZN+lJ4t7u6HiLA+hgBvDuySSNejl7RrutM/kIkFDiqHZrp7LSxj11qC4QziHtop5VRrxUESvh4l6Uvn2EhnK7tKsmQ4YWpgi9ln64tXs5imi7pLuTjnZTl0gKX2gkJxBSeH22yzoaaWwEqxG68jIUXW4IRM1rV8B/L1QcY6xECa+lhMcITAXk4djNfWEHmiIrMkhqvBAJLysm1XrpKFFntSnsp7b9cNQB8tDc9qhjbobtmWFo3qiY9/pKc/xvj6vAAgaSJhsNjB3WkMNjXWbNY1YCUpC/xjFm/B54Ey0n7eYCI8NdK3MTGcHL2hW339Zm7Jarp5QZro1T8AiLPSDOxO1s/35+926fX2sDfxdWk9mhAAAAAElFTkSuQmCC",
      "text/plain": [
       "<IPython.core.display.Image object>"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Draw.MolsToGridImage(df.mol[0:4],molsPerRow=2,legends=list(df.canonical_smiles[0:4]))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "19971c35",
   "metadata": {},
   "source": [
    "We need to create a mapping from index to atom type, and atom type back to index so that we can decode the atoms. Think of it like a tokenizer encoder/decoder. This will be a dictionary. Same thing for the edges.\n",
    "\n",
    "We will scan through our dataset and create our vocabulary which maps atom symbol to indice, and vice versa"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "a08579ac",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Atom to token: {'C': 0, 'O': 1, 'N': 2, 'F': 3, 'S': 4, 'Cl': 5, 'Br': 6, 'I': 7}\n",
      "Token to atom: {0: 'C', 1: 'O', 2: 'N', 3: 'F', 4: 'S', 5: 'Cl', 6: 'Br', 7: 'I'}\n",
      "Edge to token: {'NONE': 0, 'SINGLE': 1, 'DOUBLE': 2, 'TRIPLE': 3}\n",
      "Token to edge: {0: 'NONE', 1: 'SINGLE', 2: 'DOUBLE', 3: 'TRIPLE'}\n",
      "Bond types in dataset: {'SINGLE', 'AROMATIC', 'DOUBLE', 'TRIPLE'}\n",
      "Max number of atoms in any given molecule: 37\n"
     ]
    }
   ],
   "source": [
    "a2t = dict() # atom to token\n",
    "t2a = dict() # token to atom\n",
    "e2t = dict() # edge to token\n",
    "t2e = dict() # token to edge\n",
    "\n",
    "# for a given molecule, what is the maximum amount of atoms which are present \n",
    "# used to set the output matrix size when doing softmax\n",
    "max_atoms = float('-inf') \n",
    "\n",
    "# RDKit has a lot of options here, but we will constrain to these 4\n",
    "# and maybe make a note of which molecules have bonds we wouldn't expect\n",
    "# https://www.rdkit.org/docs/source/rdkit.Chem.rdchem.html#rdkit.Chem.rdchem.Bond.GetIdx\n",
    "# note that we are treating aromatic as a \"double\" bond\n",
    "BOND_TYPES = [\"NONE\", \"SINGLE\", \"DOUBLE\", \"TRIPLE\"]\n",
    "for idx, bond_type in enumerate(BOND_TYPES):\n",
    "    e2t[bond_type] = idx\n",
    "    t2e[idx] = bond_type\n",
    "\n",
    "bond_types_in_dataset = set()\n",
    "\n",
    "for mol in list(df.mol):\n",
    "    max_atoms = max(max_atoms, mol.GetNumAtoms())\n",
    "    \n",
    "    for atom in mol.GetAtoms():\n",
    "        \n",
    "        symbol = atom.GetSymbol()\n",
    "        \n",
    "        if symbol not in a2t:\n",
    "            t = len(a2t)\n",
    "            a2t[atom.GetSymbol()] = t\n",
    "            t2a[t] = atom.GetSymbol()\n",
    "\n",
    "        for bond in atom.GetBonds():\n",
    "            bond_types_in_dataset.add(bond.GetBondType().name)\n",
    "\n",
    "print(\"Atom to token:\", a2t)\n",
    "print(\"Token to atom:\", t2a)\n",
    "print(\"Edge to token:\", e2t)\n",
    "print(\"Token to edge:\", t2e)\n",
    "print(\"Bond types in dataset:\", bond_types_in_dataset)\n",
    "print(\"Max number of atoms in any given molecule:\", max_atoms)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bf01d73f",
   "metadata": {},
   "source": [
    "Next, we'll define some data parsing functions which will be applied to each molecule."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "11ca4e75",
   "metadata": {},
   "outputs": [],
   "source": [
    "# TODO -> make these utility functions that we download from my repository,\n",
    "# or really simplify these to be much smaller (we only really need to do GetSymbol and GetIdx)\n",
    "# for atom_info() function, e.g.\n",
    "\n",
    "def atom_info(mol, verbose=False):\n",
    "    \"\"\"Extract RDKit info from mol object's atoms\n",
    "    \n",
    "    Iterate through the atoms in `mol` and return information such as\n",
    "    the atom symbol, atomic number, bond degree, etc.\n",
    "    \"\"\"\n",
    "    atoms = []\n",
    "\n",
    "    if verbose:\n",
    "        print(\"ATOM_INFO:\")\n",
    "        print(\"SYMBOL INDEX NUMBER DEGREE\")\n",
    "        \n",
    "    for atom in mol.GetAtoms():\n",
    "        symbol = atom.GetSymbol()\n",
    "        idx = atom.GetIdx()\n",
    "        num = atom.GetAtomicNum()\n",
    "        deg = atom.GetDegree()\n",
    "\n",
    "        atoms.append((symbol, idx, num, deg))\n",
    "\n",
    "        if verbose:\n",
    "            print(*atoms[-1])\n",
    "\n",
    "    if verbose:\n",
    "        print()\n",
    "\n",
    "    return atoms\n",
    "\n",
    "\n",
    "def bond_info(mol, verbose=False):\n",
    "    \"\"\"Extract RDKit info from mol object's bonds\n",
    "    \n",
    "    Iterate through the bonds in `mol` and return information\n",
    "    such as which atoms each bond is connected to and the type\n",
    "    of bond (`bond_num`)\n",
    "    \"\"\"\n",
    "    bonds = []\n",
    "\n",
    "    if verbose:\n",
    "        print(\"BOND_INFO:\")\n",
    "        print(\"INDEX BEGIN_ATOM END_ATOM BOND_TYPE BOND_NUM\")\n",
    "\n",
    "    for bond in mol.GetBonds():\n",
    "        idx = bond.GetIdx()\n",
    "        atom1 = bond.GetBeginAtomIdx()\n",
    "        atom2 = bond.GetEndAtomIdx()\n",
    "        bond_type = bond.GetBondType()\n",
    "        bond_num = bond.GetBondTypeAsDouble()  # TODO: need to convert this\n",
    "\n",
    "        bonds.append((idx, atom1, atom2, bond_type, bond_num))\n",
    "\n",
    "        if verbose:\n",
    "            print(*bonds[-1])\n",
    "\n",
    "    if verbose:\n",
    "        print()\n",
    "\n",
    "    return bonds\n",
    "\n",
    "\n",
    "def connect_info(mol, verbose=False):\n",
    "    \"\"\"Get graph connection info from mol object\n",
    "    \n",
    "    First create an adjacency matrix using RDKit function\n",
    "    and then convert it to a sparse matrix for PyTorch Geometric\n",
    "    to batch process\n",
    "    \"\"\"\n",
    "\n",
    "    rdkit_adj_matrix = rdmolops.GetAdjacencyMatrix(mol, useBO=True)\n",
    "    adj_matrix_tensor = torch.tensor(rdkit_adj_matrix, dtype=torch.int32)\n",
    "    sparse_matrix = dense_to_sparse(adj_matrix_tensor)\n",
    "    # note it is inefficient to completely store the sparse matrix instead of just the dense matrix\n",
    "    # because by definition the fully connected dense matrix has a connection between every node \n",
    "    # to mitigate passing this around, we could create the dense matrix on the fly in model.py,\n",
    "    # but it just makes things more complicated as the adjacency matrices will be different sizes\n",
    "    # depending on the num of atoms in the molecule\n",
    "    fc_adj_matrix = adj_matrix_tensor\n",
    "    if torch.any(fc_adj_matrix == BondType.ZERO):\n",
    "        raise ValueError(\"adj_matrix has some BondType.ZERO edges which we override! Unexpected results will occur\")\n",
    "    if torch.any(fc_adj_matrix.diag() > 0):\n",
    "        raise ValueError(\"Adjacency matrix has non-zero diagonal elements which leads to unexpected results\")\n",
    "        \n",
    "    # we set these to a non-zero value so that later we can include them for our bond prediction loss\n",
    "    fc_adj_matrix[fc_adj_matrix == 0] = BondType.ZERO\n",
    "    fc_adj_matrix.fill_diagonal_(0) # removes self-loops\n",
    "    fc_sparse_matrix = dense_to_sparse(fc_adj_matrix)\n",
    "    \n",
    "    if verbose:\n",
    "        print(\"RDKIT_ADJ_MATRIX:\")\n",
    "        print(rdkit_adj_matrix)\n",
    "        print()\n",
    "\n",
    "        print(\"SPARSE_MATRIX:\")\n",
    "        print(sparse_matrix)\n",
    "        print()\n",
    "\n",
    "        print(\"FC_ADJ_MATRIX:\")\n",
    "        print(fc_adj_matrix)\n",
    "        print()\n",
    "        \n",
    "        print(\"FC_SPARSE_MATRIX:\")\n",
    "        print(fc_sparse_matrix)\n",
    "        print()\n",
    "\n",
    "    return sparse_matrix, fc_sparse_matrix\n",
    "\n",
    "\n",
    "def node_feature_matrix(mol, a2t, verbose=False):\n",
    "    \"\"\"Returns a node feature matrix for PyTorch\n",
    "\n",
    "    Also return a label for the number of atoms of each token\n",
    "    in the molecule\n",
    "    \"\"\"\n",
    "    atoms = atom_info(mol)\n",
    "    \n",
    "    # SMILES -> atom index mapping, i.e. smile2atom[0]\n",
    "    # means that the first (zeroth) element of the smiles string\n",
    "    # is the atom index smile2atom[0]\n",
    "    # we flip this around to get atom2smile[0] means atom index 0 appears\n",
    "    # appears at the position atom2smile[0]\n",
    "    # note that this is kind of unnecessary since we already canicalize the SMILES \n",
    "    # and so it will always be in the right order... thus we simply need to convert the position\n",
    "    # into one hot vector. we add it for completeness... it would otherwise just be torch.arange(len(atoms))\n",
    "    smile2atom = mol.GetProp(\"_smilesAtomOutputOrder\") \n",
    "    smile2atom = [int(x) for x in smile2atom.strip('[]').split(',')]\n",
    "    atom2smile = torch.zeros(len(atoms), 1, dtype=torch.int32)\n",
    "    for smile_pos, atom_idx in enumerate(smile2atom):\n",
    "        atom2smile[atom_idx] = smile_pos\n",
    "\n",
    "    matrix = torch.zeros(len(atoms), 1, dtype=torch.int32)\n",
    "    # we have an extra dimension here so that PyG working concatenates all graphs along row 0\n",
    "    boa = torch.zeros(1, len(a2t), dtype=torch.int32)\n",
    "    \n",
    "    for atom in atoms:\n",
    "        symbol, idx, _, _ = atom\n",
    "        matrix[idx] = a2t[symbol]\n",
    "        boa[:, a2t[symbol]] += 1\n",
    "\n",
    "    if verbose:\n",
    "        print(\"NODE_FEATURE_MATRIX:\")\n",
    "        print(matrix)\n",
    "        print()\n",
    "\n",
    "        print(\"POS_FEATURE:\")\n",
    "        print(atom2smile)\n",
    "        print()\n",
    "      \n",
    "        print(\"BAG_OF_ATOMS_LABEL:\")\n",
    "        print(boa)\n",
    "        print()\n",
    "\n",
    "\n",
    "    return matrix, atom2smile, boa\n",
    "\n",
    "\n",
    "def map_rdkit_bond_types(rdkit_edge_types, verbose=False):\n",
    "    \"\"\"Convert RDKit bond types to our edge type set\"\"\"\n",
    "    # RDKit has a lot of options, but we are squashing them down to 4\n",
    "    # https://www.rdkit.org/docs/source/rdkit.Chem.rdchem.html#rdkit.Chem.rdchem.Bond.GetIdx\n",
    "\n",
    "    our_edge_types = rdkit_edge_types.clone()\n",
    "\n",
    "    # convert all bond types other than single, double, and triple to single\n",
    "    # TODO we can use classes/enum here to make this more explicit\n",
    "    our_edge_types[rdkit_edge_types > BondType.TRIPLE] = BondType.SINGLE\n",
    "\n",
    "    # specially set aromatic to double\n",
    "    our_edge_types[rdkit_edge_types == BondType.AROMATIC] = BondType.DOUBLE\n",
    "\n",
    "    # set zerobond type to 0 (earlier we set all unconnected bonds to this bondtype)\n",
    "    our_edge_types[rdkit_edge_types == BondType.ZERO] = 0.0\n",
    "\n",
    "    if verbose:\n",
    "        print(\"OUR_EDGE_TYPES:\")\n",
    "        print(our_edge_types)\n",
    "        print()\n",
    "\n",
    "    return our_edge_types"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "96f704a2",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "C(O)=CN\n",
      "\n",
      "ATOM_INFO:\n",
      "SYMBOL INDEX NUMBER DEGREE\n",
      "C 0 6 2\n",
      "O 1 8 1\n",
      "C 2 6 2\n",
      "N 3 7 1\n",
      "\n",
      "BOND_INFO:\n",
      "INDEX BEGIN_ATOM END_ATOM BOND_TYPE BOND_NUM\n",
      "0 0 1 SINGLE 1.0\n",
      "1 0 2 DOUBLE 2.0\n",
      "2 2 3 SINGLE 1.0\n",
      "\n",
      "RDKIT_ADJ_MATRIX:\n",
      "[[0. 1. 2. 0.]\n",
      " [1. 0. 0. 0.]\n",
      " [2. 0. 0. 1.]\n",
      " [0. 0. 1. 0.]]\n",
      "\n",
      "SPARSE_MATRIX:\n",
      "(tensor([[0, 0, 1, 2, 2, 3],\n",
      "        [1, 2, 0, 0, 3, 2]]), tensor([1, 2, 1, 2, 1, 1], dtype=torch.int32))\n",
      "\n",
      "FC_ADJ_MATRIX:\n",
      "tensor([[ 0,  1,  2, 21],\n",
      "        [ 1,  0, 21, 21],\n",
      "        [ 2, 21,  0,  1],\n",
      "        [21, 21,  1,  0]], dtype=torch.int32)\n",
      "\n",
      "FC_SPARSE_MATRIX:\n",
      "(tensor([[0, 0, 0, 1, 1, 1, 2, 2, 2, 3, 3, 3],\n",
      "        [1, 2, 3, 0, 2, 3, 0, 1, 3, 0, 1, 2]]), tensor([ 1,  2, 21,  1, 21, 21,  2, 21,  1, 21, 21,  1], dtype=torch.int32))\n",
      "\n",
      "NODE_FEATURE_MATRIX:\n",
      "tensor([[0],\n",
      "        [1],\n",
      "        [0],\n",
      "        [2]], dtype=torch.int32)\n",
      "\n",
      "POS_FEATURE:\n",
      "tensor([[2],\n",
      "        [3],\n",
      "        [1],\n",
      "        [0]], dtype=torch.int32)\n",
      "\n",
      "BAG_OF_ATOMS_LABEL:\n",
      "tensor([[2, 1, 1, 0, 0, 0, 0, 0]], dtype=torch.int32)\n",
      "\n",
      "OUR_EDGE_TYPES:\n",
      "tensor([1, 2, 1, 2, 1, 1], dtype=torch.int32)\n",
      "\n",
      "OUR_EDGE_TYPES:\n",
      "tensor([1, 2, 0, 1, 0, 0, 2, 0, 1, 0, 0, 1], dtype=torch.int32)\n",
      "\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "tensor([1, 2, 0, 1, 0, 0, 2, 0, 1, 0, 0, 1], dtype=torch.int32)"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAakAAAGiCAYAAABd6zmYAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjMsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvZiW1igAAAAlwSFlzAAAPYQAAD2EBqD+naQAAReVJREFUeJzt3XlYVNfBP/DvsA0gDKswoIC4xA1xRUTjEkVxjVuzGJuoNaZJMU8UaxNtYmKbX0y0fWPTpsmbTW3eqNFENFKXIipuiEpcUAQFEUEFFGRYhGFgzu+P0xklUQMDMhf8fp5nHp25dy7nwNz53nvOueeqhBACRERECmRj7QIQERHdD0OKiIgUiyFFRESKxZAiIiLFYkgREZFiMaSIiEixGFJERKRYDCkiIlIshhQRESkWQ4qIiBTLaiH18ccfo0OHDnB0dER4eDiOHTtmraIQEZFCWSWkvv32W8TExODtt9/Gjz/+iN69eyMqKgqFhYXWKA4RESmUyhoTzIaHhyMsLAz/+Mc/AABGoxEBAQF49dVX8cYbbzR3cYiISKHsmvsHVldXIyUlBUuWLDG/ZmNjg8jISCQlJd3zPXq9Hnq93vzcaDSiuLgYXl5eUKlUD73MRETUtIQQKCsrg7+/P2xs7t+o1+whdfPmTdTW1sLX17fO676+vkhPT7/ne1asWIHly5c3R/GIiKgZ5ebmon379vdd3uwhZYklS5YgJibG/Fyn0yEwMBC5ubnQaDRWLBkREVmitLQUAQEBcHV1feB6zR5S3t7esLW1RUFBQZ3XCwoKoNVq7/ketVoNtVr9s9c1Gg1DioioBfulLptmH93n4OCA/v37IyEhwfya0WhEQkICIiIimrs4RESkYFZp7ouJicGsWbMwYMAADBw4EKtXr0ZFRQXmzJljjeIQEZFCWSWknnnmGdy4cQPLli1Dfn4++vTpg127dv1sMAURET3arHKdVGOVlpbCzc0NOp2OfVJERC1Qfb/HOXcfEREpFkOKiIgUiyFFRESKxZAiIiLFYkgREZFiMaSIiEixGFJERKRYDCkiIlIshhQRESkWQ4qIiBSLIUVERIrFkCIiIsViSBERkWIxpIiISLEYUkREpFgMKSIiUiyGFBERKRZDioiIFIshRUREisWQIiIixWJIERGRYjGkiIhIsRhSRESkWAwpIiJSLIYUEREpFkOKiIgUiyFFRESKxZAiIiLFYkgREZFiMaSIiEixGFJERKRYDCkiIlIshhQRESkWQ4qIiBSLIUVERIrFkCIiIsViSBERkWIxpIiISLEYUkREpFgMKSIiUiyGFBERKRZDioiIFIshRUREisWQIiIixWJIERGRYjGkiIhIsRhSRESkWAwpIiJSLIYUEREpFkOKiIgUiyFFRESK1eQh9c4770ClUtV5dOvWzby8qqoK0dHR8PLygouLC6ZPn46CgoKmLgYREbUCD+VMqmfPnrh+/br5cejQIfOyhQsXYvv27di8eTMSExNx7do1TJs27WEUg4iIWji7h7JROztotdqfva7T6fDll19i/fr1GDlyJABgzZo16N69O44ePYpBgwY9jOIQEVEL9VDOpC5evAh/f3907NgRM2fOxJUrVwAAKSkpMBgMiIyMNK/brVs3BAYGIikp6b7b0+v1KC0trfMgIqLWr8lDKjw8HGvXrsWuXbvwySefIDs7G0OHDkVZWRny8/Ph4OAAd3f3Ou/x9fVFfn7+fbe5YsUKuLm5mR8BAQFNXWwiIlKgJm/uGzdunPn/oaGhCA8PR1BQEDZt2gQnJyeLtrlkyRLExMSYn5eWljKoiIgeAQ99CLq7uzsee+wxZGZmQqvVorq6GiUlJXXWKSgouGcflolarYZGo6nzICKi1u+hh1R5eTmysrLg5+eH/v37w97eHgkJCeblGRkZuHLlCiIiIh52UYiIqIVp8ua+3//+95g0aRKCgoJw7do1vP3227C1tcWMGTPg5uaGuXPnIiYmBp6entBoNHj11VcRERHBkX1ERPQzTR5SeXl5mDFjBoqKitC2bVs8/vjjOHr0KNq2bQsA+PDDD2FjY4Pp06dDr9cjKioK//znP5u6GERE1AqohBDC2oVoqNLSUri5uUGn07F/ioioBarv9zjn7iMiIsViSBERkWIxpIiISLEYUkREpFgMKSIiUiyGFBERKRZDioiIFIshRUREisWQIiIixWJIERGRYjGkiIhIsRhSRESkWAwpIiJSLIYUEREpFkOKiIgUiyFFRESKxZAiIiLFYkgREZFiMaSIiEixGFJERKRYDCkiIlIshhQRESkWQ4qIiBSLIUVERIrFkCIiIsViSBERkWIxpIiISLEYUkREpFgMKSIiUiyGFBERKRZDioiIFIshRUREisWQIiIixWJIERGRYjGkiIhIsRhSRESkWAwpIiJSLIYUEREpFkOKiIgUiyFFRESKxZAiIiLFYkgREZFiMaSIiEixGFJERKRYDCkiIlIshhQRESkWQ4qIiBSLIUVERIrFkCIiIsViSBERkWI1OKQOHDiASZMmwd/fHyqVClu3bq2zXAiBZcuWwc/PD05OToiMjMTFixfrrFNcXIyZM2dCo9HA3d0dc+fORXl5eaMqQkRErU+DQ6qiogK9e/fGxx9/fM/lK1euxEcffYRPP/0UycnJaNOmDaKiolBVVWVeZ+bMmTh37hzi4+MRFxeHAwcO4KWXXrK8FkRE1DqJRgAgYmNjzc+NRqPQarVi1apV5tdKSkqEWq0WGzZsEEIIkZaWJgCI48ePm9fZuXOnUKlU4urVq/X6uTqdTgAQOp2uMcUnIiIrqe/3eJP2SWVnZyM/Px+RkZHm19zc3BAeHo6kpCQAQFJSEtzd3TFgwADzOpGRkbCxsUFycvI9t6vX61FaWlrnQURErV+ThlR+fj4AwNfXt87rvr6+5mX5+fnw8fGps9zOzg6enp7mdX5qxYoVcHNzMz8CAgKasthERKRQLWJ035IlS6DT6cyP3NxcaxeJiIiaQZOGlFarBQAUFBTUeb2goMC8TKvVorCwsM7ympoaFBcXm9f5KbVaDY1GU+dBREStX5OGVHBwMLRaLRISEsyvlZaWIjk5GREREQCAiIgIlJSUICUlxbzO3r17YTQaER4e3pTFISKiFs6uoW8oLy9HZmam+Xl2djZOnToFT09PBAYGYsGCBXj33XfRpUsXBAcH46233oK/vz+mTJkCAOjevTvGjh2LefPm4dNPP4XBYMD8+fPx7LPPwt/fv8kqRkRErUBDhw3u27dPAPjZY9asWUIIOQz9rbfeEr6+vkKtVotRo0aJjIyMOtsoKioSM2bMEC4uLkKj0Yg5c+aIsrKyJh+6SEREylTf73GVEEJYMSMtUlpaCjc3N+h0OvZPERG1QPX9Hm8Ro/uIiOjRxJAiIiLFYkgREZFiMaSIiEixGFJERKRYDCkiIlIshhQRESkWQ4qIiBSLIUVERIrFkCIiIsViSBERkWIxpIiISLEYUkREpFgMKSIiUiyGFBERKRZDioiIFIshRUREisWQIiIixWJIEVGj5ebmoqioCHq93tpFoVbGztoFIKKWSQiB6upqbNiwAceOHQMAeHt747HHHsO4cePg4eEBGxseB1PjMKSIqMEMBgOKi4tx5swZfPHFF0hJSYFer4enpydCQ0NhY2OD7t27w9fXF25ubnB2doZKpbJ2sakFUgkhhLUL0VClpaVwc3ODTqeDRqOxdnGIHim1tbUoKChAfHw8Fi9ejBs3bsDe3h62trbm5bW1tRg5ciQmT56MQYMGoWvXrnBwcIC9vT1UKhUDi+r9Pc6QIqIGSUtLw4cffogNGzagoqICAPDSSy+hT58+qKmpwbFjx/Ddd9+hqqoKAODu7o4uXbrgV7/6FX7zm9/A3d0ddnZsxHnUMaSIqEnV1tZi8+bN+PDDD5Geno7Kykp4eXlh9erVGDp0KFxdXSGEQFVVFfLy8rB161bs2rULmZmZqKiogLOzM3x9fTFixAiMHDkSERERCAgIsHa1yErq+z3OwxkieiCDwYCCggJs2bIFGzduxLlz5+Dn54eBAwfiySefxPDhw9G2bVtzc5+rqytcXV3h6OiIiIgIZGZmIjU1FYcPH0ZmZiYqKyuRmpqKH374AX369MHQoUPRq1cvODk5sRmQfuaRD6naWsBolP9XqQBbW/lvfQkhH7W1d7ZhYyMfTcVgkD/DxNZWbr++5TTV0bQNOzv5Xn4f0C8pKytDVlYWjhw5gvXr1+PHH39Ejx49MHz4cIwePRpjx46Fra1tnXBRqVRwcnJCjx490L17dxQWFiI0NBQBAQFITk5GTk4OLly4gNTUVJw9exZXr15FTk4OgoKC4OfnB61Wa+67Inrkm/tu3gSKi+X/7eyAgADA3r7+76+pASorgevX5XMnJ8DDA3BxaVSx6rh8GaiqkkGjUslt+/nVP1CLioDSUllOGxugXTugTZumDVJqXYQQqKysxJkzZ7Bp0yZ8++23uHHjBgICAvCHP/wBkZGR6NSpU4O3eenSJcTFxWHfvn348ccfodPpUFFRAa1Wi8GDB5ubArVaLZydnWFnZ8dh7K1Uvb/HRQuk0+kEAKHT6Rq9rbfeMp0LCeHnJ0RWVsPen58vxPr1d7YxdKgQ33/f6GLV8fjjQrRpI7dvaytESIgQ164JUVtbv/d/+KEQYWHy/c7OQvzwgxAVFU1bRmodjEaj+fHVV1+Jnj17CltbW+Hk5CQGDhwo0tPTRVVVVaN/TmVlpbhw4YJYtWqVCA4OFnZ2dgKAsLW1FZ6enmLevHkiKSlJlJSU1CmT0WhsglqSEtT3e5yHKC1MbS1w/jywbh2QnW3t0lBrU1NTg5ycHLz44otYuHAh0tPT0b17dyxYsABxcXHo3LkzHBwcGv1z1Go1OnbsiOjoaJw4cQLbt2/HggULEBoaipKSEvzrX/9CVFQUxowZg5iYGMTHx6PW1KZOj5RHvk+qJaqtBf7v/4AuXQBvb8DNzdolotbg+vXrOHz4MDZt2oSDBw/CYDBgxowZGDNmDCIiIuDl5dVk1zipVCrY2trC0dERjo6OCA8PR3BwMCZNmoT09HTs3r0biYmJSE9Px/Xr13HkyBGsW7cOo0ePxrBhw+Dv7w9HR8cmqDUpHUOqhbGxkX1TFy4Ahw4B/v5ARIS1S0UtXWZmJhISEhAfH4/ExESoVCo8/fTTmDZtGvr27Yv27ds/lJ9rCjwPDw94eHigQ4cO6Nq1K9q2bYvOnTsjMzMTly9fxoULF5CRkYGbN2/i4sWL6Nq1Kzp16oSuXbuaw5NaJ4ZUC2JrCzg6yn/LyoCEBMDXFwgNlQMhiBpCCGGePeLf//431q1bh4yMDLi6umLw4MFYsmQJAgMDm/WMRa1Wo127dnjqqacwfvx4pKSk4ODBg+azqqNHj+LAgQMICgpCREQEJk+ejNDQULi5uaFNmzZQq9UMrFaGIdWCODgAWi0wYACwbRuQmgrs2QP06weMHi3X4f5J9SGEgMFgQGFhIRYtWoQffvgBer0enTt3xqRJk/Duu+/C0dHRql/4bdq0wbBhwzB06FAsWLAASUlJWLduHRISEpCZmYkLFy5g3bp16NOnD5577jlERkaie/fudYavM7BaPoZUC+PsDLz9NpCYCBQUAMeOAf/v/wFDhshlRPWRl5eH+Ph4rFy5EhcvXoSTkxNee+01TJ8+HeHh4YqbtsjZ2RlPPPEEhg0bhry8PCQnJ+M///kPtm7dilOnTiE1NRUffPABOnfujHHjxmHu3Lnw9fVVXD2o4fgXbGFsbIDgYGDqVCAuDrh6VfZPffkl8PLL8myL6EESExPx7bffYu/evcjNzUXnzp3xxz/+EQMHDkT79u1h35ALBZuB6WzI1tYWtra2aN++Pdzd3REWFoZZs2Zhx44d2L17N3Jzc3HmzBkUXruGiNxcePr7wy4iAujVS14cSC0SQ6qFUakAtVqGVEYGUFgI3LoFfPcdMGmS3BcZVPRTRqMRVVVV2LFjB7Zt24akpCQUFxcjJCQEL7zwAkaNGoW2bdtaPLy8rKwMx44dg16vR58+feDv79/ENbjDwcEBnp6ecHd3R/v27eHi4oJu3bohIyMDGRkZuJKeDv8TJ2ArBHDmDPDYY0D37rJdvGtXuYOwGbDFYEi1UEOGyH0uO1s+kpLkY/RoOSyd+yCZGAwGlJSU4Ny5c/jss8+QlJSENm3aoE+fPuamMUsHHAghcOvWLZw4cQIbNmxAdXU1ampq8OSTTz6EmtRlY2MDR0dHDBgwAP379zefSR3fvx+PnT8Pu7Q02WmbmAh07AhMnAg88YTs2PX0lNdu8IhO8RhSdxFCTj90+3b931NZCVRXP7wy3Y+zMzB+vDyL+uorOT3T6tVA585yWiY2xRMgZy4vLCzEwYMH8fbbb+PixYtwd3fHU089hWeeeQaPP/64RdsVQphnPN+3bx/eeOMN5OXlwdPTE0KIZgmpu6lUKgQGBiIwMBATx48HcnKATZuAnTuBc+fkGdWPPwIffwxERsqjuSeekNdw2Nk1bDJMalb8KrtLfj7Qt2/DJ5g1TVDb3IYPl3Py/fgjcPIkcPy4bPYDgIEDrVMmUpbdu3dj7dq1iIuLQ1VVFbp164bVq1ejf//+8PT0tHi7RqMRly5dwpo1a7Bq1SrU1NSgd+/emDZtGqKjo5uwBhZQqYAOHYDFi4EFC+Tkl/HxwGefAWlpwMaNckdxdwdGjQLmzAEGDeJV8QrFkPoJa5wVWUqlksPRFy6U+1ltrdz/XF1lM7y7u7VLSNZgNBpx+/ZtvPvuu4iLi0NOTg6cnZ0xYcIELF++HIGBgY26LcbNmzfx/fff4/vvv0dKSgqEEJg1axaeeeYZhIWFwd3aHzxTvVQq2ZwXHAz8+teyue/0aWD3buDIERlYO3bIq+J79JBBNXQo8Pjj8oJEnlkpAkPqLo6OsgWgIRfG6vVy8EJy8sMr1/2oVICXlwyqYcOAAwfk2eDRo3Kfmz69+ctE1lVRUYGcnBysX78eW7duxbVr19CtWzc88cQTGD9+PLp06QI7OzuL+58yMjKwfv167N+/HxkZGbCzs8PChQsxceJE9OzZU3mzP5iCysFBnimZbiEwbBiQni6bHw4dAlJSgLw84MQJ4N//ltO4DB4MtG0rvxjIahhSd3F2BqZMkbM41FdJibyo1hohBcj9p107YNo0uZ+Vlckm+N27ZUsGWzAeHTdv3kRGRgYSExPxr3/9C7du3UJoaCjGjx+PcePGoV+/fhZt19T3dPnyZcTGxmLDhg0oLi6Gt7c3IiIiMHv2bHTp0qVJJp59qFQqOarI21tO01JYKCfA9PCQo4+uX5dnWIcOAVlZd5a3by+DzdQ8qqQQfgQwpO6iVss+1Y4d6/+eggLrDxBycQGefx744gt5zVROjhzQdOqUPGCk1kv893ZwFRUVOHToEDZt2oTY2FgIIdCnTx/z9U/e3t4Wbd80dP3y5cv48ssv8dFHH8HOzg7du3fH2LFjsXDhQnh7eyvr7Kk+HB2BwEDgueeAX/1KDo3dswfYvx84e1b2Ye3cKYesDxkimwpNzYAODg2/O2p9me6gWlt7546qwJ07qZp+bn1/ttFY946nDX3/3WW6uyz/vQtzc2BItQI2NoBGA7zxBrB8uWzFyMsDli2T8/tR61ZZWYk//vGP2LJlC/Ly8qDRaDB16lSsXr0arq6ujbppYFFREb777jt89tlnOHXqFGxtbTFnzhzMnDkTERERTTYrutWYmgOHD5ePigrZFPHVV0BsrLwYMS1NPu/cGZg1S16Q2KWLPKptatXVwKVL8ucWF8vhxiqV7GAOCLgzUWd9Q6KkRB5Jl5XJ5507y07rhlywrdfL+wMZDPJ5u3bNenE0Q6oVefJJeQZVWQlcuSKb17duBXS6e69vMMhRgVu3ApMnA/37c+h6S1JWVob09HQsXrwYP/74IwwGA4YPH47nn38e06dPh0ajsThEhBBISkrCqlWrkJycjJKSErRr1w5/+9vfMHjw4Ca9bYfV3V0HZ2c5xHflSjkiKSkJ2LcP+M9/gMxM4L33gE8+kRcHjxwJzJghmwMtPRAQQu6wu3bJM7kjR+Q0MjU1dYcNq1QymDQaOYPGiBGyjT8o6MFnRYcOyVGNR4/K5198Ic8M27atfxmvXZMDTwoK5PPXX5cjJ5sJv5JaEScn+bnV6YDcXHkQtnatHI17r2HyFy7IM61Nm+S+0bevHLoeGioPFK3djEn3d/78eRw4cABbtmzByZMn4erqiqeeegojR45Ev379LB5hJ4RAeXk5duzYgc8//xxnzpyBra0tBg8ejLlz52Lo0KHw9PRsvXPi2djcGWjh6ChDITRUHsUdOSJHJ126JCfNvHJFHhWGhclBFj16yMCqb3DX1Mgmj7/8RbbNX7kC3Lghd9y7qVR3mtp0OnlWlJUlA2juXNn5fL/BHdXV8j1FRXeeN/SamdpaeUGmaRuVlQ17fyO10k/ao8e0X/TtK8/MU1JkCCUny8/8rVs/f4/RKA/O1Gq5XlaWPFg8exYICZHD2B97TB5cNmMTNN2H6dYap06dwp49e5CQkIDDhw/D398fEydOxFNPPYWePXtaHFDV1dXIy8vD8ePHsX79ehw8eBCBgYEICwvDqFGjMH78ePPZ2SNBrZYX+2q1QM+esg8rMFAOY8/KkgETH39n2pdeveQZVlCQ3HHs7e9/hlVdLQdmbNgAbNkiwwmQTXGhobI5rU0b2bRhmmWgoED+3Js3ZRNkdrY8MvX0lAHZSkdJMaSamemAqKZGHqAA8nNsaiJu7P7v7S3PhjIz5QFfUZFs9tPrf75uYKBshq+ulk1+V6/Klo19++R2xo8Hnn1WrqfRyLAyNcM/Kt9TSmE0GmEwGJCXl4cvvvgCu3fvxvXr1+Hj44PJkydj6dKlcHd3h62FRxNVVVXIy8vDf/7zH3z11Vc4efIkfH198eSTT2LKlCkYMmRIo/q2WjQbG3mmEhYmH1euyDOfhAQ5uKKwUDZHxMXJgBo7FnjmGTlM2NVV7jR39wEJIfubjhwBVqyQZ0YODnL9nj3le4cNk88dHeXR5K1b8sgzNlYeUV66BJSXA+vXy/6quXNlSCpscuCmwJBqRncHVHa2PCAyGuVnzNS8JkTjA6B/f/lZ/eEHOdKvtPTe67m5AeHhMtQWL5b72+bNsvn60iXZ9P7pp/I6rDFjgKgo2Zx9d+sDw+rhE0Lg9u3byMjIwJw5c5Ceng6VSoWwsDC88sormDlzpsXbNTl+/DhWrVqFxMREVFRUoG3btvjf//1fPP744/Dw8GiqqrQOpjOqCROApUuBb76RM1ikp8vwOnUK+Mc/gHHj5CCLoUPr9h0ZDDJo3nrrzoCGPn1kv8+LL8qzo7vZ2gI+PnJ748bJvqsNG4A1a+SO+L//K5sZNRo5MKKVYUg1o5wcGQQffywPxkyjOu3s5KUaf/gD8NRT8vPYGHZ28iL7P/0J+M1v7pyxPYiDg9znoqJkq0JqKvD99zK0Tp6ULRyffAJ06iQHOE2bJs+2WuGBm+KcOnUK27Ztw8cff4zi4mIEBQUhOjoaEydORMeGXC9xD+Xl5fj73/+Ojz76CEVFRfD29kZkZCRWrlyJgIAAxd22Q1FsbOQAhPnzgXnz5Ii8AwfkTpOcLM964uJkSK1dK5sNVSrZlxQXB1y8KLfTpQvwzjuyWaM+Fw6PGHHnmq2vvpI7+Pr1csdvxgENzYUh1UySkuTn8rvvZNPb9OkyjKqr5dRie/YAH34oD5pGjZKfW0upVPKgavRoOdPLyZMPnjTXdIBnby8f/v7yLOuxx2QYHTokz67S0uTo3A8/lAEWGirPwoYObVh/Mf0y051zN2/ejO3btyMpKQmlpaUYNWoUXn31VfTu3Rs+Pj4Wh8jt27eRmpqKr7/+GnFxcSguLsbQoUMxbtw4jBs3zhxQj0z/kyVM1xs5OMgdp3t3edHviBFyR9m7V14g3K3bnVAB5KCLpKQ7zRFz58p16jsVk52dPGOaM0c2M1ZUyL6qM2dkE01w8EOprrU0OKQOHDiAVatWISUlBdevX0dsbCymTJliXj579mysW7euznuioqKwa9cu8/Pi4mK8+uqr2L59O2xsbDB9+nT87W9/g4uLi+U1UTCjUQ5GOHFChsVvfiObzTw8ZNNfbq4Mp9hYOVNE+/aNCylAfo59fOSZWU6OHJBzV+vOA5kGN7m7y/5bPz/ZVH7+vNz3UlPvDG66cEHuG337ytAKCJD9vfxus1xNTQ1u3bqFbdu2YevWrTh9+jSqqqowcuRIzJo1C0OHDoWrq6vF/U/Xrl1DUlIS9u7di127duHGjRuYMGECxo8fj8GDB6NHjx4Mp4ZSqeQHv00beZRnmqWiqEgGlKkzV6eTgZKbe+c9I0bIM7KG9Pm5uMgLjUND5dROlZWyUzk9nSFVUVGB3r174ze/+Q2mTZt2z3XGjh2LNWvWmJ+rf3LR28yZM3H9+nXEx8fDYDBgzpw5eOmll7B+/fqGFqfRPD3lDBMqlfycNPTA1NZWfl5MrS7+/j+f+08IGUZubvKMfuFCeaZj+kyGhsplcXHyAGvYMBlspuXt2snPthAyBOrLxkaese3dK8toMMjm7jZt6rc/qFTyfWFh8t5VJSVyf9i2TbZmFBYChw/L7ffuLW/EOHCgbK53c5Mh/LAuzG+tKisrcePGDZw4cQJ/+ctfcPXqVXh4eGDw4MH49a9/jV/96lcWX/ckhMCNGzewf/9+fPPNN0hKSoIQAgMGDMDvfvc79OvXD15eXg+hVo8YlUoeIUZG/nzZ1atygs3SUrlz+PnJI9KGTBgKyB3Y2Vke7Z4+LXfuoiJ5BDluXNPUQyEaHFKm5oAHUavV0Gq191x2/vx57Nq1C8ePH8eAAQMAAH//+98xfvx4/OUvf3mod/S8l/Hj5WdEpZJn2w3dR11d5UCFjz6Sz7285HVJd7O1lbd2nzNHnkn9dISws7M8mFKr5Rf/zZvywMj0uV28WPavCiHXrS+VSh7QffCB/PxWVcmyhIRYFsZeXnLg0tix8qwqIUGOwN2/Xw48OnZMDkjq21fuJ888Iw8CLJmJ5VFiGsBgNBpx4cIF/PDDD/jggw9QWVmJjh07Yu7cuXj66act7n+6+95PX375JVavXm3ufxoyZAg+/vhj+Pj4WHxmRg1w/vyd60Hs7eXOYm9v2c5hZydHNZl25uJieSb1IKbmlPo2qzR03YfgofRJ7d+/Hz4+PvDw8MDIkSPx7rvvmo/QkpKS4O7ubg4oAIiMjISNjQ2Sk5MxderUn21Pr9dDf9cY6tL7DVezQJcudQfENPSz4uAgD4buzuT7bUOtvv9MKqbrlcrLf/6Z6Nu37vOGlrFr17rbbIqw6NZN9lnNnSsDcNMm4PPPZRPg7t2yj+2dd+SZ3NNPy7PFhkzc+6jR6/X4/PPP8d133+HQoUMAgKeffhpvvvkmOnXq9LPWiIa4ffs2Tp8+jeXLlyMhIQFGoxGRkZF45pln8Otf/1r5E8O2Jrm5d0b02dnJHcnSgwNbW9kOb7qwuqxM7oAPUlgo2//Ly+v/c/Ly6jf66iFp8pAaO3Yspk2bhuDgYGRlZWHp0qUYN24ckpKSYGtri/z8fPj8ZPianZ0dPD09kZ+ff89trlixAsuXL2/qogJo/BH+3beuqc9691JTI2ceqaiQTWSennVHoTb28pSHcRZjmqXF0VGGz/PPy7On06dlk+WBA3dGCCYkyBG4ffrIEYQjRrAZ0MR07dMf//hHHDt2DIWFhfDz88Pzzz+P3/72t9BqtRbf2h0A0tLSEBsbi9jYWGRkZECj0SAmJgYjR45Et27d4ODgwP6n5lRWdmcOPNOkm5b+/lUq2a5uen919Z0AvJ/335fNMQ2ZMaS6Wp6lWUmTh9Szzz5r/n+vXr0QGhqKTp06Yf/+/Rg1apRF21yyZAliYmLMz0tLSxHQkM4Zhbt9W/brGAzy7D0goPHB1FxUKtna0LatHJLu4SH758LC7tzC5OxZOcIwL0/2GR89KgOre3fZ3/ao3q7n5s2bOHv2LDZu3Ih9+/ahpqYGYWFhGD16NKKiohAUFNSo/ifTtEmHDh1CdnY2/P39MXv2bEycOBEdOnSAq6vrQ6gVPVBV1Z2zEpVKHo02JqTufn9t7c+nVPqpa9cs+1lW9NCHoHfs2BHe3t7IzMzEqFGjoNVqUVhYWGedmpoaFBcX37cfS61WN6q5Q8n0evm5iYuTBzeDB8trkVoilUo2e2q1cqDFsGFyoJFpurP8fNmHdfy4HEAyaJDsH2vXTvad1XdAR0tnNBqRn5+PkydPYseOHVi3bh1cXFwQFhaGSZMmYdKkSWjfvr1F266trcXt27dx8eJFfPvtt4iNjYVer0dwcDAiIyMxb948eHh4sP/Jmpqy7f3u9999a4/7CQqSo6Eacial18spbGpqLCtjIz30kMrLy0NRURH8/PwAABERESgpKUFKSgr69+8PANi7dy+MRiPCw8MfdnEUxWiUl1EcOiRnhzBdxB4SYu2SNZ6dnazPyy/LpsAzZ2Q/1aZNMrC+/15OWebrK++G/NvfyunHnJ0F7OxkWLW2ZighBIxGI3Q6Hb7//nts2bIFiYmJUKvViIyMxO9//3v07NkTjhaeWtbW1qKsrAxpaWl46623kJiYCAcHBzz++ON4+umnMWfOHNjY2LS632uL4uR0Z6CDaQZ0SwcmCCH7CEzvt7f/5VGCL7wgO4gbMr/jtWtATMydCWabWYNDqry8HJmZmebn2dnZOHXqFDw9PeHp6Ynly5dj+vTp0Gq1yMrKwh/+8Ad07twZUVFRAGC+Wdq8efPw6aefwmAwYP78+Xj22WebfWSfteXkAP/3f8Df/iY/u198IYdyt7aTRmdnedY0aJC859XOnfIC+aNH5e/g66/lY/BgYMqUcowcaUSfPi6t7mjfaDQiKysLL730Ek6dOoXy8nJotVq8/vrrePnllxs9gCEtLQ0bN27EF198gcLCQnh7e+Odd95BVFQUOv339JwBZWUazZ2QMhplX09jQqq4+M6s5g4OcvsPEhIir75vyLQ2mZlWvSVCg0PqxIkTeOKJJ8zPTX1Fs2bNwieffIIzZ85g3bp1KCkpgb+/P8aMGYM///nPdZrrvvnmG8yfPx+jRo0yX8z7kWkM9yNACDm918qV8izKzU3epiY8vP4XnbckpvqYpoCKjAQiImTzX1qabALcvBn48Ufg1q0d2Lp1F+zsLmHKlCl49tln4eXl1eJHoBUWFuLAgQN48803kZOTA3d3d0ycOBEvvvgi+vXrZ/EABtPw9fXr1+OLL77AqVOnYDAYEBERgb/+9a/o2rUrXF1dGU5KERQkr1sBZPNZerrlI+dqa+UOZGqGc3Or/4W8Lejz0OCQGjFiRJ2JKX9q9+7dv7gNT09Pq1y4qwRVVfIWNJ99JpvAOneWc1AOH964PtSWwFQ3Z2f5cHWVAy46dpTXJCYnV+PcuQvIyTmOkpIrKCkpwbFjx9C3b1/07t0bISEhaNeMdwRtCgaDASdOnMDevXuxY8cOXLp0CT169MDUqVPxxBNPIDQ0FJpfOvp9wLaLiorw9ddfY+vWrUhLS4NGo0FERARmzZqFXr16wcnJqdWdkbZonTvfuaVGTY2cwsV0j6eGdsjW1MgvkbtDqqV2aD8A5+5rJrW1dy4I37BBztLfq5e8MPbJJ+teZ/WocHSU9TbdoaBTJyA5OQhnz/ZHVpY7cnJysHXrVqSnpyMtLQ19+/ZFjx490LlzZ2i1Wjg6Oir2DMF088Djx49j586d2LdvH86cOYOePXti+vTpmDx5Mh577DGLBwSVlpYiJycHR44cwTfffIMLFy6gS5cuGDRoEKKiojB69GjY2toq9vfzyPL3l01tbdrIYb15efLh5NSwK/WNRtmfdeLEnSHtXl5yyGwrw5BqBkajDKhDh+Ss+v/5jxyCPXu2bPby8JBTDpnY29+Zs/JRoFLJ/XPYMAcMHfo8btwYi5MnT2LLli3YtWsXsrOzcf78eXz77bfo2rUrXnjhBURGRsLPzw9OTk5wcnJS1IAAo9GIyspKpKam4t1338WZM2dQWVlpnr38ueeeg3NDvpD+y9SCUVlZibS0NGzfvh2ff/45dDodOnTogHnz5mHMmDF47LHHmrpK1FQ8POTRWECAbOorL5fDX00XR9b3M2waFpyUJEPKwUEGYJ8+D7X41sCQagZlZXJqok2b5EGTnZ2crWHvXjml0E8NGCCHb3fr1uxFtTqVSgUfHx9ERUVhzJgxuHz5Mv79739j9+7dOHjwIE6fPo1FixbB09MTAwcORFRUFJ566in4+/ubv8StFVamn3/r1i3Ex8fjd7/7HXQ6HTw8PDBu3DgsXboUffv2bVTfkxACGzduxGeffYYTJ07A1tYWgwYNwueff47AwECLRwZSMxo6VM48YZrC6H/+R34h+Pr+8hRJpq6WixeB1avl6D5ADogIC5PXcrQ2ogXS6XQCgNDpdNYuSr1kZQkxdaoQNjamCxmEcHAQQq2+9+O554TYu9fapbY+o9EoampqRGVlpbh165bIyMgQ//jHP0SvXr2Es7OzsLe3F23atBFarVbMnj1b7N69W+Tn51utvHq9XuzcuVPMmjVLtGnTRqhUKjFixAjx9ddfi/z8fGEwGCzednV1tcjNzRVPP/208PLyEnZ2dqJTp05i0aJF4ubNm8JgMAij0diEtaGHpqpKiG3bhAgJkV8GNjZCDB4sxD/+IUR5+YPfazQKkZAgxG9/K79ETF8mq1YJkZ398/U3bxbi8cfvfPF8+60QDd1HLlwQws/vzjaWL2/Y+++jvt/jPJNqBr6+wB//KO+LVh9abcNmO2+tVCoVbG1tYWtrC7VaDScnJ0yePBl9+vRBamoqTpw4gRMnTiAtLQ27du1CamoqOnfujJCQEAwaNAhDhw5tlml/xH9nF1+zZg327t2L1NRU1NbW4sUXX8SMGTPQvXt3eHl5wa4hF1DeJT8/H8ePH8fatWtx8OBB1NTUYPr06YiKisKQIUPg4eEBlUqlmOZO+gUODvJak/nz78wenZoqr0E5fVpeLDlggOxjcnCQ/QVlZXKQRUKC7NA2DbgA5LVPQ4e22o5thlQzaNNGzpROllOpVFCr1Wjfvj3at2+PoKAgcyCdPn0a586dw4ULF5CTk4P09HRcuHABmZmZ6NGjh3nWk4fRFFZdXY3c3FzExsZiy5YtuHz5MhwdHfGrX/0Kzz77LMLDwxvV/3ThwgUcOnQIe/bswd69e+Hg4IApU6bgySefxMCBAy2emYKsSKWSR64jRwLPPitvJFdSIoeTFxTIWdJTUuQFt2q1PH8pK5Nzih07Ji8uLCuTI48ef1zO4typU6udX4whRS2SKayGDx+OGzduIDY2Ftu2bUNOTg6uXr2KjIwM/Pvf/8b48eMRGRmJnj17ws/PD+7u7nB2dm70WYcQArdv38a1a9ewc+dOvPfeezAYDPD398fgwYMRExODkJAQi/ufamtrce3aNWzfvh3ff/89Tp48CY1Gg+HDh2PRokXo1KmTReFHCuHoKK9peuMNGUznzgE3bsjHd9/dWc80E7NpmLlpvj4fHzmly/z5csbmVhpQANgnRa2HXq8Xhw8fFu+8844IDQ0VNjY2QqVSCRsbGxEYGChmzJghdu/eLfR6vaitrRVGo9Gifhyj0SiqqqrEnj17xMyZMwUAoVKpxBNPPCHWrVsnSktLLa6D0WgU1dXV4urVq2LixInC1dVVqFQqERQUJF5//XWh1+vZ99SaGI1CVFbKvqIXXhAiIED2UdnYCKFS3XmYXtNohHjiCSE++ECI3Fz5/gf57jshhg278/5Nm4QoKGhYGS9cEKJ9eyFsbeU2/vQny+t7l/p+j6uEsPIdrSxQWloKNzc36HQ6iy+EpNZH/Hf279raWlRXV+P06dNYs2YN9uzZg7y8PAghYGtri969e+PJJ5/EyJEj0a9fvwY1AwohcPXqVXzwwQfYvXs3srOzYWdnh2XLlmHmzJnw9/dv1PVJ169fx+HDh7F06VJcunQJLi4umDFjBp577jkMGjTI3K/F/qdWwvT1azTemcU8O1veHLGoSA41V6nk0PXAQHlxpYuLHAVYn/vdlJbKqZNMowADAhp+qw69XjYxmq7H8vGRV+E3Un2/xxlS1CoJIVBRUYEbN24gLy8PaWlpSEpKwo4dO3D79m24u7vD29sbHTp0wIgRI/DUU0/Bx8cH9g+4OE2v1yM5ORnvvfceUlNTcfv2bbRv3x4xMTEYM2YM2rZta9H0TaZdcN++fdi+fTt27tyJy5cvIzg4GDExMYiIiECHDh3g4uJi8e+DWgijUYZCZaVs4jPNy2dnJ/unnJ1Nsy/Xb3u1tfJh2o69fcPebyqTwXAnUO3sGhZy91Hf73H2SVGrpFKp4OLiAhcXF/j5+dUZaJGSkoLU1FTk5eUhJycH+fn5yM7ORvfu3dG1a1d07tzZPCBBpVKhpqYGWVlZ2L17N44cOWK+u/SIESMwevRojB49Gv7+/had3Zgu/DWFU3JyMq5du4Z+/frh+eefx+jRo82za9AjwMZG9jndfdfTxrC1tfzOv3eXyYqzXvNMih4ZQgjU1NTgyJEj2LNnD06fPo3MzEwUFRWhqKgI3bp1w6BBgzBo0CD069cPnp6ecHFxwdWrVxEfH49PP/0UWVlZaNeuHUaOHIlJkyZh3LhxFp/hGAwGlJaW4vTp0/jzn/+MU6dOwc7ODl27dsXTTz+Nl19+Gfb29mzao1aJzX1Ev6CgoACpqanYsmULNm3ahPLychgMBjg6OiIwMBBjx47FgAED8K9//QuJiYnQ6/VwcHDAG2+8gRdeeAEdO3a0ePSe0WhEQUEBjh49ipiYGOTl5UGj0WDixImYM2dOnTsNELVGDCmiX2D66JsC47vvvsOOHTtw/PhxFBcXmwNICAGVSgVPT0+sWrUKM2bMME8Ma2lI7dixAxs3bsSmTZtQXV2Nvn37Yvny5YiIiICXlxfPnqjVY0gR1ZNpRGBZWRnKyspw5coVJCQk4K9//SvKy8vh6uqKkSNHYtGiRejVqxdcXV1hY8F97sV/Z0Z///33sWvXLmRmZkIIgalTp+L1119HQEAAnJ2deWsNeiRw4ARRPalUKtjZ2cHDwwPu7u7w9PSEh4cHAgICcO3aNXh6eqJbt27o06ePxf1Pt2/fRm5uLr766ivExcXh+vXrCAwMxOjRozF9+nR07tyZ/U9E98CQIrqLaVRgz5490bNnT+Tn58PV1dXiWSqEECgqKsLFixeRmJiIdevWQa/Xo1u3bhg5ciSmTp2KAQMGPISaELUODCmiB9BaOGmn6cLiiooKHD58GFu2bMGGDRtga2uLsLAwvPbaaxg+fDi8vb2buMRErQtDiughKS8vx9KlS7Fz505cunQJTk5OePHFF7Fs2TJ4enqyaY+oHhhSRE2soqICaWlpePXVV3H+/HnU1NQgPDwcr7zyCqZOnQoXFxfeWoOonhhSRE0oKysLBw8exDfffIOzZ8/Cy8sL48ePR1RUFAYOHAhXV1eGE1EDMKSIGkkIAYPBgDNnzuDgwYPYs2cPEhMTERwcjAkTJmDixIno378/3NzcrF1UohaHIUXUCEajEQaDAdnZ2fjyyy+RkJCAnJwceHt7Y9q0aXjttdfQtm1bXvtEZCGGFJGFhBDQ6/VITU3FrFmzkJ2dDQAICQnB0qVLMW3aNDbtETUSQ4rIQgaDAenp6ZgyZQoKCgrQpUsXzJw5EzNmzEBQUJC1i0fUKjCkiCxkZ2cHrVaLF154AZmZmZg5cyb69u0LPz+/B96XiojqjyFFZCEbGxu4u7tj6tSpKCgowODBg+Hp6WnRvH5EdG8MKaJGcHJyQnh4uLWLQdRq8ZCPiIgUiyFFRESKxZAiIiLFYkgREZFiMaSIiEixGFJERKRYDCkiIlIshhQRESkWQ4qIiBSLIUVERIrFkCIiIsViSBERkWIxpIiISLEYUkREpFgMKSIiUiyGFBERKRZDioiIFIshRUREisWQIiIixWJIERGRYjGkiIhIsRhSRESkWA0KqRUrViAsLAyurq7w8fHBlClTkJGRUWedqqoqREdHw8vLCy4uLpg+fToKCgrqrHPlyhVMmDABzs7O8PHxweLFi1FTU9P42hARUavSoJBKTExEdHQ0jh49ivj4eBgMBowZMwYVFRXmdRYuXIjt27dj8+bNSExMxLVr1zBt2jTz8traWkyYMAHV1dU4cuQI1q1bh7Vr12LZsmVNVysiImodRCMUFhYKACIxMVEIIURJSYmwt7cXmzdvNq9z/vx5AUAkJSUJIYTYsWOHsLGxEfn5+eZ1PvnkE6HRaIRer6/Xz9XpdAKA0Ol0jSk+ERFZSX2/xxvVJ6XT6QAAnp6eAICUlBQYDAZERkaa1+nWrRsCAwORlJQEAEhKSkKvXr3g6+trXicqKgqlpaU4d+5cY4pDREStjJ2lbzQajViwYAGGDBmCkJAQAEB+fj4cHBzg7u5eZ11fX1/k5+eb17k7oEzLTcvuRa/XQ6/Xm5+XlpZaWmwiImpBLD6Tio6OxtmzZ7Fx48amLM89rVixAm5ubuZHQEDAQ/+ZRERkfRaF1Pz58xEXF4d9+/ahffv25te1Wi2qq6tRUlJSZ/2CggJotVrzOj8d7Wd6blrnp5YsWQKdTmd+5ObmWlJsIiJqYRoUUkIIzJ8/H7Gxsdi7dy+Cg4PrLO/fvz/s7e2RkJBgfi0jIwNXrlxBREQEACAiIgKpqakoLCw0rxMfHw+NRoMePXrc8+eq1WpoNJo6DyIiav0a1CcVHR2N9evXY9u2bXB1dTX3Ibm5ucHJyQlubm6YO3cuYmJi4OnpCY1Gg1dffRUREREYNGgQAGDMmDHo0aMHnn/+eaxcuRL5+fl48803ER0dDbVa3fQ1JCKiFkslhBD1Xlmluufra9aswezZswHIi3kXLVqEDRs2QK/XIyoqCv/85z/rNOXl5OTglVdewf79+9GmTRvMmjUL77//Puzs6peZpaWlcHNzg06n41kVEVELVN/v8QaFlFIwpIiIWrb6fo9z7j4iIlIshhQRESkWQ4qIiBSLIUVERIrFkCIiIsViSBERkWIxpIiISLEYUkREpFgMKSIiUiyGFBERKRZDioiIFIshRUREisWQIiIixWJIERGRYjGkiIhIsRhSRESkWAwpIiJSLIYUEREpFkOKiIgUiyFFRESKxZAiIiLFYkgREZFiMaSIiEixGFJERKRYDCkiIlIshhQRESkWQ4qIiBSLIUVERIrFkCIiIsViSBERkWIxpIiISLEYUkREpFgMKSIiUiyGFBERKRZDioiIFIshRUREisWQIiIixWJIERGRYjGkiIhIsRhSRESkWAwpIiJSLIYUEREpFkOKiIgUiyFFRESKxZAiIiLFYkgREZFiMaSIiEixGFJERKRYDCkiIlIshhQRESkWQ4qIiBSrQSG1YsUKhIWFwdXVFT4+PpgyZQoyMjLqrDNixAioVKo6j5dffrnOOleuXMGECRPg7OwMHx8fLF68GDU1NY2vDRERtSp2DVk5MTER0dHRCAsLQ01NDZYuXYoxY8YgLS0Nbdq0Ma83b948/OlPfzI/d3Z2Nv+/trYWEyZMgFarxZEjR3D9+nW88MILsLe3x3vvvdcEVSIiotZCJYQQlr75xo0b8PHxQWJiIoYNGwZAnkn16dMHq1evvud7du7ciYkTJ+LatWvw9fUFAHz66ad4/fXXcePGDTg4OPzizy0tLYWbmxt0Oh00Go2lxSciIiup7/d4o/qkdDodAMDT07PO69988w28vb0REhKCJUuW4Pbt2+ZlSUlJ6NWrlzmgACAqKgqlpaU4d+7cPX+OXq9HaWlpnQcREbV+DWruu5vRaMSCBQswZMgQhISEmF9/7rnnEBQUBH9/f5w5cwavv/46MjIysGXLFgBAfn5+nYACYH6en59/z5+1YsUKLF++3NKiEhFRC2VxSEVHR+Ps2bM4dOhQnddfeukl8/979eoFPz8/jBo1CllZWejUqZNFP2vJkiWIiYkxPy8tLUVAQIBlBSciohbDoua++fPnIy4uDvv27UP79u0fuG54eDgAIDMzEwCg1WpRUFBQZx3Tc61We89tqNVqaDSaOg8iImr9GhRSQgjMnz8fsbGx2Lt3L4KDg3/xPadOnQIA+Pn5AQAiIiKQmpqKwsJC8zrx8fHQaDTo0aNHQ4pDREStXIOa+6Kjo7F+/Xps27YNrq6u5j4kNzc3ODk5ISsrC+vXr8f48ePh5eWFM2fOYOHChRg2bBhCQ0MBAGPGjEGPHj3w/PPPY+XKlcjPz8ebb76J6OhoqNXqpq8hERG1WA0agq5Sqe75+po1azB79mzk5ubi17/+Nc6ePYuKigoEBARg6tSpePPNN+s00eXk5OCVV17B/v370aZNG8yaNQvvv/8+7Ozql5kcgk5E1LLV93u8UddJWQtDioioZWuW66SIiIgeJoYUEREpFkOKiIgUiyFFRESKxZAiIiLFYkgREZFiMaSIiEixGFJERKRYDCkiIlIshhQRESkWQ4qIiBSLIUVERIrFkCIiIsViSBERkWIxpIiISLEYUkREpFgMKSIiUiyGFBERKRZDioiIFIshRUREisWQIiIixWJIERGRYjGkiIhIsRhSRESkWAwpIiJSLIYUEREpFkOKiIgUiyFFRESKxZAiIiLFYkgREZFiMaSIiEixGFJERKRYDCkiIlIshhQRESkWQ4qIiBSLIUVERIrFkCIiIsViSBERkWIxpIiISLEYUkREpFgMKSIiUiyGFBERKRZDioiIFIshRUREisWQIiIixWJIERGRYjGkiIhIsRhSRESkWAwpIiJSLIYUEREpFkOKiIgUiyFFRESKxZAiIiLFsrN2ASwhhAAAlJaWWrkkRERkCdP3t+n7/H5aZEiVlZUBAAICAqxcEiIiaoyysjK4ubndd7lK/FKMKZDRaERGRgZ69OiB3NxcaDQaaxepWZSWliIgIOCRqjPAej9K9X4U6ww8mvUWQqCsrAz+/v6wsbl/z1OLPJOysbFBu3btAAAajeaR+aOaPIp1BljvR8mjWGfg0av3g86gTDhwgoiIFIshRUREitViQ0qtVuPtt9+GWq22dlGazaNYZ4D1fpTq/SjWGXh0610fLXLgBBERPRpa7JkUERG1fgwpIiJSLIYUEREpFkOKiIgUq0WG1Mcff4wOHTrA0dER4eHhOHbsmLWL1KTeeecdqFSqOo9u3bqZl1dVVSE6OhpeXl5wcXHB9OnTUVBQYMUSN9yBAwcwadIk+Pv7Q6VSYevWrXWWCyGwbNky+Pn5wcnJCZGRkbh48WKddYqLizFz5kxoNBq4u7tj7ty5KC8vb8ZaNNwv1Xv27Nk/+9uPHTu2zjotrd4rVqxAWFgYXF1d4ePjgylTpiAjI6POOvX5TF+5cgUTJkyAs7MzfHx8sHjxYtTU1DRnVRqkPvUeMWLEz/7eL7/8cp11Wlq9m1qLC6lvv/0WMTExePvtt/Hjjz+id+/eiIqKQmFhobWL1qR69uyJ69evmx+HDh0yL1u4cCG2b9+OzZs3IzExEdeuXcO0adOsWNqGq6ioQO/evfHxxx/fc/nKlSvx0Ucf4dNPP0VycjLatGmDqKgoVFVVmdeZOXMmzp07h/j4eMTFxeHAgQN46aWXmqsKFvmlegPA2LFj6/ztN2zYUGd5S6t3YmIioqOjcfToUcTHx8NgMGDMmDGoqKgwr/NLn+na2lpMmDAB1dXVOHLkCNatW4e1a9di2bJl1qhSvdSn3gAwb968On/vlStXmpe1xHo3OdHCDBw4UERHR5uf19bWCn9/f7FixQorlqppvf3226J37973XFZSUiLs7e3F5s2bza+dP39eABBJSUnNVMKmBUDExsaanxuNRqHVasWqVavMr5WUlAi1Wi02bNgghBAiLS1NABDHjx83r7Nz506hUqnE1atXm63sjfHTegshxKxZs8TkyZPv+57WUO/CwkIBQCQmJgoh6veZ3rFjh7CxsRH5+fnmdT755BOh0WiEXq9v3gpY6Kf1FkKI4cOHi9dee+2+72kN9W6sFnUmVV1djZSUFERGRppfs7GxQWRkJJKSkqxYsqZ38eJF+Pv7o2PHjpg5cyauXLkCAEhJSYHBYKjzO+jWrRsCAwNbze8gOzsb+fn5dero5uaG8PBwcx2TkpLg7u6OAQMGmNeJjIyEjY0NkpOTm73MTWn//v3w8fFB165d8corr6CoqMi8rDXUW6fTAQA8PT0B1O8znZSUhF69esHX19e8TlRUFEpLS3Hu3LlmLL3lflpvk2+++Qbe3t4ICQnBkiVLcPv2bfOy1lDvxmpRE8zevHkTtbW1df5gAODr64v09HQrlarphYeHY+3atejatSuuX7+O5cuXY+jQoTh79izy8/Ph4OAAd3f3Ou/x9fVFfn6+dQrcxEz1uNff2bQsPz8fPj4+dZbb2dnB09OzRf8exo4di2nTpiE4OBhZWVlYunQpxo0bh6SkJNja2rb4ehuNRixYsABDhgxBSEgIANTrM52fn3/Pz4NpmdLdq94A8NxzzyEoKAj+/v44c+YMXn/9dWRkZGDLli0AWn69m0KLCqlHxbhx48z/Dw0NRXh4OIKCgrBp0yY4OTlZsWT0sD377LPm//fq1QuhoaHo1KkT9u/fj1GjRlmxZE0jOjoaZ8+erdPH+ii4X73v7kvs1asX/Pz8MGrUKGRlZaFTp07NXUxFalHNfd7e3rC1tf3ZqJ+CggJotVorlerhc3d3x2OPPYbMzExotVpUV1ejpKSkzjqt6XdgqseD/s5arfZng2VqampQXFzcan4PANCxY0d4e3sjMzMTQMuu9/z58xEXF4d9+/ahffv25tfr85nWarX3/DyYlinZ/ep9L+Hh4QBQ5+/dUuvdVFpUSDk4OKB///5ISEgwv2Y0GpGQkICIiAgrluzhKi8vR1ZWFvz8/NC/f3/Y29vX+R1kZGTgypUrreZ3EBwcDK1WW6eOpaWlSE5ONtcxIiICJSUlSElJMa+zd+9eGI1G847eGuTl5aGoqAh+fn4AWma9hRCYP38+YmNjsXfvXgQHB9dZXp/PdEREBFJTU+sEdHx8PDQaDXr06NE8FWmgX6r3vZw6dQoA6vy9W1q9m5y1R2401MaNG4VarRZr164VaWlp4qWXXhLu7u51Rr+0dIsWLRL79+8X2dnZ4vDhwyIyMlJ4e3uLwsJCIYQQL7/8sggMDBR79+4VJ06cEBERESIiIsLKpW6YsrIycfLkSXHy5EkBQPzP//yPOHnypMjJyRFCCPH+++8Ld3d3sW3bNnHmzBkxefJkERwcLCorK83bGDt2rOjbt69ITk4Whw4dEl26dBEzZsywVpXq5UH1LisrE7///e9FUlKSyM7OFnv27BH9+vUTXbp0EVVVVeZttLR6v/LKK8LNzU3s379fXL9+3fy4ffu2eZ1f+kzX1NSIkJAQMWbMGHHq1Cmxa9cu0bZtW7FkyRJrVKlefqnemZmZ4k9/+pM4ceKEyM7OFtu2bRMdO3YUw4YNM2+jJda7qbW4kBJCiL///e8iMDBQODg4iIEDB4qjR49au0hN6plnnhF+fn7CwcFBtGvXTjzzzDMiMzPTvLyyslL87ne/Ex4eHsLZ2VlMnTpVXL9+3Yolbrh9+/YJAD97zJo1Swghh6G/9dZbwtfXV6jVajFq1CiRkZFRZxtFRUVixowZwsXFRWg0GjFnzhxRVlZmhdrU34Pqffv2bTFmzBjRtm1bYW9vL4KCgsS8efN+dgDW0up9r/oCEGvWrDGvU5/P9OXLl8W4ceOEk5OT8Pb2FosWLRIGg6GZa1N/v1TvK1euiGHDhglPT0+hVqtF586dxeLFi4VOp6uznZZW76bGW3UQEZFitag+KSIierQwpIiISLEYUkREpFgMKSIiUiyGFBERKRZDioiIFIshRUREisWQIiIixWJIERGRYjGkiIhIsRhSRESkWAwpIiJSrP8PYCrAlp8l29IAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "test_smiles = \"C(O)=CN\"\n",
    "test_mol = Chem.MolFromSmiles(test_smiles)\n",
    "Chem.MolToSmiles(test_mol) # we have to do this so we populate the _smilesAtomOutputOrder property\n",
    "plt.imshow(Draw.MolToImage(test_mol))\n",
    "\n",
    "print(test_smiles)\n",
    "print()\n",
    "atom_info(test_mol, verbose=True)\n",
    "bond_info(test_mol, verbose=True)\n",
    "(_, edge_attr), (_, fc_edge_attr) = connect_info(test_mol, verbose=True)\n",
    "node_feature_matrix(test_mol, a2t, verbose=True )\n",
    "map_rdkit_bond_types(edge_attr, verbose=True)\n",
    "map_rdkit_bond_types(fc_edge_attr, verbose=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e4f80b6f",
   "metadata": {},
   "source": [
    "Next, we'll apply all these functions to each molecule in our Pandas dataframe."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "2cc44ed9",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 1000/1000 [00:03<00:00, 332.87it/s]\n"
     ]
    }
   ],
   "source": [
    "def prepare_df(row):\n",
    "        \n",
    "    mol = row['mol']\n",
    "    \n",
    "    (edge_index, edge_attr), (fc_edge_index, y_fc_edge_attr)  = connect_info(mol)\n",
    "    # y_boa is y labels for bag-of-atoms ->\n",
    "    # a (vocab_size,) shape labels that shows the number of atoms for each atom \n",
    "    x, pos, y_boa = node_feature_matrix(mol, a2t) \n",
    "    edge_attr = map_rdkit_bond_types(edge_attr)\n",
    "    y_fc_edge_attr = map_rdkit_bond_types(y_fc_edge_attr)\n",
    "    # note that PyG will automatically increment fc_edge_index\n",
    "    # as per the following doc:\n",
    "    # https://pytorch-geometric.readthedocs.io/en/latest/generated/torch_geometric.data.Batch.html#torch_geometric.data.Batch\n",
    "    data = Data(x=x,\n",
    "                edge_index=edge_index,\n",
    "                edge_attr=edge_attr,\n",
    "                y_boa=y_boa,\n",
    "                fc_edge_index=fc_edge_index,\n",
    "                y_fc_edge_attr=y_fc_edge_attr,\n",
    "                pos=pos,\n",
    "                graph_id = torch.tensor([row.name])) # to keep track of this data object and map to SMILES later\n",
    "    data.validate(raise_on_error=True)\n",
    "    return data\n",
    "\n",
    "df[\"data\"] = df.progress_apply(prepare_df, axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9c3412c9",
   "metadata": {},
   "source": [
    "Note that we are not quite ready to pass this into any Graph convolutional network, because we only have the tokens. We will take care of that in the forward pass though. For now, we will use batching here."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ac5bec93",
   "metadata": {},
   "source": [
    "## Training step"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "62eff811",
   "metadata": {},
   "outputs": [],
   "source": [
    "class DataFrameDataset(Dataset):\n",
    "    \"\"\"Wrapper class to use Pytorch DataLoader\"\"\"\n",
    "    def __init__(self, df, colname=\"data\"):\n",
    "        self.data = df[colname].values\n",
    "\n",
    "    def __len__(self):\n",
    "        return len(self.data)\n",
    "\n",
    "    def __getitem__(self, idx):\n",
    "        return self.data[idx]\n",
    "    \n",
    "###################################################\n",
    "# Hyperparameters\n",
    "DEBUG = True \n",
    "embd = 16 # embedding size of a vocab indice\n",
    "num_layers = 3    # number of GCN layers\n",
    "lr = 0.002\n",
    "betas = (0.9, 0.999)\n",
    "eps = 1e-08\n",
    "epochs = 2000\n",
    "lambda_boa = 0.05\n",
    "lambda_edge = 0.95\n",
    "lambda_kl = 0\n",
    "batch_size = 128\n",
    "shuffle = False\n",
    "###################################################"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "f49ed22b",
   "metadata": {},
   "outputs": [],
   "source": [
    "torch.manual_seed(42)\n",
    "\n",
    "vocab_size = len(a2t)\n",
    "\n",
    "dataset = DataFrameDataset(df, \"data\")\n",
    "loader = DataLoader(dataset, batch_size=batch_size, shuffle=shuffle)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "371e04f7",
   "metadata": {},
   "source": [
    "Let's take a look at one of the batches to see which data is bundled:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "id": "9747211b",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "DataBatch(x=[3054, 1], edge_index=[2, 6576], edge_attr=[6576], pos=[3054, 1], y_boa=[128, 8], fc_edge_index=[2, 72834], y_fc_edge_attr=[72834], graph_id=[128], batch=[3054], ptr=[129])\n"
     ]
    }
   ],
   "source": [
    "data = next(iter(loader))\n",
    "print(data)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "942e9b72",
   "metadata": {},
   "source": [
    "- `x` is the node (atom) features, and PyTorch Geometric has batched together 3054 of them.\n",
    "- `edge_index` is the edge connectivity in COO format and there are 6576 edges (bonds) in the batched graph\n",
    "- `edge_attr` is the edge (bond) features and the size matches `edge_index`\n",
    "- `pos` is an attribute we defined which encodes the unique position for each node in the molecule\n",
    "- `y_boa` is the target bag of atoms output where there are 8 possible atoms, and we have 128 different graphs (molecules)\n",
    "- `fc_edge_index` is another target label which specifies all 72834 edges for a fully connected graph where every sub-graph connects to each other. This is used for the second stage of MoleGen (bond decoding)\n",
    "- `y_fc_edge_attr` are the expected labels for each bond in the fully connected molecule\n",
    "- `graph_id` is a tag which can associate each batched subgraph to a particular entry in the Pandas dataframe\n",
    "- `batch` specifies which nodes belong to which subgraph in the `x` tensor\n",
    "- `ptr` is similar to `batch` excepts it specifies the start position of each new graph\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "874ce4fc",
   "metadata": {},
   "source": [
    "The model is defined below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "id": "d1d1ba9f",
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch.nn as nn\n",
    "from torch_geometric.data import Data\n",
    "from torch_geometric.nn import ResGatedGraphConv\n",
    "from torch_geometric.utils import scatter\n",
    "import torch\n",
    "import torch.nn.functional as F\n",
    "\n",
    "class AtomMLP(nn.Module):\n",
    "    \"\"\"Multi-layer perceptron (MLP) which predicts BOA for each graph\n",
    "    \n",
    "    Implementation of Eq. 9 in paper. This MLP takes in a graph level embedding 'z'\n",
    "    and produces a BOA where we essentially get an integer distribution of the predicted atoms\n",
    "\n",
    "    Args:\n",
    "        embd       : embedding dimension for latent space\n",
    "        vocab_size : number of unique atoms in training set\n",
    "        max_atoms  : max number of total atoms in any molecule\n",
    "    \"\"\"\n",
    "    def __init__(self, embd=16, vocab_size=8, max_atoms=100):\n",
    "        \n",
    "        super().__init__()   \n",
    "        \n",
    "        self.vocab_size = vocab_size\n",
    "        self.max_atoms = max_atoms\n",
    "        \n",
    "        # note there isn't much information on the paper on this MLP but I am assuming we\n",
    "        # use a nn.ReLU for the activation\n",
    "        self.mlp = nn.Sequential(\n",
    "            nn.Linear(embd, embd),\n",
    "            nn.ReLU(),\n",
    "            nn.Linear(embd, vocab_size*max_atoms),\n",
    "        )\n",
    "        \n",
    "    def forward(self, z):\n",
    "        \"\"\"Forward pass of Atom MLP\n",
    "        \n",
    "        We compute an output vector of size (num_graphs, vocab_size, max_atoms) with\n",
    "        the last dimension essentially representing a one-hot vector that has a dimension size\n",
    "        up to the maximum number of atoms. We perform the MLP with a flattened last dimension\n",
    "        (vocab_size*max_atoms) and then view it as a separate dimension for the loss calculation.\n",
    "        \n",
    "        Note that a small optimization here \n",
    "        is to not do the total number of atoms in the training set but simply the\n",
    "        highest number of any particular atom instead.\n",
    "        \n",
    "        Args:\n",
    "            z   : graph level embedding (num_graphs, embd)\n",
    "            \n",
    "        Returns:\n",
    "            boa : graph level BOA (num_graphs, vocab_size, max_atoms)\n",
    "        \"\"\"\n",
    "        boa = self.mlp(z)\n",
    "        \n",
    "        return boa.view(-1, self.vocab_size, self.max_atoms)\n",
    "\n",
    "\n",
    "class BondMLP(nn.Module):\n",
    "    \"\"\"Multi-layer perceptron which predicts which bond for each edge\n",
    "    \n",
    "    Implementation of Eq. 11 in paper. This MLP takes in edge embeddings 'e'\n",
    "    and produces an integer distribution of the predicted bond types\n",
    "\n",
    "    Args:\n",
    "        embd       : embedding dimension for latent space\n",
    "        num_bonds  : number of unique atoms in training set\n",
    "        num_bonds  : number of different types of bonds for classification\n",
    "        \n",
    "    \"\"\"\n",
    "    def __init__(self, embd=16, num_bonds=4):\n",
    "        super().__init__()   \n",
    "        \n",
    "        self.num_bonds = num_bonds\n",
    "        \n",
    "        self.mlp = nn.Sequential(\n",
    "            nn.Linear(embd, embd),\n",
    "            nn.ReLU(),\n",
    "            nn.Linear(embd, num_bonds),\n",
    "        )\n",
    "        \n",
    "    def forward(self, e):\n",
    "        \"\"\"Forward pass of Bond MLP\n",
    "        \n",
    "        We compute an output vector of size (num_edges, num_bonds) with\n",
    "        the last dimension representing a one-hot vector that has a dimension size\n",
    "        up to the different types of bonds.\n",
    "        \n",
    "        Args:\n",
    "            e   : edge embedding (num_edges, embd)\n",
    "            \n",
    "        Returns:\n",
    "            bonds : graph level BOA (num_edges, num_bonds)\n",
    "        \"\"\"\n",
    "        bonds = self.mlp(e)\n",
    "        return bonds\n",
    "\n",
    "        \n",
    "class AtomGCNLayer(nn.Module):\n",
    "    \"\"\"Updates atom embeddings with Conv->BN->Relu->Residual\n",
    "\n",
    "    Implementation of Eq. 4 in paper\n",
    "\n",
    "    Args:\n",
    "        embd       : embedding dimension for latent space    \n",
    "\n",
    "    \"\"\"\n",
    "    def __init__(self, embd=16):\n",
    "        super().__init__()\n",
    "\n",
    "        self.gcn = ResGatedGraphConv(in_channels=embd, out_channels=embd, edge_dim=embd)\n",
    "        self.bn = nn.BatchNorm1d(embd)\n",
    "        self.relu = nn.ReLU()\n",
    "\n",
    "    def forward(self, data):\n",
    "        \"\"\"Perform forward pass for atom GCN\n",
    "        \n",
    "        Args:\n",
    "            data  : PyG Data() object (possibly batched) with the following attributes:\n",
    "                x           : atom token, e.g. x[0] is the embedding vector for atom/node 0 (num_nodes, embd)\n",
    "                edge_index  : bond connectivity, e.g. atom at edge_index[0,0] connects to edge_index[1, 0] (2, num_edges)\n",
    "                edge_attr   : bond token, e.g. edge_attr[0] is the embedding vector for bond/edge 0 (num_edges, embd)\n",
    "                \n",
    "        Returns:\n",
    "            h     :   Updated node embedding (num_nodes, embd)\n",
    "            \n",
    "        \"\"\" \n",
    "        x, edge_index, edge_attr = data.x, data.edge_index, data.edge_attr\n",
    "\n",
    "        h = self.gcn(x, edge_index, edge_attr)\n",
    "        h = self.bn(h)\n",
    "        h = self.relu(h)\n",
    "        h = x + h # residual pathway\n",
    "        return h\n",
    "\n",
    "class BondGCNLayer(nn.Module):\n",
    "    \"\"\"Updates bond embeddings with Linear->BN->Relu->Residual\n",
    "\n",
    "    Implementation of Eq. 5 in paper\n",
    "    \n",
    "    Args:\n",
    "        embd       : embedding dimension for latent space\n",
    "\n",
    "    \"\"\"\n",
    "\n",
    "    def __init__(self, embd=16):\n",
    "        super().__init__()\n",
    "        self.v = nn.ModuleList([nn.Linear(embd,embd) for _ in range(3)])\n",
    "        self.bn = nn.BatchNorm1d(embd)\n",
    "        self.relu = nn.ReLU()\n",
    "\n",
    "    def forward(self, data):\n",
    "        \"\"\"Perform forward pass for bond GCN\n",
    "        \n",
    "        Args:\n",
    "            data  : PyG Data() object (possibly batched) with the following attributes:\n",
    "                x           : atom token, e.g. x[0] is the embedding vector for atom/node 0 (num_nodes, embd)\n",
    "                edge_index  : bond connectivity, e.g. atom at edge_index[0,0] connects to edge_index[1, 0] (2, num_edges)\n",
    "                edge_attr   : bond token, e.g. edge_attr[0] is the embedding vector for bond/edge 0 (num_edges, embd)\n",
    "                \n",
    "        Returns:\n",
    "            e     :   Updated edge embedding (num_edges, embd)\n",
    "            \n",
    "        \"\"\"\n",
    "        x, edge_index, edge_attr = data.x, data.edge_index, data.edge_attr\n",
    "\n",
    "        h_src = x[edge_index[0]] # oh... apparently we get this for free?\n",
    "        h_dest = x[edge_index[1]] # apparently pytorch geometric already handles large batch\n",
    "\n",
    "        e = self.v[0](edge_attr) + self.v[1](h_src) + self.v[2](h_dest)\n",
    "\n",
    "        e = self.bn(e)\n",
    "        e = self.relu(e)\n",
    "\n",
    "        e = edge_attr + e # residual pathway\n",
    "        return e\n",
    "\n",
    "class MoleGen(nn.Module):\n",
    "    \"\"\"Main model for VAE molecular generation\n",
    "    \n",
    "    This model is an implementation of the paper:\n",
    "    A Two-Step Graph Convolutional Decoder for Molecule Generation\n",
    "    by Bresson et Laurent (2019). A molecule graph is first encoded \n",
    "    into a latent representation 'z', which is used to produce a \n",
    "    'Bag of Atoms' (BOA). This BOA tells us how many of each atom\n",
    "    we have in the predicted molecule, ignoring connectivity. \n",
    "    The second stage takes 'z' and the original input formula and\n",
    "    decodes the edge feature connectivity from a fully connected network.\n",
    "    \n",
    "    Args:\n",
    "        vocab_size : number of unique atoms in training set\n",
    "        embd       : embedding dimension for latent space\n",
    "        num_layers : number of GNN layers (message passing/k-hop distance)\n",
    "        max_atoms  : max number of total atoms in any molecule\n",
    "        num_bonds  : number of different types of bonds for classification\n",
    "    \"\"\"\n",
    "    def __init__(self, vocab_size=8, embd=16, num_layers=4, max_atoms=100, num_bonds=4):\n",
    "        \n",
    "        super().__init__()\n",
    "        self.vocab_size = vocab_size\n",
    "        self.embd = embd\n",
    "        self.num_layers = num_layers\n",
    "        self.max_atoms = max_atoms\n",
    "\n",
    "        self.bond_embeddings = nn.Embedding(vocab_size, embd) \n",
    "        self.atom_encoder = nn.ModuleList([AtomGCNLayer(embd) for _ in range(num_layers)])\n",
    "        self.bond_encoder = nn.ModuleList([BondGCNLayer(embd) for _ in range(num_layers)])\n",
    "        self.atom_decoder = nn.ModuleList([AtomGCNLayer(embd) for _ in range(num_layers)])\n",
    "        self.bond_decoder = nn.ModuleList([BondGCNLayer(embd) for _ in range(num_layers)])\n",
    "        \n",
    "        self.N = torch.distributions.Normal(0, 1)\n",
    "\n",
    "        self.linear = nn.ModuleDict(dict(\n",
    "            atom_embedding=nn.Linear(max_atoms + vocab_size, embd),\n",
    "            a_mu=nn.Linear(embd, embd),\n",
    "            b_mu=nn.Linear(embd, embd),\n",
    "            c_mu=nn.Linear(embd, embd),\n",
    "            d_mu=nn.Linear(embd, embd),\n",
    "            a_sigma=nn.Linear(embd, embd),\n",
    "            b_sigma=nn.Linear(embd, embd),\n",
    "            c_sigma=nn.Linear(embd, embd),\n",
    "            d_sigma=nn.Linear(embd, embd),\n",
    "            u=nn.Linear(embd, embd)\n",
    "        ))\n",
    "\n",
    "        self.sigmoid = nn.Sigmoid()\n",
    "        \n",
    "        self.atom_mlp = AtomMLP(embd, vocab_size, max_atoms)\n",
    "        self.bond_mlp = BondMLP(embd, num_bonds)\n",
    "        \n",
    "    def forward(self, input_data):\n",
    "        \"\"\"Forward pass for MoleGen\n",
    "        \n",
    "        First we look up the atom and bond embeddings, and then apply GCN layers iteratively.\n",
    "        Afterwards we reduce the node and edge embeddings into a single graph latent vector 'z'.\n",
    "        We apply MLP to 'z' to produce a BOA. Then we use the original 'x' tokens and the 'z' vector\n",
    "        to produce an edge probability matrix, which we again apply an MLP to to get 's' which is\n",
    "        the predicted bond tokens for each edge of a fully connected network.\n",
    "        \n",
    "        Args:\n",
    "            input_data  : PyG Data() object (possibly batched) with the following attributes:\n",
    "                x           : atom token, e.g. x[0] is the integer token for atom/node 0 (num_nodes, 1)\n",
    "                edge_index  : bond connectivity, e.g. atom at edge_index[0,0] connects to edge_index[1, 0] (2, num_edges)\n",
    "                edge_attr   : bond token, e.g. edge_attr[0] is the integer token for bond/edge 0 (num_edges, )\n",
    "                \n",
    "        Returns:\n",
    "            boa     :   Bag of Atoms prediction (num_graphs, vocab_size, max_atoms)\n",
    "            z       :   Per graph latent representation (num_graphs, embd)\n",
    "            s       :   Edge probability matrix (num_fc_edges, num_bonds)\n",
    "        \"\"\"\n",
    "\n",
    "\n",
    "        ######################################################################################\n",
    "        # Encoding step\n",
    "        ######################################################################################\n",
    "        \n",
    "        # Break symmetry and add positional embeddings to atom embeddings\n",
    "        one_hot_positions = F.one_hot(input_data.pos.view(-1).long(), num_classes=self.max_atoms) # view avoids extra dimension added\n",
    "        one_hot_atom_tokens = F.one_hot(input_data.x.view(-1).long(), num_classes=self.vocab_size)\n",
    "        one_hot_concat = torch.cat((one_hot_positions, one_hot_atom_tokens), 1).to(dtype=torch.float32)\n",
    "        atom_embedding = self.linear['atom_embedding'](one_hot_concat)\n",
    "        \n",
    "        # Get bond embeddings\n",
    "        bond_embedding = self.bond_embeddings(input_data.edge_attr)\n",
    "\n",
    "        # Apply GCN layers iteratively (encoding)\n",
    "        h = atom_embedding\n",
    "        e = bond_embedding\n",
    "        for i in range(self.num_layers):\n",
    "            data = Data(x=h, edge_index=input_data.edge_index, edge_attr=e)\n",
    "            h = self.atom_encoder[i](data)\n",
    "            e = self.bond_encoder[i](data)\n",
    "\n",
    "        # Extract source and destination atom features \n",
    "        h_src = h[input_data.edge_index[0]]\n",
    "        h_dest = h[input_data.edge_index[1]]\n",
    "\n",
    "        # Apply linear and activation before reduction step\n",
    "        # mu = (self.sigmoid(self.linear['a_mu'](e) + self.linear['b_mu'](h_src) + self.linear['c_mu'](h_dest)))*self.linear['d_mu'](e)\n",
    "        # sigma = (self.sigmoid(self.linear['a_sigma'](e) + self.linear['b_sigma'](h_src) + self.linear['c_sigma'](h_dest)))*self.linear['d_sigma'](e)\n",
    "        \n",
    "        z = (self.sigmoid(self.linear['a_sigma'](e) + self.linear['b_sigma'](h_src) + self.linear['c_sigma'](h_dest)))*self.linear['d_sigma'](e)\n",
    "        \n",
    "        # the .batch attribute only maps to nodes\n",
    "        # to get a mapping to the edge -> graph we simply index\n",
    "        # into the batch to get the indexes of which edge corresponds to which graph \n",
    "        batch_edge = input_data.batch[input_data.edge_index[0]] # (num_edges, )\n",
    "        \n",
    "        # now batch_edge will ressemble input_data.batch but for edges instead of nodes\n",
    "        # apply scatter operation to get a per graph output\n",
    "        # mu = scatter(mu, batch_edge, dim=0, reduce='sum') # (num_graphs, embd)\n",
    "        # sigma = scatter(sigma, batch_edge, dim=0, reduce='sum') # (num_graphs, embd)\n",
    "        \n",
    "        # z = mu + sigma*self.N.sample(mu.shape)\n",
    "        z = scatter(z, batch_edge, dim=0, reduce='sum')\n",
    "        \n",
    "        # kl = -((1 + torch.log(sigma**2) - (mu**2) - (sigma**2)).sum()) # from https://arxiv.org/pdf/1312.6114\n",
    "        kl = torch.tensor([0])\n",
    "        boa = self.atom_mlp(z) # (num_graphs, vocab_size, max_atoms)\n",
    "\n",
    "\n",
    "        ######################################################################################\n",
    "        # Decoding step\n",
    "        ######################################################################################\n",
    "\n",
    "        # in the bond generation step, each edge gets the same initial feature vector\n",
    "        fc_bond_embedding = self.linear['u'](z) # (num_graphs, embd) \n",
    "        \n",
    "        # same trick as before where we convert node mappings to edge mappings\n",
    "        fc_batch_edge = input_data.batch[input_data.fc_edge_index[0]]\n",
    "        \n",
    "        # we index into our bond embeddings to get the bond embeddings\n",
    "        # for each edge per graph, since fc_bond_embedding is batched\n",
    "        fc_edge_attr = fc_bond_embedding[fc_batch_edge] # (num_fc_edges, embd)\n",
    "        \n",
    "        # re-use original atom embedding since the fc graph has the same nodes just with different bonds\n",
    "        fc_atom_embedding = atom_embedding # (num_atoms, embd)\n",
    "        \n",
    "        # Apply GCN layers iteratively (decoding)\n",
    "        h = fc_atom_embedding\n",
    "        e = fc_edge_attr\n",
    "        for i in range(self.num_layers):\n",
    "            data = Data(x=h, edge_index=input_data.fc_edge_index, edge_attr=e)\n",
    "            h = self.atom_decoder[i](data)\n",
    "            e = self.bond_decoder[i](data)  \n",
    "        \n",
    "        # now take each edge and apply MLP to predict bond type for each\n",
    "        s = self.bond_mlp(e)\n",
    "        \n",
    "        return boa, z, s, kl\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "id": "ba5bf3ba",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "MoleGen(\n",
      "  (bond_embeddings): Embedding(8, 16)\n",
      "  (atom_encoder): ModuleList(\n",
      "    (0-2): 3 x AtomGCNLayer(\n",
      "      (gcn): ResGatedGraphConv(16, 16)\n",
      "      (bn): BatchNorm1d(16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
      "      (relu): ReLU()\n",
      "    )\n",
      "  )\n",
      "  (bond_encoder): ModuleList(\n",
      "    (0-2): 3 x BondGCNLayer(\n",
      "      (v): ModuleList(\n",
      "        (0-2): 3 x Linear(in_features=16, out_features=16, bias=True)\n",
      "      )\n",
      "      (bn): BatchNorm1d(16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
      "      (relu): ReLU()\n",
      "    )\n",
      "  )\n",
      "  (atom_decoder): ModuleList(\n",
      "    (0-2): 3 x AtomGCNLayer(\n",
      "      (gcn): ResGatedGraphConv(16, 16)\n",
      "      (bn): BatchNorm1d(16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
      "      (relu): ReLU()\n",
      "    )\n",
      "  )\n",
      "  (bond_decoder): ModuleList(\n",
      "    (0-2): 3 x BondGCNLayer(\n",
      "      (v): ModuleList(\n",
      "        (0-2): 3 x Linear(in_features=16, out_features=16, bias=True)\n",
      "      )\n",
      "      (bn): BatchNorm1d(16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
      "      (relu): ReLU()\n",
      "    )\n",
      "  )\n",
      "  (linear): ModuleDict(\n",
      "    (atom_embedding): Linear(in_features=45, out_features=16, bias=True)\n",
      "    (a_mu): Linear(in_features=16, out_features=16, bias=True)\n",
      "    (b_mu): Linear(in_features=16, out_features=16, bias=True)\n",
      "    (c_mu): Linear(in_features=16, out_features=16, bias=True)\n",
      "    (d_mu): Linear(in_features=16, out_features=16, bias=True)\n",
      "    (a_sigma): Linear(in_features=16, out_features=16, bias=True)\n",
      "    (b_sigma): Linear(in_features=16, out_features=16, bias=True)\n",
      "    (c_sigma): Linear(in_features=16, out_features=16, bias=True)\n",
      "    (d_sigma): Linear(in_features=16, out_features=16, bias=True)\n",
      "    (u): Linear(in_features=16, out_features=16, bias=True)\n",
      "  )\n",
      "  (sigmoid): Sigmoid()\n",
      "  (atom_mlp): AtomMLP(\n",
      "    (mlp): Sequential(\n",
      "      (0): Linear(in_features=16, out_features=16, bias=True)\n",
      "      (1): ReLU()\n",
      "      (2): Linear(in_features=16, out_features=296, bias=True)\n",
      "    )\n",
      "  )\n",
      "  (bond_mlp): BondMLP(\n",
      "    (mlp): Sequential(\n",
      "      (0): Linear(in_features=16, out_features=16, bias=True)\n",
      "      (1): ReLU()\n",
      "      (2): Linear(in_features=16, out_features=4, bias=True)\n",
      "    )\n",
      "  )\n",
      ")\n"
     ]
    }
   ],
   "source": [
    "model = MoleGen(vocab_size=vocab_size, num_layers=num_layers, embd=embd, max_atoms=max_atoms)\n",
    "\n",
    "optimizer = torch.optim.Adam(model.parameters(), lr=lr, betas=betas, eps=eps)\n",
    "loss_fn = torch.nn.CrossEntropyLoss()\n",
    "\n",
    "print(model)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "038d9c75",
   "metadata": {},
   "source": [
    "We can optionally load a checkpoint if it exists"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "058b3037",
   "metadata": {},
   "outputs": [],
   "source": [
    "torch.serialization.add_safe_globals([float])\n",
    "checkpoint = torch.load(\"molegen.ckpt\", weights_only=True)\n",
    "loss = checkpoint['loss']\n",
    "epoch = checkpoint['epoch']\n",
    "model.load_state_dict(checkpoint['model_state_dict'])\n",
    "optimizer.load_state_dict(checkpoint['optimizer_state_dict'])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "54ed8fc3",
   "metadata": {},
   "source": [
    "We define a useful debug function that is output at each batch"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "id": "e60956bf",
   "metadata": {},
   "outputs": [],
   "source": [
    "def debug_fn(batch, boa, z, s, MAX_SMILES_STRING=50):\n",
    "    \n",
    "    smiles = df['canonical_smiles'].iloc[batch.graph_id[0].item()]\n",
    "    \n",
    "    print(f\"({idx:<2}) {smiles[:MAX_SMILES_STRING]:<{MAX_SMILES_STRING}}{'...' if len(smiles) > MAX_SMILES_STRING else '   '} | \", end=\"\")\n",
    "    \n",
    "    boa_actual = batch.y_boa[0]\n",
    "    boa_pred = torch.argmax(boa[0], dim=-1)\n",
    "    \n",
    "    for token, (count, pred_count) in enumerate(zip(boa_actual, boa_pred)):\n",
    "        print(f\"{t2a[token]}({count}, {pred_count}) | \", end=\"\")\n",
    "    \n",
    "    \n",
    "    # select only edges that correspond to first graph \n",
    "    edge = batch.batch[batch.fc_edge_index[0]] # (num_edges, )\n",
    "    edge = torch.arange((edge[edge == 0]).shape[0]) # (num_edges_graph0, )\n",
    "    edge_pred = s[edge] # (num_edges_graph0, C)\n",
    "    edge_pred = torch.argmax(edge_pred, dim=-1) # (num_edges_graph0, )\n",
    "    \n",
    "    src_atoms = batch.fc_edge_index[0, edge] # (num_edges_graph0, )\n",
    "    dest_atoms = batch.fc_edge_index[1, edge] # (num_edges_graph0, )\n",
    "    src_atom_feats = batch.x[src_atoms].squeeze(-1) # (num_nodes_graph0, )\n",
    "    dest_atom_feats = batch.x[dest_atoms].squeeze(-1) # (num_nodes_graph0, )\n",
    "\n",
    "    # only show the edges that are not zero because we would have too many otherwise\n",
    "    correct_edges = 0\n",
    "    num_edges = 0\n",
    "    for num, (src, dest, actual, predicted) in enumerate(zip(src_atom_feats, dest_atom_feats, batch.y_fc_edge_attr, edge_pred)):\n",
    "        if actual != 0:\n",
    "            num_edges += 1\n",
    "            if actual == predicted:\n",
    "                correct_edges += 1\n",
    "                \n",
    "    print(f\"edges {correct_edges}/{num_edges}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "id": "64e124cf",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(0 ) CC(C)(C)c1ccc2occ(CC(=O)Nc3ccccc3F)c2c1               | C(20, 5) | O(2, 19) | N(1, 17) | F(1, 26) | S(0, 25) | Cl(0, 7) | Br(0, 33) | I(0, 29) | edges 42/52\n",
      "(1 ) CSc1ccc(/C=c2\\sc3ncnn3c2=O)cc1                        | C(12, 5) | O(1, 12) | N(3, 17) | F(0, 26) | S(2, 23) | Cl(0, 7) | Br(0, 26) | I(0, 29) | edges 4/40\n",
      "(2 ) COc1cc(OC)cc([C@H]2CC[NH+](CCC(F)(F)F)C2)c1           | C(15, 5) | O(2, 19) | N(1, 24) | F(3, 26) | S(0, 0) | Cl(0, 27) | Br(0, 33) | I(0, 5) | edges 0/44\n",
      "(3 ) C[C@H]([NH2+]C1C[C@H](C)O[C@@H](C)C1)c1c[nH]c2cc(F... | C(17, 5) | O(1, 19) | N(2, 24) | F(1, 7) | S(0, 18) | Cl(0, 23) | Br(0, 33) | I(0, 5) | edges 0/46\n",
      "(4 ) CS[C@@H]1CC[C@H](NC(=O)CCC(=O)c2ccc(C)s2)C1           | C(15, 5) | O(2, 7) | N(1, 24) | F(0, 6) | S(2, 0) | Cl(0, 27) | Br(0, 26) | I(0, 29) | edges 0/42\n",
      "(5 ) CCCn1/c(=N/C(=O)[C@@H](CCSC)NC(N)=O)[nH]c2ccccc21     | C(16, 5) | O(2, 7) | N(5, 24) | F(0, 6) | S(1, 23) | Cl(0, 27) | Br(0, 26) | I(0, 29) | edges 0/50\n",
      "(6 ) COc1ccc(F)cc1NC(=O)N1CCO[C@H](c2ccc(C)o2)C1           | C(17, 5) | O(4, 7) | N(2, 1) | F(1, 0) | S(0, 34) | Cl(0, 27) | Br(0, 26) | I(0, 29) | edges 0/52\n",
      "(7 ) O=C(C/C(=N\\Nc1nc(-c2ccccc2)cs1)c1ccccc1)C(F)(F)F      | C(19, 5) | O(1, 7) | N(3, 24) | F(3, 0) | S(1, 34) | Cl(0, 27) | Br(0, 26) | I(0, 29) | edges 0/58\n",
      "\n",
      "Epoch 0 ------ average losses ------- | total: 1.80240 | boa: 6.70273 | edge: 1.54449 | kl: 0.00000 | \n",
      "\n",
      "(0 ) CC(C)(C)c1ccc2occ(CC(=O)Nc3ccccc3F)c2c1               | C(20, 6) | O(2, 7) | N(1, 1) | F(1, 0) | S(0, 34) | Cl(0, 23) | Br(0, 26) | I(0, 5) | edges 0/52\n",
      "(1 ) CSc1ccc(/C=c2\\sc3ncnn3c2=O)cc1                        | C(12, 5) | O(1, 7) | N(3, 24) | F(0, 0) | S(2, 1) | Cl(0, 27) | Br(0, 26) | I(0, 29) | edges 0/40\n",
      "(2 ) COc1cc(OC)cc([C@H]2CC[NH+](CCC(F)(F)F)C2)c1           | C(15, 6) | O(2, 7) | N(1, 1) | F(3, 0) | S(0, 34) | Cl(0, 23) | Br(0, 26) | I(0, 29) | edges 0/44\n",
      "(3 ) C[C@H]([NH2+]C1C[C@H](C)O[C@@H](C)C1)c1c[nH]c2cc(F... | C(17, 6) | O(1, 7) | N(2, 1) | F(1, 0) | S(0, 34) | Cl(0, 23) | Br(0, 26) | I(0, 5) | edges 0/46\n",
      "(4 ) CS[C@@H]1CC[C@H](NC(=O)CCC(=O)c2ccc(C)s2)C1           | C(15, 5) | O(2, 7) | N(1, 24) | F(0, 0) | S(2, 1) | Cl(0, 27) | Br(0, 26) | I(0, 29) | edges 0/42\n",
      "(5 ) CCCn1/c(=N/C(=O)[C@@H](CCSC)NC(N)=O)[nH]c2ccccc21     | C(16, 5) | O(2, 7) | N(5, 24) | F(0, 0) | S(1, 1) | Cl(0, 27) | Br(0, 26) | I(0, 29) | edges 0/50\n",
      "(6 ) COc1ccc(F)cc1NC(=O)N1CCO[C@H](c2ccc(C)o2)C1           | C(17, 16) | O(4, 1) | N(2, 1) | F(1, 0) | S(0, 34) | Cl(0, 23) | Br(0, 26) | I(0, 29) | edges 0/52\n",
      "(7 ) O=C(C/C(=N\\Nc1nc(-c2ccccc2)cs1)c1ccccc1)C(F)(F)F      | C(19, 6) | O(1, 1) | N(3, 1) | F(3, 0) | S(1, 34) | Cl(0, 23) | Br(0, 26) | I(0, 5) | edges 0/58\n",
      "\n",
      "Epoch 1 ------ average losses ------- | total: 0.58072 | boa: 4.44354 | edge: 0.37742 | kl: 0.00000 | \n",
      "\n",
      "(0 ) CC(C)(C)c1ccc2occ(CC(=O)Nc3ccccc3F)c2c1               | C(20, 6) | O(2, 1) | N(1, 1) | F(1, 0) | S(0, 1) | Cl(0, 27) | Br(0, 26) | I(0, 8) | edges 0/52\n",
      "(1 ) CSc1ccc(/C=c2\\sc3ncnn3c2=O)cc1                        | C(12, 5) | O(1, 1) | N(3, 17) | F(0, 0) | S(2, 1) | Cl(0, 23) | Br(0, 26) | I(0, 24) | edges 0/40\n",
      "(2 ) COc1cc(OC)cc([C@H]2CC[NH+](CCC(F)(F)F)C2)c1           | C(15, 16) | O(2, 1) | N(1, 1) | F(3, 0) | S(0, 1) | Cl(0, 27) | Br(0, 26) | I(0, 9) | edges 0/44\n",
      "(3 ) C[C@H]([NH2+]C1C[C@H](C)O[C@@H](C)C1)c1c[nH]c2cc(F... | C(17, 6) | O(1, 1) | N(2, 5) | F(1, 0) | S(0, 28) | Cl(0, 31) | Br(0, 26) | I(0, 9) | edges 0/46\n",
      "(4 ) CS[C@@H]1CC[C@H](NC(=O)CCC(=O)c2ccc(C)s2)C1           | C(15, 19) | O(2, 1) | N(1, 17) | F(0, 0) | S(2, 1) | Cl(0, 27) | Br(0, 26) | I(0, 0) | edges 0/42\n",
      "(5 ) CCCn1/c(=N/C(=O)[C@@H](CCSC)NC(N)=O)[nH]c2ccccc21     | C(16, 17) | O(2, 1) | N(5, 17) | F(0, 13) | S(1, 1) | Cl(0, 27) | Br(0, 26) | I(0, 0) | edges 0/50\n",
      "(6 ) COc1ccc(F)cc1NC(=O)N1CCO[C@H](c2ccc(C)o2)C1           | C(17, 19) | O(4, 1) | N(2, 5) | F(1, 0) | S(0, 1) | Cl(0, 31) | Br(0, 26) | I(0, 9) | edges 0/52\n",
      "(7 ) O=C(C/C(=N\\Nc1nc(-c2ccccc2)cs1)c1ccccc1)C(F)(F)F      | C(19, 11) | O(1, 1) | N(3, 32) | F(3, 0) | S(1, 1) | Cl(0, 10) | Br(0, 26) | I(0, 0) | edges 0/58\n",
      "\n",
      "Epoch 2 ------ average losses ------- | total: 0.48473 | boa: 3.06000 | edge: 0.34919 | kl: 0.00000 | \n",
      "\n",
      "(0 ) CC(C)(C)c1ccc2occ(CC(=O)Nc3ccccc3F)c2c1               | C(20, 19) | O(2, 2) | N(1, 5) | F(1, 0) | S(0, 7) | Cl(0, 27) | Br(0, 26) | I(0, 0) | edges 0/52\n",
      "(1 ) CSc1ccc(/C=c2\\sc3ncnn3c2=O)cc1                        | C(12, 11) | O(1, 2) | N(3, 32) | F(0, 0) | S(2, 1) | Cl(0, 27) | Br(0, 26) | I(0, 0) | edges 0/40\n",
      "(2 ) COc1cc(OC)cc([C@H]2CC[NH+](CCC(F)(F)F)C2)c1           | C(15, 16) | O(2, 2) | N(1, 5) | F(3, 0) | S(0, 0) | Cl(0, 27) | Br(0, 26) | I(0, 0) | edges 0/44\n",
      "(3 ) C[C@H]([NH2+]C1C[C@H](C)O[C@@H](C)C1)c1c[nH]c2cc(F... | C(17, 19) | O(1, 29) | N(2, 5) | F(1, 0) | S(0, 0) | Cl(0, 27) | Br(0, 26) | I(0, 0) | edges 0/46\n",
      "(4 ) CS[C@@H]1CC[C@H](NC(=O)CCC(=O)c2ccc(C)s2)C1           | C(15, 11) | O(2, 2) | N(1, 5) | F(0, 0) | S(2, 1) | Cl(0, 27) | Br(0, 0) | I(0, 0) | edges 0/42\n",
      "(5 ) CCCn1/c(=N/C(=O)[C@@H](CCSC)NC(N)=O)[nH]c2ccccc21     | C(16, 11) | O(2, 2) | N(5, 32) | F(0, 13) | S(1, 1) | Cl(0, 27) | Br(0, 0) | I(0, 0) | edges 0/50\n",
      "(6 ) COc1ccc(F)cc1NC(=O)N1CCO[C@H](c2ccc(C)o2)C1           | C(17, 16) | O(4, 2) | N(2, 5) | F(1, 0) | S(0, 0) | Cl(0, 27) | Br(0, 0) | I(0, 0) | edges 0/52\n",
      "(7 ) O=C(C/C(=N\\Nc1nc(-c2ccccc2)cs1)c1ccccc1)C(F)(F)F      | C(19, 24) | O(1, 2) | N(3, 5) | F(3, 0) | S(1, 1) | Cl(0, 27) | Br(0, 0) | I(0, 0) | edges 0/58\n",
      "\n",
      "Epoch 3 ------ average losses ------- | total: 0.44911 | boa: 2.31739 | edge: 0.35078 | kl: 0.00000 | \n",
      "\n",
      "(0 ) CC(C)(C)c1ccc2occ(CC(=O)Nc3ccccc3F)c2c1               | C(20, 19) | O(2, 2) | N(1, 5) | F(1, 0) | S(0, 0) | Cl(0, 27) | Br(0, 0) | I(0, 0) | edges 0/52\n",
      "(1 ) CSc1ccc(/C=c2\\sc3ncnn3c2=O)cc1                        | C(12, 18) | O(1, 2) | N(3, 5) | F(0, 0) | S(2, 1) | Cl(0, 27) | Br(0, 0) | I(0, 0) | edges 0/40\n",
      "(2 ) COc1cc(OC)cc([C@H]2CC[NH+](CCC(F)(F)F)C2)c1           | C(15, 16) | O(2, 3) | N(1, 5) | F(3, 0) | S(0, 0) | Cl(0, 27) | Br(0, 0) | I(0, 0) | edges 0/44\n",
      "(3 ) C[C@H]([NH2+]C1C[C@H](C)O[C@@H](C)C1)c1c[nH]c2cc(F... | C(17, 19) | O(1, 3) | N(2, 5) | F(1, 0) | S(0, 0) | Cl(0, 1) | Br(0, 0) | I(0, 0) | edges 0/46\n",
      "(4 ) CS[C@@H]1CC[C@H](NC(=O)CCC(=O)c2ccc(C)s2)C1           | C(15, 18) | O(2, 2) | N(1, 5) | F(0, 0) | S(2, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/42\n",
      "(5 ) CCCn1/c(=N/C(=O)[C@@H](CCSC)NC(N)=O)[nH]c2ccccc21     | C(16, 18) | O(2, 2) | N(5, 5) | F(0, 0) | S(1, 1) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/50\n",
      "(6 ) COc1ccc(F)cc1NC(=O)N1CCO[C@H](c2ccc(C)o2)C1           | C(17, 18) | O(4, 2) | N(2, 5) | F(1, 0) | S(0, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/52\n",
      "(7 ) O=C(C/C(=N\\Nc1nc(-c2ccccc2)cs1)c1ccccc1)C(F)(F)F      | C(19, 18) | O(1, 2) | N(3, 5) | F(3, 0) | S(1, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/58\n",
      "\n",
      "Epoch 4 ------ average losses ------- | total: 0.41369 | boa: 1.68418 | edge: 0.34682 | kl: 0.00000 | \n",
      "\n",
      "(0 ) CC(C)(C)c1ccc2occ(CC(=O)Nc3ccccc3F)c2c1               | C(20, 18) | O(2, 3) | N(1, 2) | F(1, 0) | S(0, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/52\n",
      "(1 ) CSc1ccc(/C=c2\\sc3ncnn3c2=O)cc1                        | C(12, 18) | O(1, 2) | N(3, 5) | F(0, 0) | S(2, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/40\n",
      "(2 ) COc1cc(OC)cc([C@H]2CC[NH+](CCC(F)(F)F)C2)c1           | C(15, 19) | O(2, 3) | N(1, 5) | F(3, 0) | S(0, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/44\n",
      "(3 ) C[C@H]([NH2+]C1C[C@H](C)O[C@@H](C)C1)c1c[nH]c2cc(F... | C(17, 18) | O(1, 3) | N(2, 5) | F(1, 0) | S(0, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/46\n",
      "(4 ) CS[C@@H]1CC[C@H](NC(=O)CCC(=O)c2ccc(C)s2)C1           | C(15, 18) | O(2, 2) | N(1, 2) | F(0, 0) | S(2, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/42\n",
      "(5 ) CCCn1/c(=N/C(=O)[C@@H](CCSC)NC(N)=O)[nH]c2ccccc21     | C(16, 18) | O(2, 2) | N(5, 2) | F(0, 0) | S(1, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/50\n",
      "(6 ) COc1ccc(F)cc1NC(=O)N1CCO[C@H](c2ccc(C)o2)C1           | C(17, 18) | O(4, 1) | N(2, 5) | F(1, 0) | S(0, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/52\n",
      "(7 ) O=C(C/C(=N\\Nc1nc(-c2ccccc2)cs1)c1ccccc1)C(F)(F)F      | C(19, 18) | O(1, 2) | N(3, 2) | F(3, 0) | S(1, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/58\n",
      "\n",
      "Epoch 5 ------ average losses ------- | total: 0.39248 | boa: 1.35423 | edge: 0.34186 | kl: 0.00000 | \n",
      "\n",
      "(0 ) CC(C)(C)c1ccc2occ(CC(=O)Nc3ccccc3F)c2c1               | C(20, 18) | O(2, 3) | N(1, 2) | F(1, 0) | S(0, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/52\n",
      "(1 ) CSc1ccc(/C=c2\\sc3ncnn3c2=O)cc1                        | C(12, 18) | O(1, 1) | N(3, 4) | F(0, 0) | S(2, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/40\n",
      "(2 ) COc1cc(OC)cc([C@H]2CC[NH+](CCC(F)(F)F)C2)c1           | C(15, 16) | O(2, 3) | N(1, 4) | F(3, 0) | S(0, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/44\n",
      "(3 ) C[C@H]([NH2+]C1C[C@H](C)O[C@@H](C)C1)c1c[nH]c2cc(F... | C(17, 18) | O(1, 1) | N(2, 1) | F(1, 0) | S(0, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/46\n",
      "(4 ) CS[C@@H]1CC[C@H](NC(=O)CCC(=O)c2ccc(C)s2)C1           | C(15, 18) | O(2, 1) | N(1, 4) | F(0, 0) | S(2, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/42\n",
      "(5 ) CCCn1/c(=N/C(=O)[C@@H](CCSC)NC(N)=O)[nH]c2ccccc21     | C(16, 15) | O(2, 2) | N(5, 4) | F(0, 0) | S(1, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/50\n",
      "(6 ) COc1ccc(F)cc1NC(=O)N1CCO[C@H](c2ccc(C)o2)C1           | C(17, 20) | O(4, 1) | N(2, 4) | F(1, 0) | S(0, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/52\n",
      "(7 ) O=C(C/C(=N\\Nc1nc(-c2ccccc2)cs1)c1ccccc1)C(F)(F)F      | C(19, 18) | O(1, 1) | N(3, 4) | F(3, 0) | S(1, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/58\n",
      "\n",
      "Epoch 6 ------ average losses ------- | total: 0.38166 | boa: 1.20856 | edge: 0.33814 | kl: 0.00000 | \n",
      "\n",
      "(0 ) CC(C)(C)c1ccc2occ(CC(=O)Nc3ccccc3F)c2c1               | C(20, 18) | O(2, 3) | N(1, 3) | F(1, 0) | S(0, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/52\n",
      "(1 ) CSc1ccc(/C=c2\\sc3ncnn3c2=O)cc1                        | C(12, 18) | O(1, 1) | N(3, 3) | F(0, 0) | S(2, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/40\n",
      "(2 ) COc1cc(OC)cc([C@H]2CC[NH+](CCC(F)(F)F)C2)c1           | C(15, 20) | O(2, 1) | N(1, 4) | F(3, 0) | S(0, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/44\n",
      "(3 ) C[C@H]([NH2+]C1C[C@H](C)O[C@@H](C)C1)c1c[nH]c2cc(F... | C(17, 20) | O(1, 1) | N(2, 3) | F(1, 0) | S(0, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/46\n",
      "(4 ) CS[C@@H]1CC[C@H](NC(=O)CCC(=O)c2ccc(C)s2)C1           | C(15, 18) | O(2, 3) | N(1, 3) | F(0, 0) | S(2, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/42\n",
      "(5 ) CCCn1/c(=N/C(=O)[C@@H](CCSC)NC(N)=O)[nH]c2ccccc21     | C(16, 15) | O(2, 3) | N(5, 3) | F(0, 0) | S(1, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/50\n",
      "(6 ) COc1ccc(F)cc1NC(=O)N1CCO[C@H](c2ccc(C)o2)C1           | C(17, 20) | O(4, 1) | N(2, 3) | F(1, 0) | S(0, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/52\n",
      "(7 ) O=C(C/C(=N\\Nc1nc(-c2ccccc2)cs1)c1ccccc1)C(F)(F)F      | C(19, 18) | O(1, 3) | N(3, 3) | F(3, 0) | S(1, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/58\n",
      "\n",
      "Epoch 7 ------ average losses ------- | total: 0.37631 | boa: 1.13699 | edge: 0.33627 | kl: 0.00000 | \n",
      "\n",
      "(0 ) CC(C)(C)c1ccc2occ(CC(=O)Nc3ccccc3F)c2c1               | C(20, 18) | O(2, 1) | N(1, 3) | F(1, 0) | S(0, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/52\n",
      "(1 ) CSc1ccc(/C=c2\\sc3ncnn3c2=O)cc1                        | C(12, 18) | O(1, 3) | N(3, 3) | F(0, 0) | S(2, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/40\n",
      "(2 ) COc1cc(OC)cc([C@H]2CC[NH+](CCC(F)(F)F)C2)c1           | C(15, 16) | O(2, 1) | N(1, 3) | F(3, 0) | S(0, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/44\n",
      "(3 ) C[C@H]([NH2+]C1C[C@H](C)O[C@@H](C)C1)c1c[nH]c2cc(F... | C(17, 16) | O(1, 1) | N(2, 3) | F(1, 0) | S(0, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/46\n",
      "(4 ) CS[C@@H]1CC[C@H](NC(=O)CCC(=O)c2ccc(C)s2)C1           | C(15, 19) | O(2, 2) | N(1, 2) | F(0, 0) | S(2, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/42\n",
      "(5 ) CCCn1/c(=N/C(=O)[C@@H](CCSC)NC(N)=O)[nH]c2ccccc21     | C(16, 17) | O(2, 3) | N(5, 3) | F(0, 0) | S(1, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/50\n",
      "(6 ) COc1ccc(F)cc1NC(=O)N1CCO[C@H](c2ccc(C)o2)C1           | C(17, 16) | O(4, 2) | N(2, 3) | F(1, 0) | S(0, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/52\n",
      "(7 ) O=C(C/C(=N\\Nc1nc(-c2ccccc2)cs1)c1ccccc1)C(F)(F)F      | C(19, 19) | O(1, 2) | N(3, 2) | F(3, 0) | S(1, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/58\n",
      "\n",
      "Epoch 8 ------ average losses ------- | total: 0.37323 | boa: 1.10028 | edge: 0.33496 | kl: 0.00000 | \n",
      "\n",
      "(0 ) CC(C)(C)c1ccc2occ(CC(=O)Nc3ccccc3F)c2c1               | C(20, 18) | O(2, 2) | N(1, 3) | F(1, 0) | S(0, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/52\n",
      "(1 ) CSc1ccc(/C=c2\\sc3ncnn3c2=O)cc1                        | C(12, 18) | O(1, 2) | N(3, 3) | F(0, 0) | S(2, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/40\n",
      "(2 ) COc1cc(OC)cc([C@H]2CC[NH+](CCC(F)(F)F)C2)c1           | C(15, 16) | O(2, 2) | N(1, 4) | F(3, 0) | S(0, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/44\n",
      "(3 ) C[C@H]([NH2+]C1C[C@H](C)O[C@@H](C)C1)c1c[nH]c2cc(F... | C(17, 16) | O(1, 1) | N(2, 3) | F(1, 0) | S(0, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/46\n",
      "(4 ) CS[C@@H]1CC[C@H](NC(=O)CCC(=O)c2ccc(C)s2)C1           | C(15, 19) | O(2, 2) | N(1, 2) | F(0, 0) | S(2, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/42\n",
      "(5 ) CCCn1/c(=N/C(=O)[C@@H](CCSC)NC(N)=O)[nH]c2ccccc21     | C(16, 17) | O(2, 2) | N(5, 3) | F(0, 0) | S(1, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/50\n",
      "(6 ) COc1ccc(F)cc1NC(=O)N1CCO[C@H](c2ccc(C)o2)C1           | C(17, 16) | O(4, 1) | N(2, 1) | F(1, 0) | S(0, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/52\n",
      "(7 ) O=C(C/C(=N\\Nc1nc(-c2ccccc2)cs1)c1ccccc1)C(F)(F)F      | C(19, 17) | O(1, 2) | N(3, 2) | F(3, 0) | S(1, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/58\n",
      "\n",
      "Epoch 9 ------ average losses ------- | total: 0.37103 | boa: 1.07886 | edge: 0.33377 | kl: 0.00000 | \n",
      "\n",
      "(0 ) CC(C)(C)c1ccc2occ(CC(=O)Nc3ccccc3F)c2c1               | C(20, 18) | O(2, 1) | N(1, 3) | F(1, 0) | S(0, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/52\n",
      "(1 ) CSc1ccc(/C=c2\\sc3ncnn3c2=O)cc1                        | C(12, 18) | O(1, 2) | N(3, 3) | F(0, 0) | S(2, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/40\n",
      "(2 ) COc1cc(OC)cc([C@H]2CC[NH+](CCC(F)(F)F)C2)c1           | C(15, 16) | O(2, 1) | N(1, 4) | F(3, 0) | S(0, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/44\n",
      "(3 ) C[C@H]([NH2+]C1C[C@H](C)O[C@@H](C)C1)c1c[nH]c2cc(F... | C(17, 18) | O(1, 1) | N(2, 3) | F(1, 0) | S(0, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/46\n",
      "(4 ) CS[C@@H]1CC[C@H](NC(=O)CCC(=O)c2ccc(C)s2)C1           | C(15, 18) | O(2, 2) | N(1, 2) | F(0, 0) | S(2, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/42\n",
      "(5 ) CCCn1/c(=N/C(=O)[C@@H](CCSC)NC(N)=O)[nH]c2ccccc21     | C(16, 17) | O(2, 3) | N(5, 3) | F(0, 0) | S(1, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/50\n",
      "(6 ) COc1ccc(F)cc1NC(=O)N1CCO[C@H](c2ccc(C)o2)C1           | C(17, 18) | O(4, 1) | N(2, 1) | F(1, 0) | S(0, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/52\n",
      "(7 ) O=C(C/C(=N\\Nc1nc(-c2ccccc2)cs1)c1ccccc1)C(F)(F)F      | C(19, 18) | O(1, 2) | N(3, 2) | F(3, 0) | S(1, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/58\n",
      "\n",
      "Epoch 10 ------ average losses ------- | total: 0.36943 | boa: 1.06623 | edge: 0.33276 | kl: 0.00000 | \n",
      "\n",
      "(0 ) CC(C)(C)c1ccc2occ(CC(=O)Nc3ccccc3F)c2c1               | C(20, 18) | O(2, 1) | N(1, 3) | F(1, 0) | S(0, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/52\n",
      "(1 ) CSc1ccc(/C=c2\\sc3ncnn3c2=O)cc1                        | C(12, 18) | O(1, 2) | N(3, 3) | F(0, 0) | S(2, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/40\n",
      "(2 ) COc1cc(OC)cc([C@H]2CC[NH+](CCC(F)(F)F)C2)c1           | C(15, 16) | O(2, 1) | N(1, 4) | F(3, 0) | S(0, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/44\n",
      "(3 ) C[C@H]([NH2+]C1C[C@H](C)O[C@@H](C)C1)c1c[nH]c2cc(F... | C(17, 18) | O(1, 1) | N(2, 3) | F(1, 0) | S(0, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/46\n",
      "(4 ) CS[C@@H]1CC[C@H](NC(=O)CCC(=O)c2ccc(C)s2)C1           | C(15, 18) | O(2, 2) | N(1, 2) | F(0, 0) | S(2, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/42\n",
      "(5 ) CCCn1/c(=N/C(=O)[C@@H](CCSC)NC(N)=O)[nH]c2ccccc21     | C(16, 17) | O(2, 3) | N(5, 2) | F(0, 0) | S(1, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/50\n",
      "(6 ) COc1ccc(F)cc1NC(=O)N1CCO[C@H](c2ccc(C)o2)C1           | C(17, 16) | O(4, 1) | N(2, 3) | F(1, 0) | S(0, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/52\n",
      "(7 ) O=C(C/C(=N\\Nc1nc(-c2ccccc2)cs1)c1ccccc1)C(F)(F)F      | C(19, 17) | O(1, 2) | N(3, 2) | F(3, 0) | S(1, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/58\n",
      "\n",
      "Epoch 11 ------ average losses ------- | total: 0.36813 | boa: 1.05828 | edge: 0.33180 | kl: 0.00000 | \n",
      "\n",
      "(0 ) CC(C)(C)c1ccc2occ(CC(=O)Nc3ccccc3F)c2c1               | C(20, 18) | O(2, 1) | N(1, 3) | F(1, 0) | S(0, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/52\n",
      "(1 ) CSc1ccc(/C=c2\\sc3ncnn3c2=O)cc1                        | C(12, 18) | O(1, 2) | N(3, 3) | F(0, 0) | S(2, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/40\n",
      "(2 ) COc1cc(OC)cc([C@H]2CC[NH+](CCC(F)(F)F)C2)c1           | C(15, 16) | O(2, 1) | N(1, 4) | F(3, 0) | S(0, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/44\n",
      "(3 ) C[C@H]([NH2+]C1C[C@H](C)O[C@@H](C)C1)c1c[nH]c2cc(F... | C(17, 16) | O(1, 1) | N(2, 3) | F(1, 0) | S(0, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/46\n",
      "(4 ) CS[C@@H]1CC[C@H](NC(=O)CCC(=O)c2ccc(C)s2)C1           | C(15, 18) | O(2, 2) | N(1, 2) | F(0, 0) | S(2, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/42\n",
      "(5 ) CCCn1/c(=N/C(=O)[C@@H](CCSC)NC(N)=O)[nH]c2ccccc21     | C(16, 17) | O(2, 3) | N(5, 3) | F(0, 0) | S(1, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/50\n",
      "(6 ) COc1ccc(F)cc1NC(=O)N1CCO[C@H](c2ccc(C)o2)C1           | C(17, 16) | O(4, 1) | N(2, 3) | F(1, 0) | S(0, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/52\n",
      "(7 ) O=C(C/C(=N\\Nc1nc(-c2ccccc2)cs1)c1ccccc1)C(F)(F)F      | C(19, 17) | O(1, 2) | N(3, 2) | F(3, 0) | S(1, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/58\n",
      "\n",
      "Epoch 12 ------ average losses ------- | total: 0.36671 | boa: 1.05243 | edge: 0.33062 | kl: 0.00000 | \n",
      "\n",
      "(0 ) CC(C)(C)c1ccc2occ(CC(=O)Nc3ccccc3F)c2c1               | C(20, 18) | O(2, 2) | N(1, 3) | F(1, 0) | S(0, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/52\n",
      "(1 ) CSc1ccc(/C=c2\\sc3ncnn3c2=O)cc1                        | C(12, 18) | O(1, 2) | N(3, 3) | F(0, 0) | S(2, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/40\n",
      "(2 ) COc1cc(OC)cc([C@H]2CC[NH+](CCC(F)(F)F)C2)c1           | C(15, 16) | O(2, 1) | N(1, 4) | F(3, 0) | S(0, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/44\n",
      "(3 ) C[C@H]([NH2+]C1C[C@H](C)O[C@@H](C)C1)c1c[nH]c2cc(F... | C(17, 16) | O(1, 1) | N(2, 3) | F(1, 0) | S(0, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/46\n",
      "(4 ) CS[C@@H]1CC[C@H](NC(=O)CCC(=O)c2ccc(C)s2)C1           | C(15, 18) | O(2, 2) | N(1, 2) | F(0, 0) | S(2, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/42\n",
      "(5 ) CCCn1/c(=N/C(=O)[C@@H](CCSC)NC(N)=O)[nH]c2ccccc21     | C(16, 17) | O(2, 3) | N(5, 3) | F(0, 0) | S(1, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/50\n",
      "(6 ) COc1ccc(F)cc1NC(=O)N1CCO[C@H](c2ccc(C)o2)C1           | C(17, 19) | O(4, 1) | N(2, 1) | F(1, 0) | S(0, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/52\n",
      "(7 ) O=C(C/C(=N\\Nc1nc(-c2ccccc2)cs1)c1ccccc1)C(F)(F)F      | C(19, 17) | O(1, 2) | N(3, 2) | F(3, 0) | S(1, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/58\n",
      "\n",
      "Epoch 13 ------ average losses ------- | total: 0.36537 | boa: 1.04751 | edge: 0.32946 | kl: 0.00000 | \n",
      "\n",
      "(0 ) CC(C)(C)c1ccc2occ(CC(=O)Nc3ccccc3F)c2c1               | C(20, 18) | O(2, 2) | N(1, 3) | F(1, 0) | S(0, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/52\n",
      "(1 ) CSc1ccc(/C=c2\\sc3ncnn3c2=O)cc1                        | C(12, 18) | O(1, 2) | N(3, 3) | F(0, 0) | S(2, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/40\n",
      "(2 ) COc1cc(OC)cc([C@H]2CC[NH+](CCC(F)(F)F)C2)c1           | C(15, 16) | O(2, 1) | N(1, 4) | F(3, 0) | S(0, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/44\n",
      "(3 ) C[C@H]([NH2+]C1C[C@H](C)O[C@@H](C)C1)c1c[nH]c2cc(F... | C(17, 16) | O(1, 1) | N(2, 3) | F(1, 0) | S(0, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/46\n",
      "(4 ) CS[C@@H]1CC[C@H](NC(=O)CCC(=O)c2ccc(C)s2)C1           | C(15, 18) | O(2, 2) | N(1, 2) | F(0, 0) | S(2, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/42\n",
      "(5 ) CCCn1/c(=N/C(=O)[C@@H](CCSC)NC(N)=O)[nH]c2ccccc21     | C(16, 17) | O(2, 3) | N(5, 3) | F(0, 0) | S(1, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/50\n",
      "(6 ) COc1ccc(F)cc1NC(=O)N1CCO[C@H](c2ccc(C)o2)C1           | C(17, 16) | O(4, 1) | N(2, 1) | F(1, 0) | S(0, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/52\n",
      "(7 ) O=C(C/C(=N\\Nc1nc(-c2ccccc2)cs1)c1ccccc1)C(F)(F)F      | C(19, 17) | O(1, 2) | N(3, 2) | F(3, 0) | S(1, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/58\n",
      "\n",
      "Epoch 14 ------ average losses ------- | total: 0.36414 | boa: 1.04349 | edge: 0.32839 | kl: 0.00000 | \n",
      "\n",
      "(0 ) CC(C)(C)c1ccc2occ(CC(=O)Nc3ccccc3F)c2c1               | C(20, 18) | O(2, 2) | N(1, 3) | F(1, 0) | S(0, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/52\n",
      "(1 ) CSc1ccc(/C=c2\\sc3ncnn3c2=O)cc1                        | C(12, 18) | O(1, 2) | N(3, 3) | F(0, 0) | S(2, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/40\n",
      "(2 ) COc1cc(OC)cc([C@H]2CC[NH+](CCC(F)(F)F)C2)c1           | C(15, 16) | O(2, 2) | N(1, 4) | F(3, 0) | S(0, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/44\n",
      "(3 ) C[C@H]([NH2+]C1C[C@H](C)O[C@@H](C)C1)c1c[nH]c2cc(F... | C(17, 16) | O(1, 1) | N(2, 3) | F(1, 0) | S(0, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/46\n",
      "(4 ) CS[C@@H]1CC[C@H](NC(=O)CCC(=O)c2ccc(C)s2)C1           | C(15, 18) | O(2, 2) | N(1, 2) | F(0, 0) | S(2, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/42\n",
      "(5 ) CCCn1/c(=N/C(=O)[C@@H](CCSC)NC(N)=O)[nH]c2ccccc21     | C(16, 18) | O(2, 3) | N(5, 3) | F(0, 0) | S(1, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/50\n",
      "(6 ) COc1ccc(F)cc1NC(=O)N1CCO[C@H](c2ccc(C)o2)C1           | C(17, 16) | O(4, 1) | N(2, 1) | F(1, 0) | S(0, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/52\n",
      "(7 ) O=C(C/C(=N\\Nc1nc(-c2ccccc2)cs1)c1ccccc1)C(F)(F)F      | C(19, 17) | O(1, 2) | N(3, 2) | F(3, 0) | S(1, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/58\n",
      "\n",
      "Epoch 15 ------ average losses ------- | total: 0.36298 | boa: 1.03997 | edge: 0.32735 | kl: 0.00000 | \n",
      "\n",
      "(0 ) CC(C)(C)c1ccc2occ(CC(=O)Nc3ccccc3F)c2c1               | C(20, 18) | O(2, 2) | N(1, 3) | F(1, 0) | S(0, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/52\n",
      "(1 ) CSc1ccc(/C=c2\\sc3ncnn3c2=O)cc1                        | C(12, 18) | O(1, 2) | N(3, 3) | F(0, 0) | S(2, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/40\n",
      "(2 ) COc1cc(OC)cc([C@H]2CC[NH+](CCC(F)(F)F)C2)c1           | C(15, 16) | O(2, 2) | N(1, 4) | F(3, 0) | S(0, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/44\n",
      "(3 ) C[C@H]([NH2+]C1C[C@H](C)O[C@@H](C)C1)c1c[nH]c2cc(F... | C(17, 16) | O(1, 1) | N(2, 3) | F(1, 0) | S(0, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/46\n",
      "(4 ) CS[C@@H]1CC[C@H](NC(=O)CCC(=O)c2ccc(C)s2)C1           | C(15, 18) | O(2, 2) | N(1, 2) | F(0, 0) | S(2, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/42\n",
      "(5 ) CCCn1/c(=N/C(=O)[C@@H](CCSC)NC(N)=O)[nH]c2ccccc21     | C(16, 18) | O(2, 3) | N(5, 3) | F(0, 0) | S(1, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/50\n",
      "(6 ) COc1ccc(F)cc1NC(=O)N1CCO[C@H](c2ccc(C)o2)C1           | C(17, 16) | O(4, 1) | N(2, 1) | F(1, 0) | S(0, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/52\n",
      "(7 ) O=C(C/C(=N\\Nc1nc(-c2ccccc2)cs1)c1ccccc1)C(F)(F)F      | C(19, 17) | O(1, 2) | N(3, 2) | F(3, 0) | S(1, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/58\n",
      "\n",
      "Epoch 16 ------ average losses ------- | total: 0.36186 | boa: 1.03654 | edge: 0.32635 | kl: 0.00000 | \n",
      "\n",
      "(0 ) CC(C)(C)c1ccc2occ(CC(=O)Nc3ccccc3F)c2c1               | C(20, 18) | O(2, 2) | N(1, 3) | F(1, 0) | S(0, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/52\n",
      "(1 ) CSc1ccc(/C=c2\\sc3ncnn3c2=O)cc1                        | C(12, 18) | O(1, 2) | N(3, 3) | F(0, 0) | S(2, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/40\n",
      "(2 ) COc1cc(OC)cc([C@H]2CC[NH+](CCC(F)(F)F)C2)c1           | C(15, 16) | O(2, 2) | N(1, 4) | F(3, 0) | S(0, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/44\n",
      "(3 ) C[C@H]([NH2+]C1C[C@H](C)O[C@@H](C)C1)c1c[nH]c2cc(F... | C(17, 16) | O(1, 1) | N(2, 3) | F(1, 0) | S(0, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/46\n",
      "(4 ) CS[C@@H]1CC[C@H](NC(=O)CCC(=O)c2ccc(C)s2)C1           | C(15, 18) | O(2, 2) | N(1, 2) | F(0, 0) | S(2, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/42\n",
      "(5 ) CCCn1/c(=N/C(=O)[C@@H](CCSC)NC(N)=O)[nH]c2ccccc21     | C(16, 18) | O(2, 3) | N(5, 3) | F(0, 0) | S(1, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/50\n",
      "(6 ) COc1ccc(F)cc1NC(=O)N1CCO[C@H](c2ccc(C)o2)C1           | C(17, 16) | O(4, 1) | N(2, 4) | F(1, 0) | S(0, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/52\n",
      "(7 ) O=C(C/C(=N\\Nc1nc(-c2ccccc2)cs1)c1ccccc1)C(F)(F)F      | C(19, 17) | O(1, 2) | N(3, 2) | F(3, 0) | S(1, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/58\n",
      "\n",
      "Epoch 17 ------ average losses ------- | total: 0.36076 | boa: 1.03336 | edge: 0.32536 | kl: 0.00000 | \n",
      "\n",
      "(0 ) CC(C)(C)c1ccc2occ(CC(=O)Nc3ccccc3F)c2c1               | C(20, 18) | O(2, 2) | N(1, 3) | F(1, 0) | S(0, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/52\n",
      "(1 ) CSc1ccc(/C=c2\\sc3ncnn3c2=O)cc1                        | C(12, 18) | O(1, 2) | N(3, 3) | F(0, 0) | S(2, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/40\n",
      "(2 ) COc1cc(OC)cc([C@H]2CC[NH+](CCC(F)(F)F)C2)c1           | C(15, 16) | O(2, 2) | N(1, 4) | F(3, 0) | S(0, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/44\n",
      "(3 ) C[C@H]([NH2+]C1C[C@H](C)O[C@@H](C)C1)c1c[nH]c2cc(F... | C(17, 16) | O(1, 1) | N(2, 3) | F(1, 0) | S(0, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/46\n",
      "(4 ) CS[C@@H]1CC[C@H](NC(=O)CCC(=O)c2ccc(C)s2)C1           | C(15, 18) | O(2, 2) | N(1, 2) | F(0, 0) | S(2, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/42\n",
      "(5 ) CCCn1/c(=N/C(=O)[C@@H](CCSC)NC(N)=O)[nH]c2ccccc21     | C(16, 18) | O(2, 3) | N(5, 3) | F(0, 0) | S(1, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/50\n",
      "(6 ) COc1ccc(F)cc1NC(=O)N1CCO[C@H](c2ccc(C)o2)C1           | C(17, 19) | O(4, 1) | N(2, 4) | F(1, 0) | S(0, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/52\n",
      "(7 ) O=C(C/C(=N\\Nc1nc(-c2ccccc2)cs1)c1ccccc1)C(F)(F)F      | C(19, 17) | O(1, 2) | N(3, 2) | F(3, 0) | S(1, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | "
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Exception ignored in: <bound method IPythonKernel._clean_thread_parent_frames of <ipykernel.ipkernel.IPythonKernel object at 0x708df4891fc0>>\n",
      "Traceback (most recent call last):\n",
      "  File \"/home/justin/files/projects/mole_gen/venv/lib/python3.10/site-packages/ipykernel/ipkernel.py\", line 781, in _clean_thread_parent_frames\n",
      "    def _clean_thread_parent_frames(\n",
      "KeyboardInterrupt: \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "edges 0/58\n",
      "\n",
      "Epoch 18 ------ average losses ------- | total: 0.35968 | boa: 1.03027 | edge: 0.32439 | kl: 0.00000 | \n",
      "\n",
      "(0 ) CC(C)(C)c1ccc2occ(CC(=O)Nc3ccccc3F)c2c1               | C(20, 18) | O(2, 2) | N(1, 3) | F(1, 0) | S(0, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/52\n",
      "(1 ) CSc1ccc(/C=c2\\sc3ncnn3c2=O)cc1                        | C(12, 18) | O(1, 2) | N(3, 3) | F(0, 0) | S(2, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/40\n",
      "(2 ) COc1cc(OC)cc([C@H]2CC[NH+](CCC(F)(F)F)C2)c1           | C(15, 16) | O(2, 2) | N(1, 4) | F(3, 0) | S(0, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/44\n",
      "(3 ) C[C@H]([NH2+]C1C[C@H](C)O[C@@H](C)C1)c1c[nH]c2cc(F... | C(17, 16) | O(1, 1) | N(2, 3) | F(1, 0) | S(0, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/46\n",
      "(4 ) CS[C@@H]1CC[C@H](NC(=O)CCC(=O)c2ccc(C)s2)C1           | C(15, 18) | O(2, 2) | N(1, 2) | F(0, 0) | S(2, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/42\n",
      "(5 ) CCCn1/c(=N/C(=O)[C@@H](CCSC)NC(N)=O)[nH]c2ccccc21     | C(16, 18) | O(2, 3) | N(5, 3) | F(0, 0) | S(1, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/50\n",
      "(6 ) COc1ccc(F)cc1NC(=O)N1CCO[C@H](c2ccc(C)o2)C1           | C(17, 16) | O(4, 2) | N(2, 4) | F(1, 0) | S(0, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/52\n",
      "(7 ) O=C(C/C(=N\\Nc1nc(-c2ccccc2)cs1)c1ccccc1)C(F)(F)F      | C(19, 17) | O(1, 2) | N(3, 2) | F(3, 0) | S(1, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/58\n",
      "\n",
      "Epoch 19 ------ average losses ------- | total: 0.35866 | boa: 1.02728 | edge: 0.32347 | kl: 0.00000 | \n",
      "\n",
      "(0 ) CC(C)(C)c1ccc2occ(CC(=O)Nc3ccccc3F)c2c1               | C(20, 18) | O(2, 2) | N(1, 3) | F(1, 0) | S(0, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/52\n",
      "(1 ) CSc1ccc(/C=c2\\sc3ncnn3c2=O)cc1                        | C(12, 18) | O(1, 2) | N(3, 3) | F(0, 0) | S(2, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/40\n",
      "(2 ) COc1cc(OC)cc([C@H]2CC[NH+](CCC(F)(F)F)C2)c1           | C(15, 16) | O(2, 2) | N(1, 4) | F(3, 0) | S(0, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/44\n",
      "(3 ) C[C@H]([NH2+]C1C[C@H](C)O[C@@H](C)C1)c1c[nH]c2cc(F... | C(17, 16) | O(1, 1) | N(2, 3) | F(1, 0) | S(0, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/46\n",
      "(4 ) CS[C@@H]1CC[C@H](NC(=O)CCC(=O)c2ccc(C)s2)C1           | C(15, 18) | O(2, 2) | N(1, 2) | F(0, 0) | S(2, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/42\n",
      "(5 ) CCCn1/c(=N/C(=O)[C@@H](CCSC)NC(N)=O)[nH]c2ccccc21     | C(16, 18) | O(2, 3) | N(5, 3) | F(0, 0) | S(1, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/50\n",
      "(6 ) COc1ccc(F)cc1NC(=O)N1CCO[C@H](c2ccc(C)o2)C1           | C(17, 19) | O(4, 2) | N(2, 4) | F(1, 0) | S(0, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/52\n",
      "(7 ) O=C(C/C(=N\\Nc1nc(-c2ccccc2)cs1)c1ccccc1)C(F)(F)F      | C(19, 17) | O(1, 2) | N(3, 2) | F(3, 0) | S(1, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/58\n",
      "\n",
      "Epoch 20 ------ average losses ------- | total: 0.35765 | boa: 1.02426 | edge: 0.32257 | kl: 0.00000 | \n",
      "\n",
      "(0 ) CC(C)(C)c1ccc2occ(CC(=O)Nc3ccccc3F)c2c1               | C(20, 18) | O(2, 2) | N(1, 3) | F(1, 0) | S(0, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/52\n",
      "(1 ) CSc1ccc(/C=c2\\sc3ncnn3c2=O)cc1                        | C(12, 18) | O(1, 2) | N(3, 3) | F(0, 0) | S(2, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/40\n",
      "(2 ) COc1cc(OC)cc([C@H]2CC[NH+](CCC(F)(F)F)C2)c1           | C(15, 16) | O(2, 2) | N(1, 2) | F(3, 0) | S(0, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/44\n",
      "(3 ) C[C@H]([NH2+]C1C[C@H](C)O[C@@H](C)C1)c1c[nH]c2cc(F... | C(17, 16) | O(1, 1) | N(2, 3) | F(1, 0) | S(0, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/46\n",
      "(4 ) CS[C@@H]1CC[C@H](NC(=O)CCC(=O)c2ccc(C)s2)C1           | C(15, 18) | O(2, 2) | N(1, 2) | F(0, 0) | S(2, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/42\n",
      "(5 ) CCCn1/c(=N/C(=O)[C@@H](CCSC)NC(N)=O)[nH]c2ccccc21     | C(16, 18) | O(2, 3) | N(5, 3) | F(0, 0) | S(1, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/50\n",
      "(6 ) COc1ccc(F)cc1NC(=O)N1CCO[C@H](c2ccc(C)o2)C1           | C(17, 19) | O(4, 2) | N(2, 4) | F(1, 0) | S(0, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/52\n",
      "(7 ) O=C(C/C(=N\\Nc1nc(-c2ccccc2)cs1)c1ccccc1)C(F)(F)F      | C(19, 17) | O(1, 2) | N(3, 2) | F(3, 0) | S(1, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/58\n",
      "\n",
      "Epoch 21 ------ average losses ------- | total: 0.35666 | boa: 1.02121 | edge: 0.32168 | kl: 0.00000 | \n",
      "\n",
      "(0 ) CC(C)(C)c1ccc2occ(CC(=O)Nc3ccccc3F)c2c1               | C(20, 18) | O(2, 2) | N(1, 3) | F(1, 0) | S(0, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/52\n",
      "(1 ) CSc1ccc(/C=c2\\sc3ncnn3c2=O)cc1                        | C(12, 18) | O(1, 2) | N(3, 3) | F(0, 0) | S(2, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/40\n",
      "(2 ) COc1cc(OC)cc([C@H]2CC[NH+](CCC(F)(F)F)C2)c1           | C(15, 16) | O(2, 2) | N(1, 2) | F(3, 0) | S(0, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/44\n",
      "(3 ) C[C@H]([NH2+]C1C[C@H](C)O[C@@H](C)C1)c1c[nH]c2cc(F... | C(17, 16) | O(1, 1) | N(2, 3) | F(1, 0) | S(0, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/46\n",
      "(4 ) CS[C@@H]1CC[C@H](NC(=O)CCC(=O)c2ccc(C)s2)C1           | C(15, 18) | O(2, 2) | N(1, 2) | F(0, 0) | S(2, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/42\n",
      "(5 ) CCCn1/c(=N/C(=O)[C@@H](CCSC)NC(N)=O)[nH]c2ccccc21     | C(16, 18) | O(2, 3) | N(5, 3) | F(0, 0) | S(1, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/50\n",
      "(6 ) COc1ccc(F)cc1NC(=O)N1CCO[C@H](c2ccc(C)o2)C1           | C(17, 19) | O(4, 2) | N(2, 4) | F(1, 0) | S(0, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/52\n",
      "(7 ) O=C(C/C(=N\\Nc1nc(-c2ccccc2)cs1)c1ccccc1)C(F)(F)F      | C(19, 17) | O(1, 2) | N(3, 2) | F(3, 0) | S(1, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/58\n",
      "\n",
      "Epoch 22 ------ average losses ------- | total: 0.35564 | boa: 1.01817 | edge: 0.32077 | kl: 0.00000 | \n",
      "\n",
      "(0 ) CC(C)(C)c1ccc2occ(CC(=O)Nc3ccccc3F)c2c1               | C(20, 18) | O(2, 2) | N(1, 3) | F(1, 0) | S(0, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/52\n",
      "(1 ) CSc1ccc(/C=c2\\sc3ncnn3c2=O)cc1                        | C(12, 18) | O(1, 1) | N(3, 3) | F(0, 0) | S(2, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/40\n",
      "(2 ) COc1cc(OC)cc([C@H]2CC[NH+](CCC(F)(F)F)C2)c1           | C(15, 16) | O(2, 2) | N(1, 2) | F(3, 0) | S(0, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/44\n",
      "(3 ) C[C@H]([NH2+]C1C[C@H](C)O[C@@H](C)C1)c1c[nH]c2cc(F... | C(17, 16) | O(1, 1) | N(2, 3) | F(1, 0) | S(0, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/46\n",
      "(4 ) CS[C@@H]1CC[C@H](NC(=O)CCC(=O)c2ccc(C)s2)C1           | C(15, 18) | O(2, 2) | N(1, 2) | F(0, 0) | S(2, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/42\n",
      "(5 ) CCCn1/c(=N/C(=O)[C@@H](CCSC)NC(N)=O)[nH]c2ccccc21     | C(16, 18) | O(2, 3) | N(5, 3) | F(0, 0) | S(1, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/50\n",
      "(6 ) COc1ccc(F)cc1NC(=O)N1CCO[C@H](c2ccc(C)o2)C1           | C(17, 16) | O(4, 2) | N(2, 4) | F(1, 0) | S(0, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/52\n",
      "(7 ) O=C(C/C(=N\\Nc1nc(-c2ccccc2)cs1)c1ccccc1)C(F)(F)F      | C(19, 17) | O(1, 2) | N(3, 2) | F(3, 0) | S(1, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/58\n",
      "\n",
      "Epoch 23 ------ average losses ------- | total: 0.35464 | boa: 1.01511 | edge: 0.31987 | kl: 0.00000 | \n",
      "\n",
      "(0 ) CC(C)(C)c1ccc2occ(CC(=O)Nc3ccccc3F)c2c1               | C(20, 18) | O(2, 2) | N(1, 3) | F(1, 0) | S(0, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/52\n",
      "(1 ) CSc1ccc(/C=c2\\sc3ncnn3c2=O)cc1                        | C(12, 18) | O(1, 1) | N(3, 3) | F(0, 0) | S(2, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/40\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Exception ignored in: <bound method IPythonKernel._clean_thread_parent_frames of <ipykernel.ipkernel.IPythonKernel object at 0x708df4891fc0>>\n",
      "Traceback (most recent call last):\n",
      "  File \"/home/justin/files/projects/mole_gen/venv/lib/python3.10/site-packages/ipykernel/ipkernel.py\", line 781, in _clean_thread_parent_frames\n",
      "    def _clean_thread_parent_frames(\n",
      "KeyboardInterrupt: \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(2 ) COc1cc(OC)cc([C@H]2CC[NH+](CCC(F)(F)F)C2)c1           | C(15, 16) | O(2, 2) | N(1, 2) | F(3, 0) | S(0, 0) | Cl(0, 0) | Br(0, 0) | I(0, 0) | edges 0/44\n"
     ]
    },
    {
     "ename": "KeyboardInterrupt",
     "evalue": "",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mKeyboardInterrupt\u001b[0m                         Traceback (most recent call last)",
      "Cell \u001b[0;32mIn[41], line 8\u001b[0m\n\u001b[1;32m      5\u001b[0m avg_loss \u001b[38;5;241m=\u001b[39m defaultdict(\u001b[38;5;28mfloat\u001b[39m)\n\u001b[1;32m      7\u001b[0m \u001b[38;5;28;01mfor\u001b[39;00m idx, batch \u001b[38;5;129;01min\u001b[39;00m \u001b[38;5;28menumerate\u001b[39m(loader):\n\u001b[0;32m----> 8\u001b[0m     boa, z, s, kl \u001b[38;5;241m=\u001b[39m \u001b[43mmodel\u001b[49m\u001b[43m(\u001b[49m\u001b[43mbatch\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m     10\u001b[0m     \u001b[38;5;28;01mif\u001b[39;00m DEBUG:   \n\u001b[1;32m     11\u001b[0m         debug_fn(batch, boa, z, s, MAX_SMILES_STRING) \n",
      "File \u001b[0;32m~/files/projects/mole_gen/venv/lib/python3.10/site-packages/torch/nn/modules/module.py:1751\u001b[0m, in \u001b[0;36mModule._wrapped_call_impl\u001b[0;34m(self, *args, **kwargs)\u001b[0m\n\u001b[1;32m   1749\u001b[0m     \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_compiled_call_impl(\u001b[38;5;241m*\u001b[39margs, \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mkwargs)  \u001b[38;5;66;03m# type: ignore[misc]\u001b[39;00m\n\u001b[1;32m   1750\u001b[0m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[0;32m-> 1751\u001b[0m     \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_call_impl\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43margs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n",
      "File \u001b[0;32m~/files/projects/mole_gen/venv/lib/python3.10/site-packages/torch/nn/modules/module.py:1762\u001b[0m, in \u001b[0;36mModule._call_impl\u001b[0;34m(self, *args, **kwargs)\u001b[0m\n\u001b[1;32m   1757\u001b[0m \u001b[38;5;66;03m# If we don't have any hooks, we want to skip the rest of the logic in\u001b[39;00m\n\u001b[1;32m   1758\u001b[0m \u001b[38;5;66;03m# this function, and just call forward.\u001b[39;00m\n\u001b[1;32m   1759\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m (\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_backward_hooks \u001b[38;5;129;01mor\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_backward_pre_hooks \u001b[38;5;129;01mor\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_forward_hooks \u001b[38;5;129;01mor\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_forward_pre_hooks\n\u001b[1;32m   1760\u001b[0m         \u001b[38;5;129;01mor\u001b[39;00m _global_backward_pre_hooks \u001b[38;5;129;01mor\u001b[39;00m _global_backward_hooks\n\u001b[1;32m   1761\u001b[0m         \u001b[38;5;129;01mor\u001b[39;00m _global_forward_hooks \u001b[38;5;129;01mor\u001b[39;00m _global_forward_pre_hooks):\n\u001b[0;32m-> 1762\u001b[0m     \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[43mforward_call\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43margs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m   1764\u001b[0m result \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;01mNone\u001b[39;00m\n\u001b[1;32m   1765\u001b[0m called_always_called_hooks \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mset\u001b[39m()\n",
      "Cell \u001b[0;32mIn[35], line 322\u001b[0m, in \u001b[0;36mMoleGen.forward\u001b[0;34m(self, input_data)\u001b[0m\n\u001b[1;32m    320\u001b[0m \u001b[38;5;28;01mfor\u001b[39;00m i \u001b[38;5;129;01min\u001b[39;00m \u001b[38;5;28mrange\u001b[39m(\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mnum_layers):\n\u001b[1;32m    321\u001b[0m     data \u001b[38;5;241m=\u001b[39m Data(x\u001b[38;5;241m=\u001b[39mh, edge_index\u001b[38;5;241m=\u001b[39minput_data\u001b[38;5;241m.\u001b[39mfc_edge_index, edge_attr\u001b[38;5;241m=\u001b[39me)\n\u001b[0;32m--> 322\u001b[0m     h \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43matom_decoder\u001b[49m\u001b[43m[\u001b[49m\u001b[43mi\u001b[49m\u001b[43m]\u001b[49m\u001b[43m(\u001b[49m\u001b[43mdata\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m    323\u001b[0m     e \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mbond_decoder[i](data)  \n\u001b[1;32m    325\u001b[0m \u001b[38;5;66;03m# now take each edge and apply MLP to predict bond type for each\u001b[39;00m\n",
      "File \u001b[0;32m~/files/projects/mole_gen/venv/lib/python3.10/site-packages/torch/nn/modules/module.py:1751\u001b[0m, in \u001b[0;36mModule._wrapped_call_impl\u001b[0;34m(self, *args, **kwargs)\u001b[0m\n\u001b[1;32m   1749\u001b[0m     \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_compiled_call_impl(\u001b[38;5;241m*\u001b[39margs, \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mkwargs)  \u001b[38;5;66;03m# type: ignore[misc]\u001b[39;00m\n\u001b[1;32m   1750\u001b[0m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[0;32m-> 1751\u001b[0m     \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_call_impl\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43margs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n",
      "File \u001b[0;32m~/files/projects/mole_gen/venv/lib/python3.10/site-packages/torch/nn/modules/module.py:1762\u001b[0m, in \u001b[0;36mModule._call_impl\u001b[0;34m(self, *args, **kwargs)\u001b[0m\n\u001b[1;32m   1757\u001b[0m \u001b[38;5;66;03m# If we don't have any hooks, we want to skip the rest of the logic in\u001b[39;00m\n\u001b[1;32m   1758\u001b[0m \u001b[38;5;66;03m# this function, and just call forward.\u001b[39;00m\n\u001b[1;32m   1759\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m (\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_backward_hooks \u001b[38;5;129;01mor\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_backward_pre_hooks \u001b[38;5;129;01mor\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_forward_hooks \u001b[38;5;129;01mor\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_forward_pre_hooks\n\u001b[1;32m   1760\u001b[0m         \u001b[38;5;129;01mor\u001b[39;00m _global_backward_pre_hooks \u001b[38;5;129;01mor\u001b[39;00m _global_backward_hooks\n\u001b[1;32m   1761\u001b[0m         \u001b[38;5;129;01mor\u001b[39;00m _global_forward_hooks \u001b[38;5;129;01mor\u001b[39;00m _global_forward_pre_hooks):\n\u001b[0;32m-> 1762\u001b[0m     \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[43mforward_call\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43margs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m   1764\u001b[0m result \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;01mNone\u001b[39;00m\n\u001b[1;32m   1765\u001b[0m called_always_called_hooks \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mset\u001b[39m()\n",
      "Cell \u001b[0;32mIn[35], line 128\u001b[0m, in \u001b[0;36mAtomGCNLayer.forward\u001b[0;34m(self, data)\u001b[0m\n\u001b[1;32m    114\u001b[0m \u001b[38;5;250m\u001b[39m\u001b[38;5;124;03m\"\"\"Perform forward pass for atom GCN\u001b[39;00m\n\u001b[1;32m    115\u001b[0m \u001b[38;5;124;03m\u001b[39;00m\n\u001b[1;32m    116\u001b[0m \u001b[38;5;124;03mArgs:\u001b[39;00m\n\u001b[0;32m   (...)\u001b[0m\n\u001b[1;32m    124\u001b[0m \u001b[38;5;124;03m    \u001b[39;00m\n\u001b[1;32m    125\u001b[0m \u001b[38;5;124;03m\"\"\"\u001b[39;00m \n\u001b[1;32m    126\u001b[0m x, edge_index, edge_attr \u001b[38;5;241m=\u001b[39m data\u001b[38;5;241m.\u001b[39mx, data\u001b[38;5;241m.\u001b[39medge_index, data\u001b[38;5;241m.\u001b[39medge_attr\n\u001b[0;32m--> 128\u001b[0m h \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mgcn\u001b[49m\u001b[43m(\u001b[49m\u001b[43mx\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43medge_index\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43medge_attr\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m    129\u001b[0m h \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mbn(h)\n\u001b[1;32m    130\u001b[0m h \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mrelu(h)\n",
      "File \u001b[0;32m~/files/projects/mole_gen/venv/lib/python3.10/site-packages/torch/nn/modules/module.py:1751\u001b[0m, in \u001b[0;36mModule._wrapped_call_impl\u001b[0;34m(self, *args, **kwargs)\u001b[0m\n\u001b[1;32m   1749\u001b[0m     \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_compiled_call_impl(\u001b[38;5;241m*\u001b[39margs, \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mkwargs)  \u001b[38;5;66;03m# type: ignore[misc]\u001b[39;00m\n\u001b[1;32m   1750\u001b[0m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[0;32m-> 1751\u001b[0m     \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_call_impl\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43margs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n",
      "File \u001b[0;32m~/files/projects/mole_gen/venv/lib/python3.10/site-packages/torch/nn/modules/module.py:1762\u001b[0m, in \u001b[0;36mModule._call_impl\u001b[0;34m(self, *args, **kwargs)\u001b[0m\n\u001b[1;32m   1757\u001b[0m \u001b[38;5;66;03m# If we don't have any hooks, we want to skip the rest of the logic in\u001b[39;00m\n\u001b[1;32m   1758\u001b[0m \u001b[38;5;66;03m# this function, and just call forward.\u001b[39;00m\n\u001b[1;32m   1759\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m (\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_backward_hooks \u001b[38;5;129;01mor\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_backward_pre_hooks \u001b[38;5;129;01mor\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_forward_hooks \u001b[38;5;129;01mor\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_forward_pre_hooks\n\u001b[1;32m   1760\u001b[0m         \u001b[38;5;129;01mor\u001b[39;00m _global_backward_pre_hooks \u001b[38;5;129;01mor\u001b[39;00m _global_backward_hooks\n\u001b[1;32m   1761\u001b[0m         \u001b[38;5;129;01mor\u001b[39;00m _global_forward_hooks \u001b[38;5;129;01mor\u001b[39;00m _global_forward_pre_hooks):\n\u001b[0;32m-> 1762\u001b[0m     \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[43mforward_call\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43margs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m   1764\u001b[0m result \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;01mNone\u001b[39;00m\n\u001b[1;32m   1765\u001b[0m called_always_called_hooks \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mset\u001b[39m()\n",
      "File \u001b[0;32m~/files/projects/mole_gen/venv/lib/python3.10/site-packages/torch_geometric/nn/conv/res_gated_graph_conv.py:128\u001b[0m, in \u001b[0;36mResGatedGraphConv.forward\u001b[0;34m(self, x, edge_index, edge_attr)\u001b[0m\n\u001b[1;32m    124\u001b[0m     k, q, v \u001b[38;5;241m=\u001b[39m x[\u001b[38;5;241m1\u001b[39m], x[\u001b[38;5;241m0\u001b[39m], x[\u001b[38;5;241m0\u001b[39m]\n\u001b[1;32m    126\u001b[0m \u001b[38;5;66;03m# propagate_type: (k: Tensor, q: Tensor, v: Tensor,\u001b[39;00m\n\u001b[1;32m    127\u001b[0m \u001b[38;5;66;03m#                  edge_attr: OptTensor)\u001b[39;00m\n\u001b[0;32m--> 128\u001b[0m out \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mpropagate\u001b[49m\u001b[43m(\u001b[49m\u001b[43medge_index\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mk\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mk\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mq\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mq\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mv\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mv\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43medge_attr\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43medge_attr\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m    130\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mroot_weight:\n\u001b[1;32m    131\u001b[0m     out \u001b[38;5;241m=\u001b[39m out \u001b[38;5;241m+\u001b[39m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mlin_skip(x[\u001b[38;5;241m1\u001b[39m])\n",
      "File \u001b[0;32m/tmp/torch_geometric.nn.conv.res_gated_graph_conv_ResGatedGraphConv_propagate_z_yik5tg.py:231\u001b[0m, in \u001b[0;36mpropagate\u001b[0;34m(self, edge_index, k, q, v, edge_attr, size)\u001b[0m\n\u001b[1;32m    220\u001b[0m             kwargs \u001b[38;5;241m=\u001b[39m CollectArgs(\n\u001b[1;32m    221\u001b[0m                 k_i\u001b[38;5;241m=\u001b[39mhook_kwargs[\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mk_i\u001b[39m\u001b[38;5;124m'\u001b[39m],\n\u001b[1;32m    222\u001b[0m                 q_j\u001b[38;5;241m=\u001b[39mhook_kwargs[\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mq_j\u001b[39m\u001b[38;5;124m'\u001b[39m],\n\u001b[0;32m   (...)\u001b[0m\n\u001b[1;32m    227\u001b[0m                 dim_size\u001b[38;5;241m=\u001b[39mkwargs\u001b[38;5;241m.\u001b[39mdim_size,\n\u001b[1;32m    228\u001b[0m             )\n\u001b[1;32m    229\u001b[0m \u001b[38;5;66;03m# End Message Forward Pre Hook #########################################\u001b[39;00m\n\u001b[0;32m--> 231\u001b[0m out \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mmessage\u001b[49m\u001b[43m(\u001b[49m\n\u001b[1;32m    232\u001b[0m \u001b[43m    \u001b[49m\u001b[43mk_i\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mkwargs\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mk_i\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    233\u001b[0m \u001b[43m    \u001b[49m\u001b[43mq_j\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mkwargs\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mq_j\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    234\u001b[0m \u001b[43m    \u001b[49m\u001b[43mv_j\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mkwargs\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mv_j\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    235\u001b[0m \u001b[43m    \u001b[49m\u001b[43medge_attr\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mkwargs\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43medge_attr\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    236\u001b[0m \u001b[43m\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m    238\u001b[0m \u001b[38;5;66;03m# Begin Message Forward Hook ###########################################\u001b[39;00m\n\u001b[1;32m    239\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m torch\u001b[38;5;241m.\u001b[39mjit\u001b[38;5;241m.\u001b[39mis_scripting() \u001b[38;5;129;01mand\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m is_compiling():\n",
      "File \u001b[0;32m~/files/projects/mole_gen/venv/lib/python3.10/site-packages/torch_geometric/nn/conv/res_gated_graph_conv.py:148\u001b[0m, in \u001b[0;36mResGatedGraphConv.message\u001b[0;34m(self, k_i, q_j, v_j, edge_attr)\u001b[0m\n\u001b[1;32m    145\u001b[0m     q_j \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mlin_query(torch\u001b[38;5;241m.\u001b[39mcat([q_j, edge_attr], dim\u001b[38;5;241m=\u001b[39m\u001b[38;5;241m-\u001b[39m\u001b[38;5;241m1\u001b[39m))\n\u001b[1;32m    146\u001b[0m     v_j \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mlin_value(torch\u001b[38;5;241m.\u001b[39mcat([v_j, edge_attr], dim\u001b[38;5;241m=\u001b[39m\u001b[38;5;241m-\u001b[39m\u001b[38;5;241m1\u001b[39m))\n\u001b[0;32m--> 148\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mact\u001b[49m\u001b[43m(\u001b[49m\u001b[43mk_i\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m+\u001b[39;49m\u001b[43m \u001b[49m\u001b[43mq_j\u001b[49m\u001b[43m)\u001b[49m \u001b[38;5;241m*\u001b[39m v_j\n",
      "\u001b[0;31mKeyboardInterrupt\u001b[0m: "
     ]
    }
   ],
   "source": [
    "model.train()\n",
    "MAX_SMILES_STRING = 50\n",
    "for epoch in range(epochs):\n",
    "    loss = dict()\n",
    "    avg_loss = defaultdict(float)\n",
    "\n",
    "    for idx, batch in enumerate(loader):\n",
    "        boa, z, s, kl = model(batch)\n",
    "        \n",
    "        if DEBUG:   \n",
    "            debug_fn(batch, boa, z, s, MAX_SMILES_STRING) \n",
    "                    \n",
    "        optimizer.zero_grad()\n",
    "        \n",
    "        # we have to permute because loss function expects (N, C, d1, d2, dK)\n",
    "        # and we have a K-dimensional loss here\n",
    "        loss['boa'] = loss_fn(torch.permute(boa, (0, 2, 1)), batch.y_boa.long())\n",
    "        \n",
    "        # we don't need to change input because s is already with shape (B, C) and\n",
    "        # y_fc_edge_attr has shape (B,) with each value between 0 and C\n",
    "        loss['edge'] = loss_fn(s, batch.y_fc_edge_attr.long())\n",
    "        \n",
    "        loss['kl'] = kl\n",
    "        \n",
    "        loss['total'] = lambda_boa*loss['boa'] + lambda_edge*loss['edge'] + lambda_kl*loss['kl']\n",
    "        loss['total'].backward()\n",
    "        \n",
    "        # TODO: add torch inference mode wrapper here?\n",
    "        avg_loss['total'] += loss['total'].item()\n",
    "        avg_loss['boa'] += loss['boa'].item()\n",
    "        avg_loss['edge'] += loss['edge'].item()\n",
    "        avg_loss['kl'] += loss['kl'].item()\n",
    "        \n",
    "        optimizer.step()\n",
    "\n",
    "    print()\n",
    "    print(f\"Epoch {epoch} ------ average losses ------- | \", end=\"\")\n",
    "    for name,value in avg_loss.items():\n",
    "        print(f\"{name}: {value/len(loader):.5f} | \", end=\"\")\n",
    "    print()\n",
    "    print()\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8511c588",
   "metadata": {},
   "source": [
    "We can optionally save our current checkpoint"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a7fb7893",
   "metadata": {},
   "outputs": [],
   "source": [
    "torch.save({\n",
    "    'epoch': epoch,\n",
    "    'model_state_dict': model.state_dict(),\n",
    "    'optimizer_state_dict': optimizer.state_dict(),\n",
    "    'loss': loss,\n",
    "    'avg_loss': avg_loss\n",
    "}, \"molegen.ckpt\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9d0870de",
   "metadata": {},
   "source": [
    "## Visualizing latent space"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fdec9095",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(128, 2)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.collections.PathCollection at 0x708d62ff5fc0>"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAigAAAGdCAYAAAA44ojeAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjMsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvZiW1igAAAAlwSFlzAAAPYQAAD2EBqD+naQAASU9JREFUeJzt3XtwXOV9N/Dv3u9aSZZlWbJ8kWxsgo0hcezYvA2meDAJbZO0pQmlLbiUBGqSEDNpcOZtXCfTGBJPSMObcUiaGGZo7k1ImgYMuRgGMMSAIcjYIAnLlq3LSpa1V+31PO8fYtdaaXe1l3POnrP7/cwwg7S3szqWzm+f3+UxCCEEiIiIiDTEWO0DICIiIpqNAQoRERFpDgMUIiIi0hwGKERERKQ5DFCIiIhIcxigEBERkeYwQCEiIiLNYYBCREREmmOu9gEUIkkShoaG4PF4YDAYqn04REREVAQhBILBINrb22E0lrcWoukAZWhoCJ2dndU+DCIiIirD4OAglixZUtZjNR2geDweANNvsKGhocpHQ0RERMUIBALo7OzMXMfLoekAJZ3WaWhoYIBCRESkM5WUZ7BIloiIiDSHAQoRERFpDgMUIiIi0hwGKERERKQ5DFCIiIhIcxigEBERkeYwQCEiIiLNYYBCREREmqPpQW1EapIkgXOTUwjHk3BZzehodMBo5B5QRETVwACFCECfL4hDPaPoHwshmkzBbjahe6Eb29cuwsrW8kc1ExFReRigUN3r8wVx8LkBTITjWOy1w2l1IBJPomfIjyH/FHZctZxBChGRyliDQnVNkgQO9YxiIhzHqlY3PHYLTEYDPHYLVrW6MRGO48njo5AkUe1DJSKqKwxQqK6dm5xC/1gIi732OZtaGQwGLPba0ecL4dzkVJWOkIioPjFAoboWjicRTabgtObOdjqsJsSSKYTjSZWPjIiovrEGheqay2qG3WxCJJ6Ex26Zc/tUPAWb2QTXjACG3T5ERMpjgEJ1raPRge6FbvQM+eG2mbPSPEIIDPujWNfhRUejAwC7fYiI1MIAheqa0WjA9rWLMOSfQq9vuhbFYTVhKp7CsD+KZpcV1122CEajgd0+REQqYg0K1b2VrR7suGo51rZ7MRlJYGA8jMlIAus6vJmgg90+RETq4goKEaaDlK6t7ry1Jelun7YGG4LRJOIpCVaTER67eU63T2ezs8rvhohI/xigEL3DaDTkDS7C8STGQzEMTU5hciqBZEqC2WREk9OKla1uNDjMGA1E2e1DRCQTBihERRgLxjA4EYEkgCaXFRa7GYmUwFgwilAsiVWtrjndPkREVD7WoBDNQ5IEXhuchMVkhNkIWE0GGA0G2MxGNLus7xTKBtC90JXp9iEiosowQCGax7nJKbw9FsbajgY4bWZMhOOIJVOQhEA8JSGZkpBISljf2ch5KEREMuF6NNE80tNmu1rccNnM6PeFMRGJIxxLwmQ0YnGjA1aTES0eW7UPlYioZjBAIZrHzGmzzS4bmpZbszp5AAH/VJL1J0REMuJfVKoJSo6fzzVttsExPRZfCIFeXyhr2iwREVWOAQrpntLj50uZNktERPJggEK6ptb4+fS02XQgNBqIwmY2YV2HF9ddxn14iIjkxgCFdGv2+Pn0Rn8euwVumxm9vhCePD6Krha3LKsb802bJSIi+TBAId1Kj59f7LVn7UIMQLHx84WmzRIRkXw4B4V0K93+68zTPeOwmhBLpjh+nohIhxigkG7NbP/NZSqe4vh5IiKdYoBCupVu/x32RyGEyLpNCIFhfxQrW91s/yUi0iEGKKRb6fbfZpcVvb4QgtEEkpKEYDSBXl+I7b9ERDrGtW/SldkD2bpa3Gz/JSKqQQxQSDcKDWS7c2s323+JiGoIAxTSBbUGshERkTawBoU0b/ZANo/dApPRAI/dglWtbkyE43jy+CgkScz/ZEREpAsMUEjzShnIpgWSJDA4EcHJkQAGJyIMnIiIysAUD2nexYFsuduFHVYTRgNRTQxkU3rjQiKiesEAhTRv5kA2j90y53atDGRjnQwRkXyY4iHN08NANtbJEBHJiwEKaZ4eBrLprU6GiEjrmOIhXVjZ6lFsINvs4W/lzFDRU50MEZEeMEAh3VjZ6kHXVresA9nkKmrVS50MEZFe8K8l6YrRaEBns1OW55KzqDVdJ9Mz5IfbZs5K86TrZNZ1eLlxIRFRkViDQnVJ7qJWPdTJEBHpiaIByrlz5/B3f/d3WLBgARwOB9atW4eXXnpJyZckKooSRa3pOpm17V5MRhIYGA9jMpLAug4vW4yJiEqkWIrnwoULuOqqq3DNNdfg8ccfx8KFC9Hb24umpialXpKoaEoVtSpRJ0NEVI8UC1Duv/9+dHZ24uDBg5nvrVixQqmXIyqJkkWtctbJEBHVK8VSPL/85S+xYcMG3HjjjWhtbcWVV16J73znOwUfE4vFEAgEsv4jUoIehr8REdUzxQKUt99+GwcOHMCqVatw6NAh3HnnnfjUpz6FRx55JO9j9u3bB6/Xm/mvs7NTqcOjOseiViIibTOI2R8fZWK1WrFhwwY8//zzme996lOfwtGjR3HkyJGcj4nFYojFYpmvA4EAOjs74ff70dDQoMRhUp2bOQcllpxO66xsdVc8/I2IqJ4FAgF4vd6Krt+K1aAsXrwY73rXu7K+d+mll+K///u/8z7GZrPBZrMpdUhEc7ColYhImxQLUK666iq8+eabWd976623sGzZMqVekqgsLGolItIexWpQPvOZz+CFF17Al7/8ZfT19eH73/8+vv3tb2Pnzp1KvSQRERHVCMUClPe+9734+c9/jh/84AdYu3YtvvSlL+HrX/86br75ZqVekoiIiGqEYkWycpCjyIaIiIjUJcf1m3vxEBERkeYwQCEiIiLNYYBCREREmsMAhYiIiDSHAQoRERFpDgMUIiIi0hwGKERERKQ5DFCIiIhIcxigEBERkeYotlkgkdZIkuCuxUREOsEAhepCny+IQz2j6B8LIZpMwW42oXuhG9vXLsLKVk+1D4+IiGZhgEI1r88XxMHnBjARjmOx1w6n1YFIPImeIT+G/FPYcdVyBilERBrDGhSqaZIkcKhnFBPhOFa1uuGxW2AyGuCxW7Cq1Y2JcBxPHh+FJGl2z0wiorrEAIUqIkkCgxMRnBwJYHAiorkL/bnJKfSPhbDYa4fBkF1vYjAYsNhrR58vhHOTU1U6QiIiyoUpHiqbHuo6wvEkoskUnFZHztsdVhNGA1GE40mVj4yIiAphgEJl0Utdh8tqht1sQiSehMdumXP7VDwFm9kEl5W/CkREWsIUD5VMT3UdHY0OdC90Y9gfhRDZxyOEwLA/ipWtbnQ05l5hISKi6mCAQiWTq65DjfoVo9GA7WsXodllRa8vhGA0gaQkIRhNoNcXQrPLiusuW8R5KEREGsN1bSqZHHUdatavrGz1YMdVyzOvNxqIwmY2YV2HF9ddpp16GSIiuogBCpWs0rqOatSvrGz1oGurm5NkiYh0ggEKlSxd19Ez5IfbZs5K86TrOtZ1eHPWdcyuX0k/1mO3wG0zo9cXwpPHR9HV4pY9eDAaDehsdsr6nDNxlD4RkXwYoFDJ0nUdQ/4p9Pqma1EcVhOm4ikM+6MF6zpKqV9RMpiQmx5aromI9IRFslSWdF3H2nYvJiMJDIyHMRlJYF2Ht2CK5mL9Su7Y2GE1IZZM6WouSTpl1TPkR6PTgq4WNxqdFvQM+XHwuQH0+YLVPkQiIt3hCgqVrZy6jlqbS1LNlBURUS3Tx1WANKvUuo5K6le0qFZTVkRE1cYUD6mq1uaS1GLKiohICxigkOrKrV/Ropkpq1z0lrIiItIK/tWkqqiVuSS1lrIiItIKBihUNUrPJVFDJS3XRESUH1M8RBWqpZQVEZFWcAWF6pack19rJWVFRKQVDFCoLikx+bUWUlZERFrBAIXqTjU2KyQiotKwBoXqyuzJrx67BSajAR67Bata3ZgIx/Hk8VFIkqj2oRIR1TUGKFRXSpn8SkRE1cMUD9WVi5Nfc88lcVhNGA1Eqz75Vc4CXiIiPWKAQnVFD5sVKlHAS0SkN0zxUF1JT34d9kchRHadSXry68pWd9Umv6YLeHuG/Gh0WtDV4kaj04KeIT8OPjeAPl+wKsdFRKQ2rqCQ7IpNT1QjjaHlya+zC3jTNTIeuwVumxm9vhCePD6KrhY30z1EVPMYoJCsik1PVDONkZ78mn790UAUNrMJ6zq8uO6y6qVRSing5bwVIqp1DFBINsXOF9HCHBItTn7VSwEvEZEaGKCQLIpNTyxvdmkmjVHp5Fe5U1R6KOAlIlIL/9KRLIpNT7wyeKEm0hhKpKjSBbw9Q364beasn0+6gHddh7dqBbxERGpiFw/J4mJ6InfM67CaEEumcD4cL+p+Wk5jKNVpky7gbXZZ0esLIRhNIClJCEYT6PWFqlrAS0SkNtUClPvuuw8GgwF33323Wi9JKpqZnsglnZ5Y4LIWdT+tpjGUHpWfLuBd2+7FZCSBgfEwJiMJrOvwco8gIqorqlwFjh49ioceegiXX365Gi9HVVBseuLdnU04euqCbtMYhVJZAOC2mfHy6Qm8dLoJG5Y1l7XaocUCXiIitSm+ghIKhXDzzTfjO9/5DpqampR+OaqSYtMTZrNR12mMfKmsiXAMLw1cwGtnJ3F8KICHnn4bBw73V5Tu6Wx2Yk1bAzqbnZr9eRARKUXxAGXnzp244YYbsG3bNqVfiqqs2PSE3GkMSRIYnIjg5EgAgxMRRXcizpXKmgjH8OrgJHzBKMxGA7wOC5pdnP5KRFQJRVM8P/zhD/HKK6/g6NGjRd0/FoshFotlvg4EAkodGimk2PSEXGkMtQe+zU5lAUC/L4ypeApNTgsuRBJobbBjsXc6RcXpr0RE5VEsQBkcHMSnP/1pPPXUU7Db7UU9Zt++fdi7d69Sh0QqKXa+SKVzSKox8G32qHy3zYzxcAxWsxEXIgk4rGZ0L7w430UvbdNERFpjELN3TJPJY489ho985CMwmUyZ76VSKRgMBhiNRsRisazbgNwrKJ2dnfD7/WhoaFDiMEmnJEngwOF+9Az5swa+AdPFtr2+ENZ1eHHH1d2KrFykV25eOTOB40MBeB0WLHDb0L3QjWaXNXO/pCRhYDyMT167Cmva+G+YiOpDIBCA1+ut6Pqt2ArKtddei9dffz3rezt27MCaNWvwuc99bk5wAgA2mw02m02pQ6IaUu19a9IpqpdON+Ghp99Gs8uCxV7HnGPRets0EZFWKfZX0+PxYO3atVnfc7lcWLBgwZzvkz5UY/fhfKqxb02u979hWTOOLp1um55ND23TRERaxY91OqdW0FDN3YdzUXvfmkLvf2ZNymKvHQ6rCVPxFIb9Uc23TRMRaZWqAcrhw4fVfLmap1bQoIXdh2dTc9+aYt7/jquWZ87FaCAKm9mEdR1eXHdZdQI4IiK94wqKTqkVNBS7S7HabbSzu2mUWrko9v3fcXU37uT0VyIi2TBA0SE1g4ZqF6MWkh74puTKRanvn63ERETyYICiQ8VeNAcvRGA0GCr6RF+NYtRSKL1vjdbfPxFRrWKAokPFXDT7fCEcfO4UAlPJiupT1C5GLUelA98K0cP7JyKqRYrvxUPyy7UfzEzDk1MYnIjg1HgYjU4LulrcaHSWtzdMuhh12B/F7Jl+6WLUla3umm2jrff3T0RULQxQdKjQRVOSJPQMBWAxGXF5hxceuwUmowEeuwWrWt2YCMfx5PHRojfUK3aX4nRKRc2N+9RQ6vsnIiJ5cF1ahwp1sPSPhZBISrhyaSOMxuz4s9yi1mKLUbU2K0UuahTjEhFRNgYoOpXvotnV4oYQQHtj7uCj3KLO+YpRtTgrJZ9yhtspXYxLRETZGKDoWK6LphACX/9NryJFnfmKUbU6KyWXSlZ5lCzGJSKibAxQdG72RVOShGoTVtO0PCtlJj2t8hAR1TsWydaYahR1Xmx7zh3vOqwmxJKpqs4Kmb3KU2nxMBERKYsBSg1K16esbfdiMpLAwHgYk5EE1nV4FVklmK/tWQuzQkpZ5SlFrXUtERFpBVM8NUrNok41N+4rlxITYWu1a4mISAsYoNQwtYo61dq4rxJyT4RlPQsRkbKY4iFZqJ1WKpWcE2FZz0JEpDyuoJBstDwrRM5VHr10LRER6RkDFJKVlmeFyDURljscExEpjwEK1RU5Vnm4wzERkfL4F7TGlTPWvdZVusqjh64lIiK9Y4BSw9gGqww9dC0REekdA5QaxTZYZXGHYyIiZTFAqUF62rxPz7TctUREpHcMUGoQ22DVo+WuJSIiPeOgthqkh837iIiICmGAUoP0sHkfERFRIQxQapCcY91pGnctJiJSFz9C56Hn+SFsg5UX27WJiNTHACWHWrggsQ1WHmzXJiKqDgYos9TSBYltsJVhuzYRUfUwQJmhFi9IbIMtH9u1iYiqh0WyM5RyQaLax3ZtIqLqYYAyg9wXJHZ+6BvbtYmIqod/WWeYeUHy2C1zbi/lglQLhbb1jrsWExFVD1dQZpBrfki60LZnyI9GpwVdLW40Oi3oGfLj4HMD6PMFlXwbJJN0u3azy4peXwjBaAJJSUIwmkCvL8R2bSIiBTFAmUGOC9LsQluP3QKT0QCP3YJVrW5MhON48vgo0z06kW7XXtvuxWQkgYHxMCYjCazr8Oqqo4uISG+Y4pml0vkh7PyoPWzXJiJSHwOUHCq5IF0stM2dBnJYTRgNRNn5oTNs1yYiUhcDlDzmuyDlG4UvZ6EtERFRveJVsgyFOnS6Wtzs/NCgUvZW0vM+TEREtYIBSomKGYXPjfq0pZSWb7aHExFpA7t4SlBsh05Xi5udHxpRSss328OJiLSDKyglKKVDh50f1VfK3koAam4fJiIiPWOAUoJSO3TY+VFdpe6txPZwIiLtYIqnBHLszcL9edRTyt5K3BiQiEhbFF1B2bdvH372s5/h5MmTcDgc2LJlC+6//36sXr1ayZdVTKV7s7AAU12ltnwr3R7O7iAiouIpGqA8/fTT2LlzJ9773vcimUzi85//PK677jq88cYbcLlcSr60ItKj8Mvp0Cmm+4dBirxKDSiVbA9ncEpEVBpFA5Qnnngi6+uHH34Yra2tePnll/H+979fyZdWTDmj8Esp1uQnavmUGlAq1R7O4JSIqHSqFsn6/X4AQHNzs5ovK7tSO3S4P0/1lBJQVroPUy4MTomIyqNagCJJEu6++25cddVVWLt2bc77xGIxxGKxzNeBQECtwytZKR06auzPw/qG/EoJKOVuD2dwSkRUHtUClJ07d6KnpwfPPvts3vvs27cPe/fuVeuQVKP0/jysb5hfKQHlzPtWGvhx80giovKoEqDcdddd+NWvfoVnnnkGS5YsyXu/3bt3Y9euXZmvA4EAOjs71ThERVXa/VMI6xuUI0fgx80jiYjKo+hfRSEEPvnJT+LnP/85Dh8+jBUrVhS8v81mg81mU/KQqqKS7p9CWN+gHLkCPyWDUyKiWqbooLadO3fi0Ucfxfe//314PB6MjIxgZGQEU1NTSr6sJqULMOXcn6fUSalUnGL3XCpmyF46OG12WdHrCyEYTSApSQhGE+j1hbh5JBFRHoquoBw4cAAAsHXr1qzvHzx4ELfeequSL61Jchdgsr5BGXIXtirRHUREVOsUT/HUu1xFlnJ1a7C+oXyFil+VCPy4eSQRUWl45VKQ0t01rG8oz3znRanAj5tHEhEVjwHKLHLNE1Gju0ap4ttaVsx56WpxM/AjIqoyBigzyLXioWZ3DesbilfsebnjajcDPyKiKmOA8g45VzzUnh7K+obilHJeGPgREVUXAxQU/8l6ebMLw+8URxYKAqrRXcP6hvmVel4Y+BERVQ8DFBT3yfqVMxfwlUNvYjwUmzf9w+4abSrnvDDwIyKqDkUHtenFxU/WuQOGaCKFt0aDeGPYj0anBV0tbjQ6LegZ8uPgcwPo8wWz7p/urhn2R+e0WqeLLFe2utHR6IAkCQxORHByJIDBiUhRw7+oPKWcFyIiqq66/Ag/u1PHaTHl/WQthMCbI0EkUwIrF7oztxcqeC22u+bt8RA3+VMRu56IiPSj7gKUXJ06XS0uNDotGPZH57SVBqYSGPZHsbjRjgZHdvBSqOB1viJLANzkrwpY/EpEpA91FaDk69Q5PhyAyWiAyWiY88m6bywEs8mA1Ysa5tSnAIULXvMVWQLAgcP93OSvSlj8SkSkfXUToBTTqdPutaPJZcXbY+HMJ+t3tTfAbjHBbsldrjNfwWuuIsvBiYiqbcg0VzHFr3IN7SMiotLVTYBSTKfOhUgC/7BlOYwGQ+aitLjBjoeeeVvWqaLc5E/7lN6mgIiICqubAKXYoGAqkcKatoas2+QurGQbsrapsU0BEREVVjdtxjODglwKBQXpwsq17V5MRhIYGA9jMpLAug5vWRcrtrtq1+xUoMdugclogMduwapWNybCcTx5fLTkdnC2kxMRlaZuPqJXuvOvnIWVSrS7sl5CHkpsU8B0ERFR6eomQJEjKJBzqqic7a68AMpH7vqgdLrofCiOBrsZDXYLJEng9XNMFxERFVI3AQqgvRkYcqzKsF5CXnLWB6XTRWcmIkgmJQycDyMpSTAbjWhyWBCOJ9lOTkSUR10FKID2ZmBUsipT7CaHvAAWr9JU4EznJqdwbPACxoJRJFMCbrsZFpMZiZSEsVAMJqMBr5y5wHZyIqIc6i5AAWpnAzgl6iXqnZz1QcFoAmfOR5CSJCxw2zLnyGY2weoy4nwohsGJCILRhNJvi4hId+qmi6cWzbfJocNqQiyZ4jyVEsnVtRWKJTGVSMFmMeUMIG0WEyLxFEIxnh8iotnqcgWlVnCeinLkSAW67ebpIDEhwW0Tc9JFsYQEp9WEUCyJkyOBqqcbiYi0hFcuHZOzXoLmqjQV6LFZsLTZicGJCCbC8XdqUIxIpCSEoklIQiCZEvjRHwZhMhnYfUVENAMDFB1TYp4Kyaej0YErO5sQS0hIShIuRBIIxZIwG41w28wYCUTR4LCgvdEOl83C7isiohkYoKhIiWFqWmudpotmBpDnQzEsaXLAZDQgmRJ4bXASFpMRG5c3o8FhBcDuKyKimRigQJ0prEoOU9Na63Q9me/fzuwAMhJPIikJWMwmrFvixQK3Lev52H1FRDSt7gMUNaawqjFMrVZap/Wk2H87swPIEX8UPzp6Bu2Nuc8Xd7MmIqrzAEWNwIHD1GpTqf92ZgaQLqsZDouZ3VdEM3A/MZqtbv8CqhU4cJha7an03w67r4iyybWSzSCnttRtgKJW4CD35nNUfZX+22H3FdFFcq1kc9PU2lO3AYpagQOHqdUeOf7tsPuqNij1ib1eVgLkWsnmpqm1qW6vimoFDlzOrz1y/dth95W+KfWJvZ5WAuRYyWadX+2q27140oHDsD8KIUTWbenAYWWru+LAIb2c3+yyotcXQjCaQFKSEIwm0OsLcTlfh+T8t5Munl3T1oDOZif/HehE+hN7z5AfjU4LulrcaHRa0DPkx8HnBtDnC2rqedMkSWBwIoKTIwEMTkQgSWL+Bymo0H5i4p1Jy2OhGPrHQnmPtZQgh/SlbldQ1KwD4HJ+bWENSX1T6hO70isBWlyZybcaORGOod8XxmgwiqlECj948QyOnwvkPNZiU67BaAKDExGuVupI3QYogLqBA5fzawuDzvqlVIG9koX7Wq3RyJUCnwjH8OrgJCKxJJISsLTJifZGe95jLSblGktKeOzYEMZDMc0EZzS/ug5QAHUDBw5Tqy0MOuuTUgX2cj7vzCJbh8WEJ3pGNFmjMXs1sq3Bht7REAJTCZhNRjQ4zFi1yIMGhxUeuyXnsc5X59fre+f5jAa0Nzo0E5zR/Oo+QAEYOFD55Pq3Uy9dG7VAqQJ7uZ53dionlRIYvBDBmjaPJmcxzVyN/OO5SQxeiMBuMWFRgx3dC91odlkLHmuhlOvQZBSBqQQa7BZcssijqeCM5scAhajKtFgbQPkp1Zknx/PmSuWcvRDB+XAcb44G4bKZ0ezK3v9JC7OY0quRz/SO4T+ffRtdC6aLg2cHVPmONV/KdekCB5KShKXNTk0GZ1QYAxSiKtJqbQDlp1SRdKXPm6/ItslpRZPDgnA0if6xMJqc1qyLtVZmMRmNBnQvdKPVbYfZZJgTUACFjzVXyjUYS+D//a4vZ5cQoI3gjPKr2zZjomqbfUHx2C0wGQ3w2C1Y1erGRDiOJ4+PVr0VtJ7la8tNf2Jf2+7FZCSBgfEwJiMJrOvwVhRUVvK8+YpsPXYzmlw2CBhwPhRDMHrxYjy7Lb7abciVtvDPbtv32CyZtFkuWgnOKDeeFaIq4T5N2jZf6k2pIulynzdfka3BYMDKVjcC0QTOh2K4EInDaZu7MvP2eKjqqUa5V6c4KFPfGKAQVQn3adKuYlNvchRJ5yuQLvV5CxXZNrusWL3IjZMCmEqkMDAezmqLB6CZVKOcLfzpgOfcZASvnZ1Ek9MKj90Ms9GAkUCMM4s0jgEKUZVwnyZtUnN0upwF0vOtFkwlJNywrg1/vr4dkUQqEwwBwIHD/ZpqQ5Z7dcpuMWEsGEPfaAgwAF6HFZu7mnHTpqWs8dIw/uUjqhIuP2uTWqk3uQuki0mPbF/bhqULXFmPG5yIaDLVKMfq1Myf8aYVC5CSBILRBCYicUwlJJmOlJTCIlmiKuE+TdpUaH8YYDr1FkumKkq9KVUgXU6RrRrvtxpm/4wbHBY0uaxYusCF9UsacSFSWRF6tQuK6wFXUIiqiCPztUeN1JuSqzSlpkdqNdWo9NYB1S4orgeK/4v75je/ia9+9asYGRnB+vXr8eCDD2Ljxo1KvyyRbnBkvraokXpTukC6lPRIraYalfoZc3aRehRN8fzoRz/Crl27sGfPHrzyyitYv349tm/fDp/Pp+TL0ju4BKkfs+c3MDipHjVSbzNXLXJRc9WiVlONSvyMObtIXYoGKF/72tdw++23Y8eOHXjXu96Fb33rW3A6nfje976n5MsSpqP8A4f78cBTb+Ebv+3FA0+9hQOH+9HnC1b70Ig0T6lBbGmVDiSTm9LvtxqU+BmXkjaiyikWnsfjcbz88svYvXt35ntGoxHbtm3DkSNHcj4mFoshFotlvg4EAkodXk3jEiRR5ZRMvSk1Lr8StZZqVOJnzNlF6lIsQBkfH0cqlcKiRYuyvr9o0SKcPHky52P27duHvXv3KnVIdUHNGQ5EtU7J3aq1WCBdazu7y/0zrtWCYq3S1E9x9+7d2LVrV+brQCCAzs7OKh6R/nB8OpG2zNfxUUurFlok58+4VguKtUqxAKWlpQUmkwmjo6NZ3x8dHUVbW1vOx9hsNthstpy3UXG4BEmkHcWmW/lhQVlyrQxpMTVXyxQrkrVarXjPe96D3/72t5nvSZKE3/72t9i8ebNSL1v30kuQ4VgCgakExkMxBKYSmSIxLkESFa+STjh2fNSmWiwo1ipFr1K7du3CLbfcgg0bNmDjxo34+te/jnA4jB07dij5snWto9GBRocFz/aPwwggKQmYTUY0Oa3oXujC+XCcS5BERah0GBfTrbWLqTl1KBqgfPSjH8XY2Bi+8IUvYGRkBFdccQWeeOKJOYWzJJ+3x0PwhWKYiqdgMgANTgsMMGBocgpnL0SwvrORS5BE85CjE47p1tpWawXFWqT4Ov9dd92Fu+66S+mXIVxcUk5JAldf0oK3xyKYiMSRklKwW4wQAmj12NDV4q72oRJpllydcOz4qK5cnVP8YKYv/M2oITOXlD12C5pdNgSjScRTEqwmIwCByUiCS8pEBciVmmHHR/Vwr5zawAClhsxeUjYYDGhwXPzklpQk+IIxLikTFSBXaoYdH9XBQZW1Q9FR96QuLe3vQaRXcv4eseNDXXrvnOL+adl4paohXFImqpzcv0fs+FCPnjunmJaaiwFKDeGSMlHllPg9YseHOvTaOcW0VG5M8dQYLikTVY6/R/qkxzR3JWmpWk8JaecskWy4pExUOf4e6Y8e09zlpqXqISXEAKVGcUmZqHL8PdIXPaa5y0lL1UtKiCkeIiKqGXpLz5WaltJ7p1IpuIJCVAJOp6xfPPf6oaf0XKlpKT13KpWKAQpRkeoh50u58dzrj17Sc6WmpfTaqVQOBihERSgm59vVoo9PbFSaesn3U/Wk01LpIHg0EIXNbMK6Di+uuyw7CK6nPZ70/w6IFFbM5nE/ePEMmlxWvD0W5ifsGiLXxoFE8yk2LaXHTqVyMUAhmsd8OV+HxYjfnfRh6QInuhe6+Qm7htRTvr9W6LlWqJi0lB47lcrFAIVoHoVyvkJM/zGcSqTQ0ejILLnyE3ZtqKd8fy2ol1qhUlJCesYAhWgehXK+wWgS46E4XDYzbGZT1m38hK1/9ZTv17t6qxXSU6dSuTgHhWge6ZzvsD8KIbJnC8SSKYRiSSz02OCxz71IOawmxJIpfsLWqULnPp3vX9nqrol8v57V02yQmdIpoTVtDehsdtZUcAIwQCGaVzrn2+yyotcXQjCaQFKSEIwmcG5yCk6LCe05ahQAfsLWu0LnvtcXqql8v56VUitE+sEAhagI+aZTblzejGvWtGIqIfETdo3S22TSenSxVij3B4FaXsms5Q0D+bGOqEj5cr5vj4dw8LmBmq+or2f1kO/Xs3qtFar1ouDaOltECsvVBlgvFfX1Ti+TSetRPc0GSZuvKPiWzcvhsJp0HVAzQCGSAT9hE1VPPcwGmTnfxWEx4YmekbwDBI8NTuJLv3oDLW4rYilJtysrDFCIZMJP2ETVU8srmW+NBvDTl86hfyyElJBgNRpxdnIKa9o8c4qCL0Ti8AWiCEaTaPMuQEeTW7ft1gxQiIioJtTiSuZvT4ziG7/txVgwBqvZCJvZCJPBgNFgDAYD4LKZ0eyyAZhOZ/X7wkimJDitJljNpky7tR4HRzJAISKimlHOSqZWx+O/NRLEN37bi5FAFIsb7LCYjUikBC6EY0hKEi6EE+gfC6PJaYXBYEAwmsREJA6bxQRJAFbTxUZdPQ6OZIBCRER1S6udMJIk8NOXBzEWjKGtwQabZXpStc1sQKvHhmA0iUg8hfPB6XROg8OCeEpCIpUChAGLvPY5wyP1tjUD56AQEVFdSnfC9Az50ei0oKvFjUanBT1Dfhx8bgB9vmDVji09fM5qNsA6axsNo9GI1gYbBATORxK4EIkjKUmIJyVMxSWYzUZ0L3TPqU/RW7s1AxQiIqo7Wh+PH44nkRKAzWxCIiXNud1jt8BlNaPBbsFUIoWB8enak+6FbrS6bWhyZs+D0ePgSH2EUURERDIqZTx+Neo1XFYzmhwWBKcS8E8lYHUZs44znpwOWrZd2oq/e98yRBIpuKxmTCWSeOT50zXRbs0AhYiI6s7F8fi5VxOqXa/R0ejAylYPxsNxxJIpTITjcNvNsJiMiCdTGAnE0NZgx19vWIKlC1xZj62VdmsGKEREVHe0Ph5/5vA5AIjEkgjGkogn44gnBRY32PHJa1fhkkUNcx5bK+3WDFCIiKju6GE8/szhc32+ICanEjAagO5WN/763Z24pC3/akgtDI5kgEJERHVHL+Pxa2U1pBwMUIiIqC7pZTx+LayGlIMBChER1a16XqHQOgYoRCXS6lhsIipPva5QaB0DFKISaHUsNjFwJKo1DFCIipQeiz0RjmOx1w6n1VHWNua8kMqPgSNR7WGAQlSE2WOx0y2JpW5jXmsXUi0EW3IFjkSkLQxQiIogx1jsWruQaiHYkitwJCLt4WaBREW4OBY7d0zvsJoQS6byjsXW+sZkpdLKLrClBI5EpC8MUIiKMHMsdpoQAoGpBMZDMfgCMVhNxrxjsWvpQqqlYKvSwJGItIspHqIizB6LfSESR78vjIlIHIlUClPx6W3OpxK5L4Ra35isGOl6k/6xEP54dhLtjdXfBVbr+6kQUfn4W0tUhJljsY8NTsIXiCKZkmCzmABhgMcxfXF85PnTOWtJ9H4hnVlvMhaKon8sDP9UAqsWedDssmbdV81gSw/7qRBReRRJ8QwMDOC2227DihUr4HA40N3djT179iAejyvxckSqWNnqwS2blwMCCEaTMBgMkASwyGvHxuXNuHJpY970RvpCOuyPQojs29IX0pWtbk1eSGfXm6xY4IbDYsKwP4pXBycxEc7+vVYz2EoHjs0uK3p9IQSjCSQlCcFoAr2+kGb2UyGi0inyF+TkyZOQJAkPPfQQVq5ciZ6eHtx+++0Ih8PYv3+/Ei9JpAqH1YQWtxVt3gWwmk2wmozw2C9+cs+X3tDLxmSz5eqSEUJgkceO0cAUIvEk+sdCaHI2ZW5Te9VCL/upEFFpFAlQrr/+elx//fWZr7u6uvDmm2/iwIEDDFBI18LxJGIpCR1NbphyBBOF0ht6vJDmKu41GAzobnUhGEsgMJXAaCCKC5E4LCZj1YIt7qdCVHtUS3j7/X40NzcXvE8sFkMsFst8HQgElD4sopJUWkuitwtpvuLeZpcNV3Q2onc0hMELEQycD2Oh217VYIv7qRDVFlUClL6+Pjz44IPzrp7s27cPe/fuVeOQiMoiR1Gmni6khQKyZpcNly42wOu04KaNS9G90C1rsFVoSq0WJtgSkbIMYnbFXgH33nsv7r///oL3OXHiBNasWZP5+ty5c7j66quxdetW/Od//mfBx+ZaQens7ITf70dDQ0Oxh0mkqNkTYWfXkuhtImwhkiRw4HA/eob8WZNagemArNcXwroOL+64ulvWAKHQlFoAVZ9gS0SFBQIBeL3eiq7fJQUoY2NjOH/+fMH7dHV1wWqdbjscGhrC1q1b8b73vQ8PP/wwjMbSmobkeINESph5AY0lp9M6K1vdmq0lqYTaAdncLQHMiMSTGPZHM3U/KUnMua3WgkMiPVM9QCnFuXPncM011+A973kPHn30UZhMppKfgwEKaVk9pRlyBWTdC924vNOLhR6bbO+/0IqNJEk4dHwUMADb37Uo6wOPkqs5RFQ6Oa7fitSgnDt3Dlu3bsWyZcuwf/9+jI2NZW5ra2tT4iWJVKenWpJKzS7uHQ/G8OqZSfz8lXOyplkKbQkQiqWQEgIQ0//f4LgYoKg9wZaIlKdIgPLUU0+hr68PfX19WLJkSdZtCi3YEJHC0gFZny+Ix3tGFNmVudCWAPGUBEAAMLzz/9n0sF0AERVPkUmyt956K4QQOf8jIv1SeqPAXJsypllNRgCGGf+fTQvbBUiSwOBEBCdHAhiciOhmd2oiLdLmxh9EpEml7MpcTpqlUBu322aCyWAADNP/P5MW9t0p1HnEwl2i0jFAIaKiKb0r83xbAlzSNn2h7xsLa2q7gLmdR/KlvYjqFQMUIiqaGrsyz7clAABNbReQa78iAPDYLXDbzOj1hfDk8VF0tbjZXURUAgYoRFQ0OSbpFmO+LQG0tF2A0mkvonrFAIWIiqbmrsyF2ri11OKtdNqLqF4p0sVDRLUrnYJZ2+7FZCSBgfEwJiMJrOvw1mWtRaHOI0Ab3UVEesTfGKI6Itf0W73tyqwktdJeRPWGAQpRnZC7DVZLaZZqUjPtRVRPGKAQ1QG2wSprvs4j/myJSscAhajGsQ1WHUx7EcmLAQpRjWMbbG5K7EbNtBeRfBigENU4tsHOpfWx9EoET0R6wwCFqMapMf1VT7Rej6P14IlILZyDQlTj0m2ww/7onB3F022wK1vdddEGq/RuzJVKB089Q340Oi3oanGj0WlBz5AfB58bQJ8vWJXjmok7NpNa6uMjE1EdYxvsRVqux9FDMTNXd0hNDFCI6gDbYKdVsx5nvroSLQdPQOWpMdbVUKkYoBDVCbbBVq8ep5iVh3A8ialEEu6UGeOhGKwmIzz2i5Npq1nMXOnqDldeqBwMUIjqSL23wVZjLH2xKw9jwRhOn5/CW6MhGACYTUY0Oa1Y2epGs8sqe/BUyopGJas7Wi9KJu1igEJEdUPtepxiVx4kIfD46yNIpiSkUhJaPDYkJWAsGEUolsT6JV6cD8dlC55KXdEoNzUmZ10NU0T1hwEKEdWVXPU4VpMRnc0ObFjeDJvZBEkSslz8ill56B0NYjKSwIVIHBtXNOG1s35MRhJw281odFowFozjDwMT2LCsSZbgqZwVjXJTY3LV1TBFVJ8YoBBR3ZlZj3NiOICXBi5gLBDFY8fO4QnziGwXv2JWHk6NJxCIJrFsgRMeuwVXdBrQ7wtjIhJHSpJgNhlgMRrwwXWLKz6eclc0yk2NyVGUzBRR/WKAQkR1yWg0IJZM4em3xmZc/MyyXvyKWXkwGoCUkOB8Z/Wh2WVD03IrgtEk4ikJJoMB46EYWjy2so8jrdwVjXJTY5UWJeuh9VrPtJ42Y4BCRHVJjYtfMSsP3a1ujAViWRdxg8GABsf0/wejCdgt8hTHVrKiUU6reqVFyVpvvdYzPaTNGKAQUc0p5pOhGhe/YlYe/vrdnXjqjVFVOosqXdEotVW90qJk7iNVmmJXRPSSNmOAQkS6lesP8tvjoaI+Gap18Stm5cFohCqdRXK0WZfaql7JkEDuI1W8YldE9JQ241klIl3K9Qe50WGBLxRDShLzfjJU8+I338qDWpN+q7XtQblDAqsxt0aPcq2IhGNJ/GHgPI4P+3HTxqW4qrsFRqNBV2kzBihEpDu5/yAn8Gz/OKbiKVx9SUsm6Mj3yVDti998Kw9qTfqt1rYH5QwJ5D5S88u1IjIRjqHfF8b5cAz+qQQGxiP44No2XL+uDUlJ6CZtxgCFiHQl3xI1YIARgMkAvD0eQbPLlrkt1ydDLV781Jr0q6dtD7iPVGGzV0QmwjG8OjiJqXgKbrsZNosJ4VgSR09PYDgQxfVr23STNqv+ERARlSDfEnU8JSEpCTQ4LZgIxxGMJjOdMEDuT4b1fPHT07YH1QyotN6KO7OWSgiBfl8YU/EUml1WGAwGSEIgEp9Ol02E4/jjoB9dLS4cHw5oPm3GAIWIdCVfcavVZITZZIQBBqSkFOIpKev2fJ8M9bSaIBetX3RzqUZApYdW3Jm1VEIAE5E43DM2mUykJJiNRtjMJiz2mtA/FsJfvrsDw4GoZlYO82GAQkS6kq+41WM3o8lpxdDkFOwWI6wmY+a2+T4Z6mk1oVJ6uOhqgZZacQsFlDNrqZqcFiQlCRbT9KVdCIFQNInWBjs8djNSQmA0EEWLx6aLlUMGKESkGcV8ss9X3GowGNC90IWzFyIQAgAEkpKkyU+G1aKli66WaakVd76AcmYtVfrffiyZgtFgQCiahMNqRvfC6fcwFUtmVhE7m52aXzlkgEJEmlDsJ/tCxa3nw3Gs72xEq8eGyUgCvmBMk58Mq6FaF10tpJNKPQattOIWG1Cma6me6BnBWHAYvkAMjQ4LWhvs6F7oRrPLmnMVUesrhwxQiKjqSv1kP19xa1eLtj8ZVkM1LrpaSCeVcwxamGBbakC5stWDf97qxhWdjfj+H84gHEuiq8UFp82MYDShy1VEBihEVFXlfrKfr7hVy58Mq0Hti64W0knlHoPSQ/yU2orBaDTg/6xaiDavPROU6XkVkQEKEVVVJZ/stb5ErSVqTs7NFXQKISAE0OS04OyFCA71jKJrq3I1HJWktJQc4lfsik6lGztqvb6kGAxQiKiqtLCcXg/UnJyba3hYvy+MiUgcSUmCEMBYcAjrO734P6sWVvx6xRzDTMUEvkoM8StlRafSgLIWgnfj/HchIlLOzD/EuWhpsqWepS+6zS4ren0hBKMJJCUJwWgCvb6QrPUJF4NOc2ayqS8Yhd1iRJPTCpfNhIlwHN//wxn0+YIyvLvCx5CLw2pCLJnKG/im65zWtnsxGUlgYDyMyUgC6zq82HHVcnS1uDE4EcHJkQAGJyKQJFHweGav6HjsFpiMBnjsFqxqdWMiHMeTx0czz5MOKIf9UQiR/dzpgHJlq1sTA9WUwt94Iqqqet4QTu0OF7Um56aDznAsOWeyKQAYDQY0OiwIx5KKtevKkdLKlyp5ezyEA4f7Syq8LXVFR4tbMaiNAQoRVVW9/iGuVoeLGvUJ6aDzDwPncT4cy5psOnN4WFeLS7F2XbkC39mpknILb8tJZdbzVgwAAxQi0gAt/yFWYpWj2h0uStcnpIPO48N++KcSsFlMkIRAIiVlDQ9z2szwBWOK1BcpEfhWUnhb7opOrRS8loMBChFpghb/ECuxyqGlKaVKWtnqwU0bl2JgPIJwLIlIHDAbjVnDw4LRhKL1RXIHvpUU3layolMLBa/lYIBCRJqhpT/ESq1yVHKRkySBwQsRnBoPAwC6WlxY0uTUbCBzVXcLPri2DUdPT6Cj0QGb2QTPO+keteqL5Ax8K+k4q9dUZiUYoBARzaLkKke5F7k+XxDff+EMXjg1gcmpOAwC8DoteF/XAvztpqWarEcwGg24fl0bhgPRdwI9E1JCYCqWVPWinA580+m6t3zBsgKVSgtvtZzK1CLFA5RYLIZNmzbhtddew7Fjx3DFFVco/ZJERBVRcix8ORe5Pl8QX/9NL14bnITRYMBCtw0CAoFIAk+9MQpfMIa7t63S5AVOKxdlOdJ1chTeajGVqVWKByj/8i//gvb2drz22mtKvxQRkSyUHB5X6kVOkgSeeH0Eb40GYTUbsWBGu669wYTzoRjeGgkqPpm1EtW+KMuVrpMrTaOlVKaWKTqo7fHHH8eTTz6J/fv3K/kyRESyUnJ4XKkD085NTuH1c36kJAGP3ZIV0BgMBngcFqSEwB/PTeLc5FR5b1gF6YvymraGzJwPNZQ6IG0+8w1w0+Iqll4ptoIyOjqK22+/HY899hiczuIixVgshlgslvk6EAgodXhERHkpPTyulLRHOJ5EODEdKFlMcy/qFpMRgEAknn8qaj1TIl1X7RWheqFIgCKEwK233oo77rgDGzZswMDAQFGP27dvH/bu3avEIRERFa3cpfxSZqYUe5FzWc1wWab/VCdSAjZz9u2JlATAAKeV2wHkolS6jmka5ZX0r/nee+/F/fffX/A+J06cwJNPPolgMIjdu3eXdDC7d+/Grl27Ml8HAgF0dnaW9BxERHIotbiznCLMYi5yHY0OrOvw4tT5MILRBKwzalCEEAhOJWAyGnF5R6MutgNQe7y/mrs4k7xKOiP33HMPbr311oL36erqwu9+9zscOXIENpst67YNGzbg5ptvxiOPPJLzsTabbc5jiIiqpdhVDiUnw6ZbdU+OBvHa4CRGAzF4nRYAAv5IApIA1rd7sH2t9mdolNtJU0lQU897PemdQczeJlEGZ86cyaofGRoawvbt2/HTn/4UmzZtwpIlS4p6nkAgAK/XC7/fj4aGBrkPk4ioYpIkcOBwP3qG/FkzU4DpC2CvL4R1HV7ccXV3RQHEzDko/qk4AKDRYcEmDc9BmWluEGdGJH5xHkq+IE6O9uDZrz07XcfiVvnJcf1WZE1r6dKlWV+73W4AQHd3d9HBCRGRHig5M2Wmla0e/N8/e1dVJslWmpYpd/CdXCtTxaTr1E490fyYdCMiqoCSM1NmMxoNWLbAhWULXBU/V7HkWMEoJ4iTe5pvoXRdtXaWpsJUCVCWL18OBTJJRERVV4tFmOnVhBPDAfzv68OIJVJob3SUvYJRThCnxMpUrqLkau8sTfnp5zeGiEiDaq0IM72a0OcL4vhQAKFYEssWOLHQY88MOCt1BaOcIE6Nlal62VlarxSdJEtEVOtKnQyrZenVhJ4hPywmIwwGoNFpwVgwhlcHJzERni7Onb2CMZ90EDfsj85ZTU8HcStb3VlBnJLTfNNKWaUh9TFAISKqkNLjzyVJYHAigpMjAQxORIoey17qa8xcTbCYjUgJAZfNjGaXFVPxJPrHQpkAw2E1IZYsbnptOUFcOUFNqS6u0uQOckp5jyQ/pniIiGSg1PhztQo4Z68mWE1GmI1GJFISbGYT3HYzJsJxBKNJNDgsJa9glDr4Tq6N+QqpxfqhWsKfOhGRTOQef65mAefsmg+P3YxmpxW+YBRWlxEWkxGhWBLxlFR2bU2pQVypQU2paq1+qNYwQCEi0iC1CzhnryYYDAZ0t7oQjCUwEY7DajbCaDAgnkxVVFtTahCXDmrOXojg7Xfmv6xocaGzqfJAUI1VGiofAxQiIg1SawBcWq7VhGaXDVd0NqJvNITTExF47GYkU0K2FYxivT0eUizNpfQqDZWPAQoRkQapOQAOyL+aYDEZ4XVa8V6vHTdcvhiXtjWoOmVVjTSXUvVDVBkGKEREGlSNAs58qwmXL6nOaoKaaS6564eocgxQiIg0qFoFnFpaTVA7zUXawgCFiEiDqlnAWcpqgpKb7Kmd5iJtYYBCRKRRWi/gVHpGC+eU1DeeVSIiDdNSymUmNYpXOaekvjFAISLSOK0VcKpVvMo5JfWNe/EQEVFJ1NxkT+l9jki7uIJCREQlUbt4VatpLlIWAxQiIipJNYpXtZbmIuUxxUNERCVJF68O+6MQQmTdli5eXdnqZvEqVYQBChERlSRdvNrssqLXF0IwmkBSkhCMJiraSJBoJgYoRERUMhavktJYg0JERGVh8SopiQEKERGVjcWrpBSmeIiIiEhzGKAQERGR5jBAISIiIs1hgEJERESawwCFiIiINIcBChEREWkOAxQiIiLSHAYoREREpDkMUIiIiEhzND1JNr1LZiAQqPKREBERUbHS1+3Zu12XQtMBSjAYBAB0dnZW+UiIiIioVMFgEF6vt6zHGkQl4Y3CJEnC0NAQPB4PDAbtbz4VCATQ2dmJwcFBNDQ0VPtwZFXL7w2o7ffH96Zftfz++N70q5j3J4RAMBhEe3s7jMbyqkk0vYJiNBqxZMmSah9GyRoaGmryHyVQ2+8NqO33x/emX7X8/vje9Gu+91fuykkai2SJiIhIcxigEBERkeYwQJGRzWbDnj17YLPZqn0osqvl9wbU9vvje9OvWn5/fG/6pdb703SRLBEREdUnrqAQERGR5jBAISIiIs1hgEJERESawwCFiIiINIcBSgn+/d//HVu2bIHT6URjY2PO+5w5cwY33HADnE4nWltb8dnPfhbJZLLg805MTODmm29GQ0MDGhsbcdtttyEUCinwDop3+PBhGAyGnP8dPXo07+O2bt065/533HGHikdenOXLl885zvvuu6/gY6LRKHbu3IkFCxbA7Xbjr/7qrzA6OqrSERdvYGAAt912G1asWAGHw4Hu7m7s2bMH8Xi84OO0eu6++c1vYvny5bDb7di0aRP+8Ic/FLz/T37yE6xZswZ2ux3r1q3Dr3/9a5WOtDT79u3De9/7Xng8HrS2tuLDH/4w3nzzzYKPefjhh+ecI7vdrtIRF+/f/u3f5hznmjVrCj5GL+cNyP33w2AwYOfOnTnvr+Xz9swzz+DP//zP0d7eDoPBgMceeyzrdiEEvvCFL2Dx4sVwOBzYtm0bent7533eUn9vc2GAUoJ4PI4bb7wRd955Z87bU6kUbrjhBsTjcTz//PN45JFH8PDDD+MLX/hCwee9+eabcfz4cTz11FP41a9+hWeeeQYf//jHlXgLRduyZQuGh4ez/vunf/onrFixAhs2bCj42Ntvvz3rcV/5yldUOurSfPGLX8w6zk9+8pMF7/+Zz3wG//M//4Of/OQnePrppzE0NIS//Mu/VOloi3fy5ElIkoSHHnoIx48fxwMPPIBvfetb+PznPz/vY7V27n70ox9h165d2LNnD1555RWsX78e27dvh8/ny3n/559/HjfddBNuu+02HDt2DB/+8Ifx4Q9/GD09PSof+fyefvpp7Ny5Ey+88AKeeuopJBIJXHfddQiHwwUf19DQkHWOTp8+rdIRl+ayyy7LOs5nn3027331dN4A4OjRo1nv7amnngIA3HjjjXkfo9XzFg6HsX79enzzm9/MeftXvvIVfOMb38C3vvUtvPjii3C5XNi+fTui0Wje5yz19zYvQSU7ePCg8Hq9c77/61//WhiNRjEyMpL53oEDB0RDQ4OIxWI5n+uNN94QAMTRo0cz33v88ceFwWAQ586dk/3YyxWPx8XChQvFF7/4xYL3u/rqq8WnP/1pdQ6qAsuWLRMPPPBA0fefnJwUFotF/OQnP8l878SJEwKAOHLkiAJHKK+vfOUrYsWKFQXvo8Vzt3HjRrFz587M16lUSrS3t4t9+/blvP/f/M3fiBtuuCHre5s2bRKf+MQnFD1OOfh8PgFAPP3003nvk+9vj9bs2bNHrF+/vuj76/m8CSHEpz/9adHd3S0kScp5u17OGwDx85//PPO1JEmira1NfPWrX818b3JyUthsNvGDH/wg7/OU+nubD1dQZHTkyBGsW7cOixYtynxv+/btCAQCOH78eN7HNDY2Zq1KbNu2DUajES+++KLix1ysX/7ylzh//jx27Ngx733/67/+Cy0tLVi7di12796NSCSiwhGW7r777sOCBQtw5ZVX4qtf/WrBVNzLL7+MRCKBbdu2Zb63Zs0aLF26FEeOHFHjcCvi9/vR3Nw87/20dO7i8ThefvnlrJ+50WjEtm3b8v7Mjxw5knV/YPp3UC/nCMC85ykUCmHZsmXo7OzEhz70obx/W6qtt7cX7e3t6Orqws0334wzZ87kva+ez1s8Hsejjz6Kf/zHfyy4qa1ezttMp06dwsjISNa58Xq92LRpU95zU87vbT6a3ixQb0ZGRrKCEwCZr0dGRvI+prW1Net7ZrMZzc3NeR9TDd/97nexffv2eTdv/Nu//VssW7YM7e3t+OMf/4jPfe5zePPNN/Gzn/1MpSMtzqc+9Sm8+93vRnNzM55//nns3r0bw8PD+NrXvpbz/iMjI7BarXNqjxYtWqSp85RLX18fHnzwQezfv7/g/bR27sbHx5FKpXL+Tp08eTLnY/L9Dmr9HEmShLvvvhtXXXUV1q5dm/d+q1evxve+9z1cfvnl8Pv92L9/P7Zs2YLjx49ramPVTZs24eGHH8bq1asxPDyMvXv34k/+5E/Q09MDj8cz5/56PW8A8Nhjj2FychK33npr3vvo5bzNlv75l3Juyvm9zafuA5R7770X999/f8H7nDhxYt4CL70o5/2ePXsWhw4dwo9//ON5n39m7cy6deuwePFiXHvttejv70d3d3f5B16EUt7brl27Mt+7/PLLYbVa8YlPfAL79u3T7Hjqcs7duXPncP311+PGG2/E7bffXvCx1Tx39W7nzp3o6ekpWKcBAJs3b8bmzZszX2/ZsgWXXnopHnroIXzpS19S+jCL9oEPfCDz/5dffjk2bdqEZcuW4cc//jFuu+22Kh6Z/L773e/iAx/4ANrb2/PeRy/nTWvqPkC55557Cka+ANDV1VXUc7W1tc2pVE53ebS1teV9zOzCoWQyiYmJibyPqUQ57/fgwYNYsGAB/uIv/qLk19u0aROA6U/xSl/kKjmXmzZtQjKZxMDAAFavXj3n9ra2NsTjcUxOTmatooyOjipynnIp9f0NDQ3hmmuuwZYtW/Dtb3+75NdT89zl0tLSApPJNKdTqtDPvK2traT7a8Fdd92VKY4v9dO0xWLBlVdeib6+PoWOTh6NjY245JJL8h6nHs8bAJw+fRq/+c1vSl5l1Mt5S//8R0dHsXjx4sz3R0dHccUVV+R8TDm/t3mVVLFCQoj5i2RHR0cz33vooYdEQ0ODiEajOZ8rXST70ksvZb536NAhzRTJSpIkVqxYIe65556yHv/ss88KAOK1116T+cjk9eijjwqj0SgmJiZy3p4ukv3pT3+a+d7Jkyc1WyR79uxZsWrVKvGxj31MJJPJsp5DC+du48aN4q677sp8nUqlREdHR8Ei2T/7sz/L+t7mzZs1WWwpSZLYuXOnaG9vF2+99VZZz5FMJsXq1avFZz7zGZmPTl7BYFA0NTWJ//iP/8h5u57O20x79uwRbW1tIpFIlPQ4rZ435CmS3b9/f+Z7fr+/qCLZUn5v8x5PSfeuc6dPnxbHjh0Te/fuFW63Wxw7dkwcO3ZMBINBIcT0P7q1a9eK6667Trz66qviiSeeEAsXLhS7d+/OPMeLL74oVq9eLc6ePZv53vXXXy+uvPJK8eKLL4pnn31WrFq1Stx0002qv79cfvOb3wgA4sSJE3NuO3v2rFi9erV48cUXhRBC9PX1iS9+8YvipZdeEqdOnRK/+MUvRFdXl3j/+9+v9mEX9Pzzz4sHHnhAvPrqq6K/v188+uijYuHCheIf/uEfMveZ/d6EEOKOO+4QS5cuFb/73e/ESy+9JDZv3iw2b95cjbdQ0NmzZ8XKlSvFtddeK86ePSuGh4cz/828jx7O3Q9/+ENhs9nEww8/LN544w3x8Y9/XDQ2NmY65f7+7/9e3HvvvZn7P/fcc8JsNov9+/eLEydOiD179giLxSJef/31ar2FvO68807h9XrF4cOHs85RJBLJ3Gf2+9u7d684dOiQ6O/vFy+//LL42Mc+Jux2uzh+/Hg13kJe99xzjzh8+LA4deqUeO6558S2bdtES0uL8Pl8Qgh9n7e0VColli5dKj73uc/NuU1P5y0YDGauZQDE1772NXHs2DFx+vRpIYQQ9913n2hsbBS/+MUvxB//+EfxoQ99SKxYsUJMTU1lnuNP//RPxYMPPpj5er7f22IxQCnBLbfcIgDM+e/3v/995j4DAwPiAx/4gHA4HKKlpUXcc889WdH173//ewFAnDp1KvO98+fPi5tuukm43W7R0NAgduzYkQl6qu2mm24SW7ZsyXnbqVOnst7/mTNnxPvf/37R3NwsbDabWLlypfjsZz8r/H6/ikc8v5dfflls2rRJeL1eYbfbxaWXXiq+/OUvZ61yzX5vQggxNTUl/vmf/1k0NTUJp9MpPvKRj2Rd9LXi4MGDOf+dzlww1dO5e/DBB8XSpUuF1WoVGzduFC+88ELmtquvvlrccsstWff/8Y9/LC655BJhtVrFZZddJv73f/9X5SMuTr5zdPDgwcx9Zr+/u+++O/OzWLRokfjgBz8oXnnlFfUPfh4f/ehHxeLFi4XVahUdHR3iox/9qOjr68vcrufzlnbo0CEBQLz55ptzbtPTeUtfk2b/lz5+SZLEv/7rv4pFixYJm80mrr322jnvedmyZWLPnj1Z3yv0e1ssgxBClJYUIiIiIlIW56AQERGR5jBAISIiIs1hgEJERESawwCFiIiINIcBChEREWkOAxQiIiLSHAYoREREpDkMUIiIiEhzGKAQERGR5jBAISIiIs1hgEJERESawwCFiIiINOf/Aww4sFGT2TAIAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "model.eval()\n",
    "tsne = TSNE(random_state=10) \n",
    "tsne_repr = tsne.fit_transform(z.cpu().detach().numpy())\n",
    "print(tsne_repr.shape)\n",
    "plt.scatter(tsne_repr[:, 0], tsne_repr[:, 1], alpha=0.5)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "venv",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}