"
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ }
+ ],
+ "source": [
+ "import matplotlib.pyplot as plt\n",
+ "\n",
+ "plt.plot(losses)\n",
+ "plt.show()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "64OtEmyUWUiM"
+ },
+ "source": [
+ "You have successfully trained your model to help automatically answer questions! Try asking it a question about a story."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 25,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
+ },
+ "id": "eFniMzpp1bpz",
+ "outputId": "0ce0e2a3-3d6a-4e6e-adff-d0c16b622c9a"
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "What is south of the bedroom? Garden\n"
+ ]
+ }
+ ],
+ "source": [
+ "question, text = 'What is south of the bedroom?','The hallway is south of the garden. The garden is south of the bedroom.'\n",
+ "input_dict = tokenizer(text, question, return_tensors='tf')\n",
+ "outputs = model(input_dict)\n",
+ "start_logits = outputs[0]\n",
+ "end_logits = outputs[1]\n",
+ "\n",
+ "all_tokens = tokenizer.convert_ids_to_tokens(input_dict[\"input_ids\"].numpy()[0])\n",
+ "answer = ' '.join(all_tokens[tf.math.argmax(start_logits, 1)[0] : tf.math.argmax(end_logits, 1)[0]+1])\n",
+ "print(question, answer.capitalize())"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "f07OtnCpuKFa"
+ },
+ "source": [
+ "Congratulations! You just implemented your first QA model in TensorFlow. "
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "9UaM5pY9u8EW"
+ },
+ "source": [
+ "\n",
+ "## 2.2 PyTorch implementation\n",
+ "\n",
+ "[PyTorch](https://pytorch.org/) is an open source machine learning framework developed by Facebook's AI Research lab that can be used for computer vision and natural language processing. As you can imagine, it is quite compatible with the bAbI dataset."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "nD9akXoXxMjd"
+ },
+ "source": [
+ "#### Train and test dataset\n",
+ "\n",
+ "Go ahead and try creating a train and test dataset by importing PyTorch."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 26,
+ "metadata": {
+ "id": "JxMYWSG173ch"
+ },
+ "outputs": [],
+ "source": [
+ "from torch.utils.data import DataLoader\n",
+ "\n",
+ "columns_to_return = ['input_ids','attention_mask', 'start_positions', 'end_positions']\n",
+ "train_ds.set_format(type='pt', columns=columns_to_return)\n",
+ "test_ds.set_format(type='pt', columns=columns_to_return)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "OeuzZKlPHAAQ"
+ },
+ "source": [
+ "For the accuracy metrics for the PyTorch implementation, you will change things up a bit and use the [F1 score](https://scikit-learn.org/stable/modules/generated/sklearn.metrics.f1_score.html) for start and end indicies over the entire test dataset as the loss functions. "
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 27,
+ "metadata": {
+ "id": "aD9tDpZfJsIB"
+ },
+ "outputs": [],
+ "source": [
+ "from sklearn.metrics import f1_score\n",
+ "\n",
+ "def compute_metrics(pred):\n",
+ " start_labels = pred.label_ids[0]\n",
+ " start_preds = pred.predictions[0].argmax(-1)\n",
+ " end_labels = pred.label_ids[1]\n",
+ " end_preds = pred.predictions[1].argmax(-1)\n",
+ " \n",
+ " f1_start = f1_score(start_labels, start_preds, average='macro')\n",
+ " f1_end = f1_score(end_labels, end_preds, average='macro')\n",
+ " \n",
+ " return {\n",
+ " 'f1_start': f1_start,\n",
+ " 'f1_end': f1_end,\n",
+ " }"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "laX5cYQRHMXb"
+ },
+ "source": [
+ "#### Training\n",
+ "\n",
+ "Now it is time to load a pre-trained model. \n",
+ "\n",
+ "**Note:** You will be using the DistilBERT instead of TFDistilBERT for a PyTorch implementation."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 29,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "del model # We delete the tensorflow model to avoid memory issues"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 28,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
+ },
+ "id": "YXFCsNcY79jx",
+ "outputId": "09af112f-e1e9-4a47-c988-37ee2a068df2"
+ },
+ "outputs": [],
+ "source": [
+ "from transformers import DistilBertForQuestionAnswering\n",
+ "\n",
+ "pytorch_model = DistilBertForQuestionAnswering.from_pretrained(\"model/pytorch\")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "xCUdMmCxHP6_"
+ },
+ "source": [
+ "Instead of a custom training loop, you will use the [🤗 Trainer](https://huggingface.co/transformers/main_classes/trainer.html), which contains a basic training loop and is fairly easy to implement in PyTorch."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 30,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/",
+ "height": 329
+ },
+ "id": "1htmS3TV-2Bk",
+ "outputId": "cc21bfbb-da09-47f9-ee16-7db0096d35e7"
+ },
+ "outputs": [
+ {
+ "name": "stderr",
+ "output_type": "stream",
+ "text": [
+ "The following columns in the training set don't have a corresponding argument in `DistilBertForQuestionAnswering.forward` and have been ignored: end_idx, sentences, question, answer, str_idx. If end_idx, sentences, question, answer, str_idx are not expected by `DistilBertForQuestionAnswering.forward`, you can safely ignore this message.\n",
+ "***** Running training *****\n",
+ " Num examples = 1000\n",
+ " Num Epochs = 3\n",
+ " Instantaneous batch size per device = 8\n",
+ " Total train batch size (w. parallel, distributed & accumulation) = 8\n",
+ " Gradient Accumulation steps = 1\n",
+ " Total optimization steps = 375\n",
+ " Number of trainable parameters = 66364418\n"
+ ]
+ },
+ {
+ "data": {
+ "text/html": [
+ "\n",
+ " \n",
+ " \n",
+ "
\n",
+ " [375/375 00:11, Epoch 3/3]\n",
+ "
\n",
+ " \n",
+ " \n",
+ " | Step | \n",
+ " Training Loss | \n",
+ "
\n",
+ " \n",
+ " \n",
+ " \n",
+ " | 50 | \n",
+ " 1.532300 | \n",
+ "
\n",
+ " \n",
+ " | 100 | \n",
+ " 0.750500 | \n",
+ "
\n",
+ " \n",
+ " | 150 | \n",
+ " 0.438300 | \n",
+ "
\n",
+ " \n",
+ " | 200 | \n",
+ " 0.430300 | \n",
+ "
\n",
+ " \n",
+ " | 250 | \n",
+ " 0.418900 | \n",
+ "
\n",
+ " \n",
+ " | 300 | \n",
+ " 0.291500 | \n",
+ "
\n",
+ " \n",
+ " | 350 | \n",
+ " 0.342800 | \n",
+ "
\n",
+ " \n",
+ "
"
+ ],
+ "text/plain": [
+ ""
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ },
+ {
+ "name": "stderr",
+ "output_type": "stream",
+ "text": [
+ "\n",
+ "\n",
+ "Training completed. Do not forget to share your model on huggingface.co/models =)\n",
+ "\n",
+ "\n"
+ ]
+ },
+ {
+ "data": {
+ "text/plain": [
+ "TrainOutput(global_step=375, training_loss=0.5782912762959799, metrics={'train_runtime': 11.2987, 'train_samples_per_second': 265.517, 'train_steps_per_second': 33.19, 'total_flos': 19904183208000.0, 'train_loss': 0.5782912762959799, 'epoch': 3.0})"
+ ]
+ },
+ "execution_count": 30,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "from transformers import Trainer, TrainingArguments\n",
+ "\n",
+ "training_args = TrainingArguments(\n",
+ " output_dir='results', # output directory\n",
+ " overwrite_output_dir=True,\n",
+ " num_train_epochs=3, # total number of training epochs\n",
+ " per_device_train_batch_size=8, # batch size per device during training\n",
+ " per_device_eval_batch_size=8, # batch size for evaluation\n",
+ " warmup_steps=20, # number of warmup steps for learning rate scheduler\n",
+ " weight_decay=0.01, # strength of weight decay\n",
+ " logging_dir=None, # directory for storing logs\n",
+ " logging_steps=50\n",
+ ")\n",
+ "\n",
+ "trainer = Trainer(\n",
+ " model=pytorch_model, # the instantiated 🤗 Transformers model to be trained\n",
+ " args=training_args, # training arguments, defined above\n",
+ " train_dataset=train_ds, # training dataset\n",
+ " eval_dataset=test_ds,\n",
+ " compute_metrics=compute_metrics # evaluation dataset\n",
+ ")\n",
+ "\n",
+ "trainer.train()"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 31,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/",
+ "height": 207
+ },
+ "id": "lDzbm7vzAiPJ",
+ "outputId": "7cd62f51-a04b-4583-bc0e-e459813d3103"
+ },
+ "outputs": [
+ {
+ "name": "stderr",
+ "output_type": "stream",
+ "text": [
+ "The following columns in the evaluation set don't have a corresponding argument in `DistilBertForQuestionAnswering.forward` and have been ignored: end_idx, sentences, question, answer, str_idx. If end_idx, sentences, question, answer, str_idx are not expected by `DistilBertForQuestionAnswering.forward`, you can safely ignore this message.\n",
+ "***** Running Evaluation *****\n",
+ " Num examples = 1000\n",
+ " Batch size = 8\n"
+ ]
+ },
+ {
+ "data": {
+ "text/html": [
+ "\n",
+ " \n",
+ " \n",
+ "
\n",
+ " [125/125 00:00]\n",
+ "
\n",
+ "$$\n",
+ "PE_{(pos, 2i+1)}= cos\\left(\\frac{pos}{{10000}^{\\frac{2i}{d}}}\\right)\n",
+ "$$\n",
+ "\n",
+ "\n",
+ "\n",
+ "\n",
+ "It is a standard practice in natural language processing tasks to convert sentences into tokens before feeding texts into a language model. Each token is then converted into a numerical vector of fixed length called an embedding, which captures the meaning of the words. In the Transformer architecture, a positional encoding vector is added to the embedding to pass positional information throughout the model. \n",
+ "\n",
+ "The meaning of these vectors can be difficult to grasp solely by examining the numerical representations, but visualizations can help give some intuition as to the semantic and positional similarity of the words. As you've seen in previous assignments, when embeddings are reduced to two dimensions and plotted, semantically similar words appear closer together, while dissimilar words are plotted farther apart. A similar exercise can be performed with positional encoding vectors - words that are closer in a sentence should appear closer when plotted on a Cartesian plane, and when farther in a sentence, should appear farther on the plane. \n",
+ "\n",
+ "In this notebook, you will create a series of visualizations of word embeddings and positional encoding vectors to gain intuition into how positional encodings affect word embeddings and help transport sequential information through the Transformer architecture. \n",
+ " \n",
+ "\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "### 1.1 - Positional encoding visualizations\n",
+ "\n",
+ "The following code cell has the `positional_encoding` function which you implemented in the Transformer assignment. Nice work! You will build off that work to create some more visualizations with this function in this notebook."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "y78txxoHQtwG"
+ },
+ "outputs": [],
+ "source": [
+ "def positional_encoding(positions, d):\n",
+ " \"\"\"\n",
+ " Precomputes a matrix with all the positional encodings \n",
+ " \n",
+ " Arguments:\n",
+ " positions (int) -- Maximum number of positions to be encoded \n",
+ " d (int) -- Encoding size \n",
+ " \n",
+ " Returns:\n",
+ " pos_encoding -- (1, position, d_model) A matrix with the positional encodings\n",
+ " \"\"\"\n",
+ "\n",
+ " # initialize a matrix angle_rads of all the angles \n",
+ " angle_rads = np.arange(positions)[:, np.newaxis] / np.power(10000, (2 * (np.arange(d)[np.newaxis, :]//2)) / np.float32(d))\n",
+ " angle_rads[:, 0::2] = np.sin(angle_rads[:, 0::2])\n",
+ " angle_rads[:, 1::2] = np.cos(angle_rads[:, 1::2])\n",
+ " \n",
+ " pos_encoding = angle_rads[np.newaxis, ...]\n",
+ " \n",
+ " return tf.cast(pos_encoding, dtype=tf.float32)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Define the embedding dimension as 100. This value must match the dimensionality of the word embedding. In the [\"Attention is All You Need\"](https://arxiv.org/abs/1706.03762) paper, embedding sizes range from 100 to 1024, depending on the task. The authors also use a maximum sequence length ranging from 40 to 512 depending on the task. Define the maximum sequence length to be 100, and the maximum number of words to be 64. "
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "EMBEDDING_DIM = 100\n",
+ "MAX_SEQUENCE_LENGTH = 100\n",
+ "MAX_NB_WORDS = 64\n",
+ "pos_encoding = positional_encoding(MAX_SEQUENCE_LENGTH, EMBEDDING_DIM)\n",
+ "\n",
+ "plt.pcolormesh(pos_encoding[0], cmap='RdBu')\n",
+ "plt.xlabel('d')\n",
+ "plt.xlim((0, EMBEDDING_DIM))\n",
+ "plt.ylabel('Position')\n",
+ "plt.colorbar()\n",
+ "plt.show()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "You have already created this visualization in this assignment, but let us dive a little deeper. Notice some interesting properties of the matrix - the first is that the norm of each of the vectors is always a constant. No matter what the value of `pos` is, the norm will always be the same value, which in this case is 7.071068. From this property you can conclude that the dot product of two positional encoding vectors is not affected by the scale of the vector, which has important implications for correlation calculations. "
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "pos = 34\n",
+ "tf.norm(pos_encoding[0,pos,:])"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Another interesting property is that the norm of the difference between 2 vectors separated by `k` positions is also constant. If you keep `k` constant and change `pos`, the difference will be of approximately the same value. This property is important because it demonstrates that the difference does not depend on the positions of each encoding, but rather the relative seperation between encodings. Being able to express positional encodings as linear functions of one another can help the model to learn by focusing on the relative positions of words.\n",
+ "\n",
+ "This reflection of the difference in the positions of words with vector encodings is difficult to achieve, especially given that the values of the vector encodings must remain small enough so that they do not distort the word embeddings. "
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "pos = 70\n",
+ "k = 2\n",
+ "print(tf.norm(pos_encoding[0,pos,:] - pos_encoding[0,pos + k,:]))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "You have observed some interesting properties about the positional encoding vectors - next, you will create some visualizations to see how these properties affect the relationships between encodings and embeddings!"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "### 1.2 - Comparing positional encodings\n",
+ "\n",
+ "#### 1.2.1 - Correlation\n",
+ "\n",
+ "The positional encoding matrix help to visualize how each vector is unique for every position. However, it is still not clear how these vectors can represent the relative position of the words in a sentence. To illustrate this, you will calculate the correlation between pairs of vectors at every single position. A successful positional encoder will produce a perfectly symmetric matrix in which maximum values are located at the main diagonal - vectors in similar positions should have the highest correlation. Following the same logic, the correlation values should get smaller as they move away from the main diagonal.\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# Positional encoding correlation\n",
+ "corr = tf.matmul(pos_encoding, pos_encoding, transpose_b=True).numpy()[0]\n",
+ "plt.pcolormesh(corr, cmap='RdBu')\n",
+ "plt.xlabel('Position')\n",
+ "plt.xlim((0, MAX_SEQUENCE_LENGTH))\n",
+ "plt.ylabel('Position')\n",
+ "plt.colorbar()\n",
+ "plt.show()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "#### 1.2.2 - Euclidean distance\n",
+ "\n",
+ "You can also use the euclidean distance instead of the correlation for comparing the positional encoding vectors. In this case, your visualization will display a matrix in which the main diagonal is 0, and its off-diagonal values increase as they move away from the main diagonal.\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# Positional encoding euclidean distance\n",
+ "eu = np.zeros((MAX_SEQUENCE_LENGTH, MAX_SEQUENCE_LENGTH))\n",
+ "print(eu.shape)\n",
+ "for a in range(MAX_SEQUENCE_LENGTH):\n",
+ " for b in range(a + 1, MAX_SEQUENCE_LENGTH):\n",
+ " eu[a, b] = tf.norm(tf.math.subtract(pos_encoding[0, a], pos_encoding[0, b]))\n",
+ " eu[b, a] = eu[a, b]\n",
+ " \n",
+ "plt.pcolormesh(eu, cmap='RdBu')\n",
+ "plt.xlabel('Position')\n",
+ "plt.xlim((0, MAX_SEQUENCE_LENGTH))\n",
+ "plt.ylabel('Position')\n",
+ "plt.colorbar()\n",
+ "plt.show()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Nice work! You can use these visualizations as checks for any positional encodings you create. "
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "## 2 - Semantic embedding\n",
+ "\n",
+ "You have gained insight into the relationship positional encoding vectors have with other vectors at different positions by creating correlation and distance matrices. Similarly, you can gain a stronger intuition as to how positional encodings affect word embeddings by visualizing the sum of these vectors. \n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "### 2.1 - Load pretrained embedding\n",
+ "\n",
+ "To combine a pretrained word embedding with the positional encodings you created, start by loading one of the pretrained embeddings from the [glove](https://nlp.stanford.edu/projects/glove/) project. You will use the embedding with 100 features. "
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "embeddings_index = {}\n",
+ "GLOVE_DIR = \"glove\"\n",
+ "f = open(os.path.join(GLOVE_DIR, 'glove.6B.100d.txt'))\n",
+ "for line in f:\n",
+ " values = line.split()\n",
+ " word = values[0]\n",
+ " coefs = np.asarray(values[1:], dtype='float32')\n",
+ " embeddings_index[word] = coefs\n",
+ "f.close()\n",
+ "\n",
+ "print('Found %s word vectors.' % len(embeddings_index))\n",
+ "print('d_model:', embeddings_index['hi'].shape)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "**Note:** This embedding is composed of 400,000 words and each word embedding has 100 features."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Consider the following text that only contains two sentences. Wait a minute - these sentences have no meaning! Instead, the sentences are engineered such that:\n",
+ "* Each sentence is composed of sets of words, which have some semantic similarities among each groups.\n",
+ "* In the first sentence similar terms are consecutive, while in the second sentence, the order is random. "
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "texts = ['king queen man woman dog wolf football basketball red green yellow',\n",
+ " 'man queen yellow basketball green dog woman football king red wolf']"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "First, run the following code cell to apply the tokenization to the raw text. Don't worry too much about what this step does - it will be explained in detail in later ungraded labs. A quick summary (not crucial to understanding the lab):\n",
+ "\n",
+ "* If you feed an array of plain text of different sentence lengths, and it will produce a matrix with one row for each sentence, each of them represented by an array of size `MAX_SEQUENCE_LENGTH`.\n",
+ "* Each value in this array represents each word of the sentence using its corresponding index in a dictionary(`word_index`). \n",
+ "* The sequences shorter than the `MAX_SEQUENCE_LENGTH` are padded with zeros to create uniform length. \n",
+ "\n",
+ "Again, this is explained in detail in later ungraded labs, so don't worry about this too much right now!"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "tokenizer = Tokenizer(num_words=MAX_NB_WORDS)\n",
+ "tokenizer.fit_on_texts(texts)\n",
+ "sequences = tokenizer.texts_to_sequences(texts)\n",
+ "\n",
+ "word_index = tokenizer.word_index\n",
+ "print('Found %s unique tokens.' % len(word_index))\n",
+ "\n",
+ "data = pad_sequences(sequences, padding='post', maxlen=MAX_SEQUENCE_LENGTH)\n",
+ "\n",
+ "print(data.shape)\n",
+ "\n",
+ "print(data)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "To simplify your model, you will only need to obtain the embeddings for the different words that appear in the text you are examining. In this case, you will filter out only the 11 words appearing in our sentences. The first vector will be an array of zeros and will codify all the unknown words."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "embedding_matrix = np.zeros((len(word_index) + 1, EMBEDDING_DIM))\n",
+ "for word, i in word_index.items():\n",
+ " embedding_vector = embeddings_index.get(word)\n",
+ " if embedding_vector is not None:\n",
+ " # words not found in embedding index will be all-zeros.\n",
+ " embedding_matrix[i] = embedding_vector\n",
+ "print(embedding_matrix.shape)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Create an embedding layer using the weights extracted from the pretrained glove embeddings."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "embedding_layer = Embedding(len(word_index) + 1,\n",
+ " EMBEDDING_DIM,\n",
+ " embeddings_initializer=tf.keras.initializers.Constant(embedding_matrix),\n",
+ " trainable=False)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Transform the input tokenized data to the embedding using the previous layer. Check the shape of the embedding to make sure the last dimension of this matrix contains the embeddings of the words in the sentence. "
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "embedding = embedding_layer(data)\n",
+ "print(embedding.shape)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "### 2.2 - Visualization on a Cartesian plane\n",
+ "\n",
+ "Now, you will create a function that allows you to visualize the encoding of our words in a Cartesian plane. You will use PCA to reduce the 100 features of the glove embedding to only 2 components. "
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "from sklearn.decomposition import PCA\n",
+ "\n",
+ "def plot_words(embedding, sequences, sentence):\n",
+ " pca = PCA(n_components=2)\n",
+ " X_pca_train = pca.fit_transform(embedding[sentence,0:len(sequences[sentence]),:])\n",
+ "\n",
+ "\n",
+ " fig, ax = plt.subplots(figsize=(12, 6)) \n",
+ " plt.rcParams['font.size'] = '12'\n",
+ " ax.scatter(X_pca_train[:, 0], X_pca_train[:, 1])\n",
+ " words = list(word_index.keys())\n",
+ " for i, index in enumerate(sequences[sentence]):\n",
+ " ax.annotate(words[index-1], (X_pca_train[i, 0], X_pca_train[i, 1]))\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Nice! Now you can plot the embedding of each of the sentences. Each plot should disply the embeddings of the different words. \n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "plot_words(embedding, sequences, 0)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Plot the word of embeddings of the second sentence. Recall that the second sentence contains the same words are the first sentence, just in a different order. You can see that the order of the words does not affect the vector representations. "
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "plot_words(embedding, sequences, 1)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "## 3 - Semantic and positional embedding\n",
+ "\n",
+ "Next, you will combine the original glove embedding with the positional encoding you calculated earlier. For this exercise, you will use a 1 to 1 weight ratio between the semantic and the positional embedding. "
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "embedding2 = embedding * 1.0 + pos_encoding[:,:,:] * 1.0\n",
+ "\n",
+ "plot_words(embedding2, sequences, 0)\n",
+ "plot_words(embedding2, sequences, 1)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Wow look at the big difference between the plots! Both plots have changed drastically compared to their original counterparts. Notice that in the second image, which corresponds to the sentence in which similar words are not together, very dissimilar words such as `red` and `wolf` appear more close.\n",
+ "\n",
+ "Now you can try different relative weights and see how this strongly impacts the vector representation of the words in the sentence."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "W1 = 1 # Change me\n",
+ "W2 = 10 # Change me\n",
+ "embedding2 = embedding * W1 + pos_encoding[:,:,:] * W2\n",
+ "plot_words(embedding2, sequences, 0)\n",
+ "plot_words(embedding2, sequences, 1)\n",
+ "\n",
+ "# For reference\n",
+ "#['king queen man woman dog wolf football basketball red green yellow',\n",
+ "# 'man queen yellow basketball green dog woman football king red wolf']"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "If you set `W1 = 1` and `W2 = 10`, you can see how arrangement of the words begins to take on a clockwise or anti-clockwise order depending on the position of the words in the sentence. Under these parameters, the positional encoding vectors have dominated the embedding. \n",
+ "\n",
+ "Now try inverting the weights to `W1 = 10` and `W2 = 1`. Observe that under these parameters, the plot resembles the original embedding visualizations and there are only a few changes between the positions of the plotted words.\n",
+ "\n",
+ "In the previous Transformer assignment, the word embedding is multiplied by `sqrt(EMBEDDING_DIM)`. In this case, it will be equivalent using `W1 = sqrt(EMBEDDING_DIM) = 10` and `W2 = 1`. \n",
+ "\n",
+ "### Congratulations!\n",
+ " \n",
+ "You've completed this notebook, and have a better sense of the inputs of the Transformer network! \n",
+ "\n",
+ "By now, you've: \n",
+ "\n",
+ "* Created positional encoding matrices to visualize the relational properties of the vectors\n",
+ "* Plotted embeddings and positional encodings on a Cartesian plane to observe how they affect each other \n",
+ "\n",
+ "\n",
+ "What you should remember:\n",
+ "- Positional encodings can be expressed as linear functions of each other, which allow the model to learn according to the relative positions of words.\n",
+ "- Positional encodings can affect the word embeddings, but if the relative weight of the positional encoding is small, the sum will retain the semantic meaning of the words. \n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": []
+ }
+ ],
+ "metadata": {
+ "accelerator": "GPU",
+ "colab": {
+ "collapsed_sections": [],
+ "name": "Transformer Assignment - Subclass.ipynb",
+ "provenance": []
+ },
+ "kernelspec": {
+ "display_name": "Python 3",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.7.6"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 1
+}
diff --git a/Transformer Mechanism/Transformer Pre-Processing/home/jovyan/work/W4A4_UGL_POS/Embedding_plus_Positional_encoding.ipynb b/Transformer Mechanism/Transformer Pre-Processing/home/jovyan/work/W4A4_UGL_POS/Embedding_plus_Positional_encoding.ipynb
new file mode 100644
index 0000000000000000000000000000000000000000..7714ef655962498f7f54103007dee577718101d2
--- /dev/null
+++ b/Transformer Mechanism/Transformer Pre-Processing/home/jovyan/work/W4A4_UGL_POS/Embedding_plus_Positional_encoding.ipynb
@@ -0,0 +1,785 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
+ },
+ "id": "AbzZLqIPv6b7",
+ "outputId": "19f2fc2b-6f1d-4b43-fd50-4c513e3936fd"
+ },
+ "source": [
+ "# Transformer Pre-processing\n",
+ "\n",
+ "Welcome to Week 4's first ungraded lab. In this notebook you will delve into the pre-processing methods you apply to raw text to before passing it to the encoder and decoder blocks of the transformer architecture. \n",
+ "\n",
+ "**After this assignment you'll be able to**:\n",
+ "\n",
+ "* Create visualizations to gain intuition on positional encodings\n",
+ "* Visualize how positional encodings affect word embeddings\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Table of Contents\n",
+ "\n",
+ "- [Packages](#0)\n",
+ "- [1 - Positional Encoding](#1)\n",
+ " - [1.1 - Positional encoding visualizations](#1-1)\n",
+ " - [1.2 - Comparing positional encodings](#1-2) \n",
+ "- [2 - Semantic embedding](#2)\n",
+ " - [2.1 - Load pretrained embedding](#2-1)\n",
+ " - [2.2 - Visualization on a Cartesian plane](#2-2)\n",
+ "- [3 - Semantic and positional embedding](#3)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "## Packages\n",
+ "\n",
+ "Run the following cell to load the packages you'll need."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 1,
+ "metadata": {
+ "id": "_OpwqWL2QH5G"
+ },
+ "outputs": [],
+ "source": [
+ "import tensorflow as tf\n",
+ "import numpy as np\n",
+ "import matplotlib.pyplot as plt\n",
+ "import os\n",
+ "\n",
+ "from tensorflow.keras.layers import Embedding\n",
+ "from tensorflow.keras.preprocessing.text import Tokenizer\n",
+ "from tensorflow.keras.preprocessing.sequence import pad_sequences"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "## 1 - Positional Encoding\n",
+ "\n",
+ "Here are the positional encoding equations that you implemented in the previous assignment. This encoding uses the following formulas:\n",
+ "\n",
+ "$$\n",
+ "PE_{(pos, 2i)}= sin\\left(\\frac{pos}{{10000}^{\\frac{2i}{d}}}\\right)\n",
+ "$$\n",
+ "
\n",
+ "$$\n",
+ "PE_{(pos, 2i+1)}= cos\\left(\\frac{pos}{{10000}^{\\frac{2i}{d}}}\\right)\n",
+ "$$\n",
+ "\n",
+ "\n",
+ "\n",
+ "\n",
+ "It is a standard practice in natural language processing tasks to convert sentences into tokens before feeding texts into a language model. Each token is then converted into a numerical vector of fixed length called an embedding, which captures the meaning of the words. In the Transformer architecture, a positional encoding vector is added to the embedding to pass positional information throughout the model. \n",
+ "\n",
+ "The meaning of these vectors can be difficult to grasp solely by examining the numerical representations, but visualizations can help give some intuition as to the semantic and positional similarity of the words. As you've seen in previous assignments, when embeddings are reduced to two dimensions and plotted, semantically similar words appear closer together, while dissimilar words are plotted farther apart. A similar exercise can be performed with positional encoding vectors - words that are closer in a sentence should appear closer when plotted on a Cartesian plane, and when farther in a sentence, should appear farther on the plane. \n",
+ "\n",
+ "In this notebook, you will create a series of visualizations of word embeddings and positional encoding vectors to gain intuition into how positional encodings affect word embeddings and help transport sequential information through the Transformer architecture. \n",
+ " \n",
+ "\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "### 1.1 - Positional encoding visualizations\n",
+ "\n",
+ "The following code cell has the `positional_encoding` function which you implemented in the Transformer assignment. Nice work! You will build off that work to create some more visualizations with this function in this notebook."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 2,
+ "metadata": {
+ "id": "y78txxoHQtwG"
+ },
+ "outputs": [],
+ "source": [
+ "def positional_encoding(positions, d):\n",
+ " \"\"\"\n",
+ " Precomputes a matrix with all the positional encodings \n",
+ " \n",
+ " Arguments:\n",
+ " positions (int) -- Maximum number of positions to be encoded \n",
+ " d (int) -- Encoding size \n",
+ " \n",
+ " Returns:\n",
+ " pos_encoding -- (1, position, d_model) A matrix with the positional encodings\n",
+ " \"\"\"\n",
+ "\n",
+ " # initialize a matrix angle_rads of all the angles \n",
+ " angle_rads = np.arange(positions)[:, np.newaxis] / np.power(10000, (2 * (np.arange(d)[np.newaxis, :]//2)) / np.float32(d))\n",
+ " angle_rads[:, 0::2] = np.sin(angle_rads[:, 0::2])\n",
+ " angle_rads[:, 1::2] = np.cos(angle_rads[:, 1::2])\n",
+ " \n",
+ " pos_encoding = angle_rads[np.newaxis, ...]\n",
+ " \n",
+ " return tf.cast(pos_encoding, dtype=tf.float32)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Define the embedding dimension as 100. This value must match the dimensionality of the word embedding. In the [\"Attention is All You Need\"](https://arxiv.org/abs/1706.03762) paper, embedding sizes range from 100 to 1024, depending on the task. The authors also use a maximum sequence length ranging from 40 to 512 depending on the task. Define the maximum sequence length to be 100, and the maximum number of words to be 64. "
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 4,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYMAAAEKCAYAAADw2zkCAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nOxdZ3hU1dZ+dya9kgZJ6L1IVRTFBioqKqLXhl671/Ip13LtXa/Xeu0VsVywYG+oCCiiFBVFBUGa9JKeQHqbzP5+7HP2WsOcycyQEDJhv8+TJyvrnH3OmWQye9V3CSklDAwMDAz2b0Ts6wcwMDAwMNj3MJuBgYGBgYHZDAwMDAwMzGZgYGBgYACzGRgYGBgYwGwGBgYGBgbYi5uBEOJ1IUShEGIl06UJIb4WQvxlfU9lx24XQqwXQqwVQpywt57LwMDAoC3A6TNyt+NCCPGs9bn4hxDiQHbsROuzcr0Q4raWeJ696RlMA3DibrrbAMyTUvYFMM/6GUKIQQAmATjAWvOiEMK1F5/NwMDAYF9jGnw/IznGA+hrfV0B4CUAsD4bX7CODwJwrvUZ2izstc1ASrkAQOlu6okAplvydACnMf27Uso6KeUmAOsBHLK3ns3AwMBgX8PPZyTHRABvSIWfAHQQQmRDfTaul1JulFLWA3jXOrdZiGzuBUJEJyllHgBIKfOEEB0tfWcAP7Hztls6HwghroDaJREJcVAHRAEAuow4AACwvqhan9tQ59ZyH1kCAPgLaVrXMT1ey3l56m8yrDPpdqzequXC+GQAwIjeHbXu9w2FWh7Qp4u6/zb62w6IrtKyu65Bfa9u0LrEAwZqedmaLQCA4QO6ad223//UcmSfvgCA4qJdWpdaVqTlrkOt4/W0v7vXrtNyfkIHLXfvql5Daiw5X7+v2ablLnWVAICGXn20LmLDei136pkBAJDuRq0r2r6TnmuIMlIKltPz13btSevz1WvdnJChdYMS6rS8Yqfqij+Q/a5/Y7/rA/p2BQCs3phHz9w5U8u5heUAgMTkOK2rqaH3QkSEUN9dQusa6ui1JCSo91RFeY3Wpaclapn/DXKy1ftpxw76W9i/XwDYvLUAANC7R5bWrd9Ez92/Vw4AYO2GHVo3sA+99Vev3w4AGNS3i9at+mu7lu3fxZ9/0d9vsKUDgJXr1Ht4SH96X61YS+/roZb+D65j78E/1mz1qwOAYZZ+uYNud7393l7moON6J53/9d3ZuVv86gBA1pQUSynpjbIHiEjuIuGuDepcWVPyJwB+8lQp5dQQbtcZwDb2s/256KQfFcJ1HdHam4E/CAedI0+G9cucCgCZIkaegWwAwCMLFgEATn91qT5327piLX/kfgsAcILnHK279qKRWv7Pg28DAJY8eJDW3XboP7X83IBj1fHPrte6xNOf0/KMTx8BAJx80/taN7PbYi3vWp8PAMhfRh9qRy2m4x1GXw0AWLDoBa27KYE2i9Sn1XVfnfql1p31OZ37zKwPAACvbKMPwKKxx2n58YPOoNf6lHpdZ/ShD7iko/6l5Rs3qOfKf5peS9yZp9JzPXIFAKCumDa+V2/+WMunzf0eAPB01hCtW3PTK1q+4RH1Wi8ddYnWzT1sk5a7vaf+f5Z8eq3WxZ/2rJY/m/MkAODASY9q3V0PXa7l+575BgBw+Di6/6pV9GEda33YxydGa13uRtrMDh6lPngXzPlD6y4+f7SWX57yhZZvu/0sAMDtd72qdQ8/ebWWL5usnvXF127VulMvfEDLb7x3PwDgqLPu0rp3P32InuVUte6jLx/TuqHjb9TyTOt3MeAE+vvN+vopLfc6Vv0OZ8+n92r3sfS+nvud0ncZQ7p53z+v5eyjJgMAvmW6LEsHAPMXKH3HI0n33UJ6X2YecY2Wv7fe2+mH++q4nv8PpLFzbT3XLVr8opbt/yEnHQA0LPsf7Qx7CnctIvufGvg8db9aKeXIwGf6hb/PxaA/L0NBa28GBUKIbMsryAZgfzJuB9CVndcFQG4rP5uBgYFB0xACIqLV0pn+Phej/eibhdbeDGYCuAjAI9b3z5h+hhDiSQA5UAmTnwNdLCszHjeeoTbeb3ofrL4veVsfjz/jTS2/8scyAMD0NeSlHRFN7vrtRcrrWt6RrNFGRuLnilYW9+K6TlqX2mOwll9cvBkAsGvbaq3LmXyyllP++A0A8O1csqDH5a/Ssv0G21pOYSQWxcCgbBWmqisjbyc6gp1QqIyerSXkYmdF0Zs2MjZBy/mVKiQTwUInHDWNHgBAGrOcG9m93JUqjBSVTCG1eg/9ruobpfX8bE0DhWFc0SqUJT1MF0v3sj1r6eI6gvV43jpmF9m/S7dnz4wll/VavZ6PvX6ujxC+RprLyW4LAREOhp+TzmBfQCAi0vl9uRcwE8BkIcS7UGGgMsuQLgLQVwjRE8AOqOKb85p7s722GQgh3gEwBkCGEGI7gHuhNoH3hRCXAdgK4CwAkFL+KYR4H8AqAG4A10gpGx0vbGBgYLCv0IKegZ/PyCgAkFJOATALwElQBTXVAC6xjrmFEJMBzAHgAvC6lPJPnxuEiL22GUgpz/Vz6Fg/5z8I4MFQ7lGe3hXfXvwEAKBgmrpszUi67RlPvqHlTlerOPbw757Wul8/+EHLfY6+AQBw3Yzfte6u7pR0nTnsaADAv78ky7/3yAO0vGCxsswbqsq0zjOCPIPklHQAwLaa97SufsUiLcemqLzWHwUVWpcWTW+6ARnKsm+oqdS6FGb5N2xTyeItu9K1blA0JZMj4yg/kL9LWd7cM+BvcNvKz4mL0roqF12rvkIlxuM7Z2sd96IaPL6me6ObjkdYr8vL8o6ie9l6GeH89rTvxZ+Z31O4lL6ReQYRXpa9vZ7puBcY0fJFdhHGsG8XEKD3V3PRxGekfVwCuMbPsVlQm0WLoa0kkA0MDAzaPoRAROvlDFoVYb0ZbNuah+smPwwAqP7zXQDALf3P1senD6ScQPUzSj/1OqqQKWClp889fygA4G9XPaN1Rz1E1xofqUo3P5wxX+seuv10Ld9wyxQAQGQsWeA/5FKZ6+jeqhqlhgW3i34kLySx02EAgN+2Utlitxj683RJVnHKxnoqd0yOJAu2eqvKeeQ2DNO6NGbZR8enaHn7Tuu5qqj0lcdBay3LOZWt97BAeH25Wp/Yh/IQDcyyrrbyAzx27pUzsDwaT0M96WId4rDsn457AY45A+YF2Od66+hc7Rk4xPv9weXHtG92fiCEZzBoG2jFBHKrIqw3AwMDA4NWRetWE7UqzGZgYGBgECQEBCIiowKfGIYI680gLiUVg075GwBg2BTV6Tn/CQrdvHjszVo+cJlKFufWUulpHEuKHr71KwBAdQmV65afSI06N1sJ0Kn/oQT03wZQo9NVO1VTWUa/g7Xu3d+oqzQpurt1TwoL7FiyUctpI88EAPy6iRq5jkqg0ElaBHXo2siMIQulzOpgrUymhseEThTGiUqgMFGBlUBu3EkNcBFRdC87gZwSS2/6ChYmqa9QYSJhdWUD3qWd1Q0qjsNLSxtZbMeptDQi2vetGEoCud7tkLTmYSL2LB6HklPp4Qlk39LSUNCaoR8TZmplGM/AwMDAwAAwOYM2iQEdgB9PVRZh0lWqyuqrl27Xx4vrP9XyBQ8pioLZF1OCta6crO1vL1UlqpmHUSXXzZ9TGelrvVXpKE8Qxy8nWgLb8u48qLfW/biUPIMuqapprStLyuavogaynDNVGWv+VipNTeurGb4RWbIZgPcbMTMlVsvlWxX3UlU3ek2JHakpLD4xRsvVlSpx27iTKBpcLIFsW94pLIG9iyWrG8pV4jkijjwP3nTmmECuJys7MlZd18sziKTXwh7KVwfA4+AZBEogOyHCX1LYQe/VdNbInlt7EQ5Zbb4+gAXPfxdOpwZyAAJ5McaBaCEI0WKlpW0NYb0ZGBgYGLQmBIxn0CaRu2YL7jlUWfJvLV0IALjsBiKpyn3zSi0/+tAcAEDmghlal1xdoOWXssYCAE59nMj/Pn1voZZXC0UQ1/nAK7Ru03Siu8jop2L+x40kdklOKvd1prLS70gnIrmVxVQmOnagYrp88WfKI3QcQkyX7i3KS+HNY0lMLt+umtVqUumaiTlJWrbJ2QCgxqKjaCwr0TpXNFnmtpWfxHISsSy/0lBl0Vkk0PV501mtFb/ndBkellRw6aYz1igW4+sZcDoKrwazRl+LP5BnwL0AO3/Bm86c8wh7ljPwV4Zq4Iuw+02JCC8vuj0hrDcDAwMDg1aFMJ5Bm0RilAtHZivrOOeBywAArugj9PGn0k7Tcv9xfwEATnySKCBOOpRI3fpZpGwXn9BX6157kBrQFm5UFvuEN+j4iivWavmA+3sAAM4ZnkP3z92g5dwNvQAAXQ8nz2He+5STmNhd8eL/t5C42jOP7Kfl2g3q3NhkmgGQkkrVPFsWqKazum6UB0gcRNTtcfGUMyjarvIStSWUn4iMIy/ENrzjozidBb1V7FyLYOR33Fiv0TkDZo27eTWR5RmwRjQ4WVt+/uls6gkeu+XVRBTH594Ci/kTF6AjAuUMvPQBgvGeAN6FKwDzxZ54J6E004VyroEqLTWbgYGBgYGB2QzaIuIGDsCQbxcAAP6Truik31o9TR8/8x+PaHnVZ2qgSN/jiKJ6+0o9XxpLb1I5g8gPaU1ipx5aXrnsWwDA3UfRxK5nd1J8/p9jVBXRoHiq8/e4iW5h52ZFKtj9POpDqJxBRIODMlUuobacKoziBhym5R2ffK50qWO0LrkbTdQqqlaeSz0jyktk07+SPeQZbFur+gRqS8q1LiqWpprp+7MKomiWc6ivUqa1jKL8R6MDHQU3enmfga4mYtPFRLRDzoD1GTjlDPxVE9nw+PEMbL0/WupACHSuU+0/1+1pLsKgDcD0GRgYGBgYwISJ2iZWbS7E8IvU2L21z6nxgzXv0PjAlM5k+cvHVNVRx0FHaV3hKho7mfA/dZ2FAw/VuqH/og7klG+nAQAyV36udWUNZO0ek6MsZ88ioqi2aakB6mxOGH2x1rnENLp+hYr5uxlFdURP6onYtV6NVUzsznIGbFZukWVlN1STtR+TQzmR7Eqy4pdVOeUMqDLJRgLLGURxz8DqU5BRDr0BoJyBVzWR27cDmcPZM/CXM/D1AnjOwPYCOC21EwLlAfwNt+HYGx3Aphip7UII4dWt354Q1puBgYGBQavChIkMDAwMDACTQG6TEBERiLLoIW7veCIAIPY6Iqqbve1xLU/tqeTnfqUE8fVPUujkvLfUjOQ+uTRp7IVJI7S87XlFN/HnE69rXVYs/fo8304HAGydRY1qqb3O1HLe74oOoyKbQj98UlnjKkWkx0MTVSkU5tm5Uc05SDmQKCZiulIyu9yt1rnrKKkd2YlmZndhyd76ahUeqimhkFR0KpWJ2nA10DyGaDYPuapA0VF4J5BpXWWtmhMR54eozk4g87JLEeXLBMlnGPMy0kaPbwLZ7UBKx0tLvRO4TSeQQyktdVofCoINM/m7fWtVhvq///5HyuePxiTc0fLz/QwMDAzaKYQQEBHBfQVxrROFEGuFEOuFELc5HL9ZCLHM+lophGgUQqRZxzYLIVZYx5a2xGsLa89gcI8MfDdNNZtlHqESxO9lkuUcfc9FWo61/jgnrCUK63uvpklm9qSyIYxdre/G2VrudN0xAIC37yJyusO6UtPXqtcUUV7hCqKF7vFPakArWq08mGX5NF2Mk9aV/qL+nnzi2NZyKk0tyldWfDajmIjqQqR4lVYClU9Ci8gkzyLHQ1a8naSuaaAy2JhsX8s8opaS0by0dJdl+cvoeJ81AFBjkdKlcAoIp6azxqZLS/0RzdkJ5Ag/paW25eavtFS2JPWEw/+8o86YXe0Grhb4YwohXABeADAOwHYAvwghZkopV9nnSCn/C+C/1vkTANwgpSxllxkrpSxGC8G8RQ0MDAyChUBLeQaHAFgvpdwopawH8C6AiU2cfy6Ad1roVTgirD2D8hWrMbvnQQCAkx5SsfxTJtytj//TIp8DgPsfnQAAeP3SV7Tu6l+JuuLqCrXhnjyCaBl+uZVKSw+eqUpGV1/3kdadcQOVqc56VuUKilgj1aTDe2h59SIlf7mKyPGOZRTT+UsUXUZ0AuUpVuRT/qKoVl13cGcaUuNJo9JVe7Yyt3AbE6kMtWMls6LrlUdQW0GeASeysy1bUU9eDC8tbai0ms4iqZGNo7reqbSUvJxIa96xF22zAx0Fz0NEeDWd2URzjI6i0aG0lLFKO/1z+i0t1Z6Fcx7BNI3tv1CspS2SM+gMYBv7eTuAUU4nCiHiAZwIYDJTSwBzhRASwMtSyqnNfaCw3gwMDAwMWhcilER2xm7x/KnsQ9vpIv4aYyYAWLxbiOhwKWWuEKIjgK+FEGuklAuCfTAnhPVmUNsosbZCWZwzkuYDAI567RB9/AFG1FZ4wX8AAOtup5j/jxfTWMwB49RQnFHH0OZ8/8n/0XK+NacmLZqs0ZwLLtPyyv+oaiE+5OX8vmSZP9tD0T0sXJGvdX8f3knLuT+rprT4gyjPsGTzTi0n16s4/eAcNmoyJQ1NwZ1A9++USFa+XXHEh/vEshGbdtNVRC15JjHJlHOotazwBuH89qmsU8/Ke8s8DeQZRCT5rnOisA6UMwhER8GbzngFiD0cx9/AmsgQLD+nypLmVr3s66KZ5t6+nRbbKIiQPINiKeVIP8e2A+jKfu4CINfPuZOwW4hISplrfS8UQnwCFXZq1mZgcgYGBgYGIaCFcga/AOgrhOgphIiG+sCf6XMvIVIAHA3gM6ZLEEIk2TKA4wGsbO7rCmvPIPuAXrhz5lsAgGt7/w0A8Ouwv/TxI375TstDbp4LAPjgNKKFnv7ZOi2/9s5oAMDmGNofyxru1/Lj1gjMf/fqoHVbOhygZZuojVeSdKnepOWe/ZWVvmophQmzD6Fn+W62IppLZL0ByzeTVzjUor4YkEH9AMU1TceuS2vcWk6Pp5i/Hb+vKyNrPZYdt2P9nopdWheVTPetsTyDOochMwBQU297Bs45A1esyjV45wx8q5n45bkX4HGgmWh0qBzy+Blu41RNFAjNHVjjz1vYG16A06Maq69lIATgimz+H01K6RZCTAYwB4ALwOtSyj+FEFdZx6dYp54OYK6Usoot7wTgE6vHIxLADCnlbDQTYb0ZGBgYGLQ2WqrRTko5C8Cs3XRTdvt5GoBpu+k2AhiGFkZYbwZ/FjVg+Et5AICpY7oDAH4+epI+fvB932l5yw+KYK7/j9rbQkrqYC0PWq76DyZspt/xtZ2IvO3dRT8BAEZeO07rnvudQny9rZi712D4+VR5NGH4xQCAHz6kv32Hi4iiOrdWPV96FvURlORRh7B93ZxEsqC3ldOUFtsK530KO1llUwYbTmNb5OVsuExGMlUG2d4NH4sZFU8xfbtyqd6PZ+BUTcS9ALuayAsOlUkNDqMsAWcK63o3G1Lv0IEsWG24rfeXG3DpaqTmVw3xXERz0J7D8OEEIUS77UAO683AwMDAoLXRQqWlbQ5mMzAwMDAIAWYzaIOoqyzHhoVzAADutz8GACxOJjqGxHH3aLnnEacCAMY986PWvTJpkJa/vvwlAMCykedr3ZH3nKLliqlqnnHCuUR098l/f9Xys4NVGWt1Cd1//cc/aPmYqf8CANTspKYz0Y9KU+3ZCF26UOnoz/NpRrKNDqDrL9xFSdlEayqZK4ZKQHNZ6WjvDr5hmEpGEZGWSMfjrJCKp5ISyNHJ1CBnh4lqOcUE+/+oCdB05tJNZ6STLgp/2eEf/6Wlvk1nTglk7xnIvtcJiajOz6SyiBBmH+wNmMRwK0O0HcK8lkZYbwYGBgYGrQkBgYjI9rkFh/Vm0LN7Fh6bqsj+Trr4YQDAO+vn6OOH3/KClp8/ZzgA4KBTb9G63l+/quVnpqlmtfLtVG4adRGtj33vPgDAx1soabv199+0POSSowEAJX9u1LolM/7Q8plRig6aW5AlsTTD2MahvdO1PP/9rVq2rezIks1at7GUylyTrTdoZDR5BjsY3URkLeNmsMA9g+wUShDbVrCngpreollpqZ3MrvdjudfZTWecopt7BlH22457Br5vRe/SUvoHDDQDWZeW+ms6c6CwdoKTB7CncPlZH+GQGnbS7S2004jH3oNovxTWYb0ZGBgYGLQ2WnOGQ2sirDeD2LzN6POAirtnDbkCAPDVog/18W9Oo/h74TRFN9Gh20Ctu3k+xe+HWZZxhx5UbnrP1xu03G3kkQCApz7TDLOoyKPjiRPU9eN6LdK616b8ouUzV34HAIhKIKK5lYU0PMaO+Y/IoeO1ZcU+xxs20/3XFA7R8ugYl3V9es1bSuj6EdnUs2Jb1DXM9M5MopxBjZUAqN9Fpa0xafRcuunMTeu55euuV8ddjLrDK6aucwZEdwGXE1Gdc2mpbfHzgTduhwYz6Y/C2qFpbU9j/k501e3UcDSATVS3r59i72CfvCwhxA1CiD+tgQ3vCCFihRBpQoivhRB/Wd9T98WzGRgYGPiFFSYK5ivc0OqegRCiM4BrAQySUtYIId6H4uUYBGCelPIRa+rPbQBubepahTtr8dIHquJmVd5QAMCOxKP18Rkjz9XyhioV639oIV3y1gfe0/Kqe8cDAL5JOVzrPviQCAev/Yeiu3744Rlaxxu8lrtVNdHgQ4iSvLSe6CyKv/8eAJCcTRQWCzdSU1eONQqyXzobQlNLlnmqFX+vWb9G6zbX0tjL02NVNU5MIpHXcc9AVPh6BrUeyhmkskE79iSW2l1kucd1767lBsuyrqwnugtuIdsjLiPZWFAvIrg438om52oin9MAEF21d9MZp7C2rhmAdsIV4WwLOVcTNXmpgGivFSj7HwQi2umkon31qiIBxAkhIgHEQ7H1TQQw3To+HcBp++jZDAwMDBwhjGfQcpBS7hBCPA5gK4AaKBKmuUKITlLKPOucPIun2wdCiCsAXAEAXdKS8fB/FP3Ec32VZT/898X63OVPEh21XY3z913ztO6qrVTH775EDbJ5ilmTfZ57WctXjDwHAHBnwWaty+h3sJZfXKxI6S4/jCzoOGZObvlWxfozRlLvwqK1RVq+wYrZd4qkChuOzhadROlqun9ZMss55CjqjNgUqkbaVkzeQGNJnpbtXgSeM8iIJy+n1nruOpYzEPGUi7CriKobeJ8ByxlYNBcuxmHt1GfAISN834puP3QU3Auw4VVNZFdDOegA8hgCDawJhkqitSx+41m0HbTXprNW9wysXMBEAD0B5ABIEEKc3/QqgpRyqpRypJRyZEai8wxeAwMDg70BIZQREcxXuGFfVBMdB2CTlLIIAIQQHwMYDaBACJFteQXZAAqbuoiBgYHBvkA4ftAHg32xGWwFcKg117MGwLEAlgKoAnARgEes75/5vYKF7ZGpuDVdhYlGRX0KADjnxmn6+OonTtdydZ6aDfDOOY9pXbdzHtTyuf9TyeKvei/XurhUmoccPe8VH12vAwdoedHiLQCAxBj6lQ5gFA+bVqiQUL/zafrYmpW032UNV1GxqEJqeuMJ6swMFdrZuZ5CSxU9iJoipYtiO41j96zcRU1njSU0Yc1uTONNYxlsnsEuq4y1biclkCMSiE3Vji5VM9ZTTj3htugovBLIvIErmhrY2EP5qBoancNEjUFOOuNwiuEGmoHsb730+Mls2+sDhHS85jg4nBooIhSITdVElPYeBMLT6g8G+yJnsEQI8SGA3wC4AfwOYCqARADvCyEug9owzmrtZzMwMDBoCkIA0YaOouUgpbwXwL27qeugvISgUZJXiGkPPwsAeG7rEgDAv059VB//6ogbtHx4N9U09cuj1Kj10FWHavmyyU8CABaXfK91g/5OpHS/P6HKRHscdpvWXX5cHy1fe7MiulvA5gKcOZCSufOWqQa3U4fRjOMfvqKmtKyDewEA6lb9rHUxSVQmmpqpqCd2biTyuOoOJCd3V+cmsPtXMaK6+iLyQiJjlWXOm7oSWYNYnCXXMzqLiCRq+7DX2bOOAe/SS7eVWI70mqHASj8d5h1L1nRmW/wNfI3LN4HszzOwrfjGRl5u6ktH4XX/PW0601PVmrbWA1Uj7unshGC7YdtC1+y+f4LmQ4jQZmSHE8K6A9nAwMCgNSFgcgZtEskdMzHmGkVD0XeymiX9xCNX6uP/vIUmyHUZrryA146lRq2DGolI7qI6FX//ci1RQDw2abiW5/9HlWZe+hDlCU4bQPH/y3eqmHzeGor59zmV1r+75Et1n+5ELldVRPOQM0YdCAAo/Y2eKS6V7pXaRTW1rfuFSkRryyl/kHxgNgAg0UVWd+H2MrpXHptaFqeuy4ngElkZaIzlXdSVkWeBGF+iOp4ziGKWp8ey3KN4zqCaxcmjfT0DMCvfhr+cge0Z8Dh+o9vXC5DEKehYRhoa7UTw55oy0HYM0X5zBu0z+GVgYGCwF6A8g4igvgJeS4gThRBrhRDrLdaF3Y+PEUKUCSGWWV/3BLt2TxDWnkEvVwVmJM0HAHTOVfHtM5dRI9mtrGpl3Tw1/GbEdx9o3cIxJ2t56On/UcKvNKN4dPUyLb9do8zMJw+kmH/k0k+1bBPQlTEK7I73/03LjXcoz6CLhyz0uopSWj9EtVoUTZupdUlZx2u5Q9/tAID8WqLNrmcU04k9uwEAulbTa17Drl9TROdGxRPpnF4fRW/eaGvOcl05o5iO8a0AKmc5A69qIstj8MoZVDbtGXgc6Cga/FTtUDWRM/mcprBmuYFQBtY4zUD2Z+0H8gIC5QLaqZHZrtESnoEQwgXgBQDjAGwH8IsQYqaUctVupy6UUp6yh2tDQlhvBgYGBgatiQghWqqa6BAA66WUGwFACPEuVDNuMB/ozVnrF2G9GWzfWISbz1WUET/mqVj7w10pTv/WHzTi8uonFwIATnqTiN76LycK6+mzFXXFlreoambZ7VRNZBPJ4YtntW7jHBprmTlAkeLl/krDdSp7jtZymlWh07iM6DC41ViZpiqTiv6knEX6uVTbn9BPxfmL6z/WuoYaoouI6qoI9roXU1d2XSV5BtUo13JMKlFL6PVu6lmwcwZVBURnIaOIQM8O5RMwzEUAACAASURBVFfWkmeQyOP3DkR1vNrGqZrIi3rCqhzylzOos6uJHCgmAD6K0tkzCJQzCGXoTbAIJY/gdPvWTEM43781B+60bXcphPxRhhBiKft5qpRyqiV3BrCNHdsOYBR8cZgQYjkUf9tNUso/Q1gbEsJ6MzAwMDBoTdh0FEGiWEo50t+lHHS71zz/BqC7lLJSCHESgE8B9A1ybcgI680gPSkaF45S1UG5444DAPRLpHr1wW/doeW371Cdx8eeo3MwGBtLcequ3yqLv9vD52nd4+dTNdLY4Z0AAL8+SY3RBazmf8gDKmZfso56A77dRMft5yqcT8NvXNFkxa8rUTmPvAKy9nt1o9h+dC9Ffc1HVTbWkzUvOqr7d5d0zYYq8gaqa+nc2M6+3b4RNVR5FJOsjvOeBk9Mos8a7hmkeXUgq/xKJKPF5qRvTjkDpw7iOtYnEBGgmsjjMMjGH4W1k2cQsE/Az/+/k76dMhwbWGihaqLtALqyn7tAWf8aUspyJs8SQrwohMgIZu2eIKw3AwMDA4PWRAs2nf0CoK8QoieAHVAzXc7jJwghsgAUSCmlEOIQqOrPEgC7Aq3dE5jNwMDAwCBICLRMAllK6RZCTAYwB4ALwOtSyj+FEFdZx6cAOBPA/wkh3FA8bpOkKptzXNvcZwrrzcDTvTdqp6hS0VkDDgEAPLaaJpFd358mnd3f+b8AgPh0Kg2ddPlBWv7sWrXu6HVEEZFb+7yWD/qXmmD2+IVTta6chWwmH90bALBsIc1QfuvnrfQs/VT4aMt3VHoal3qCln/Ypko/K6sp9DKqN9FZuNNV+IeTy/FEpjulMwCgM1vvZmGk6iqafRCXpMJAPMThHSZSCeT6aura4glkGxV+Skvt2QWRLGTnRc7mECZioxF0SKjBi06CUU9Ih9JStl6XlsqmS0v9wam0tL2ibadq2x5CzBk0CSnlLACzdtNNYfLzAJ7ffZ2/tc1FWG8GBgYGBq0JQ0fRRrF+cz5OuUSVfxbPuQ8AMH4uWXNXZVIy9eWbPgIA3Dl7ttZ1Puh6LS94ViVov/icSnXHs3nEOOVaAEBu7YuOzzI2S7mO3QfTpLM/lxNtdPcx/QAA3079SesSJxA1xjd/qjLX/ozi4YgsKgEtczU9yKccytrOSSZrvrGOeQbFJNtkdtyaRyU1w8V0UMlinqyubfRNxlbW0r28PIMG5Rm4Yok0z8sziPH1MpymmjUEmHQmHMpZAUos+y0ttZLZ/mK/gSisvfQOZYZOOn/ViM2tonR6rNbMX7fTz0X/aEHPoK0hrDcDAwMDg9aEmWfQRhEVl4isIarZ6rhfFQHckhmv6eNfLP9Cy4u7KHbsh2Mpz3L5pxQnH5OmrNXbPqNGssevH6Pl15cpgriurFySU0C7v5kGAJh46Gla9/DDlL/o+K8jAADrnlygdZldM7W8bbPKGXRm1nifNLKg8yqVFc4tcD78psjKFaTxRi82d5jTTXdI8vUMGncS6V1MB9XsFsgzqGClpXzes84ZOMw6BgAR7esZOJWWBhpuE+Gn6Uw41HY6eQGBZiCHgmDmJRu0D5jNwMDAwGA/R4QZbtM2cUB2HH66eygAIGG8GmHZdyyNujzylQ1afvMfiiL6s1Nu17qZI87R8rOPq8FqVzxLYy/Tpz2p5VceVB7DSwdna11VIdE1rJ6maChOe40otO9iFNWu4RcCAErrH9C6Pr2pQe2Heb60IpmCrr+4WFnhieyN6GKx9y27VE7g6O6+JHQAUMbKdbqkqvxDHLOgG8soZxCdrI7XMMu8hq23nYBKL8+ArqU9gwSqGpIeRnpnkdJxa98pP+A13MYrZ9Bo6fx4BhG+upDoKDSdhTNRXaBBOKHQXTvBaXX7/PgJQ5icgYGBgYGBgGj2Zt9WEdabQeHKv/Bc3/EAgLs+/hwAcNMRVM3TYfTVWk76bjoA4JspB2gdHy5TPEF5AUkfEBHdU0spjr75Z0UjMeIG8jxKlxPp3ZdPq3GZlzZSBRG3ILdHqJ4BXtt/zMBOWp7zlqKu5rF3V/5aLa8p6AgAyGDjKaPjqdpos+UZHJvBJrowcM8gu4Oy2LmF4y71zRnUsAqdGrdvTN/NKp9czGNpdMwZMM8g0jeX4PbqE1DXCpQz8BplyfI3duWPPwprp/xAILpqf3QUwSLCT0W/P31Lo50as/sEbZ1Ib08R1puBgYGBQWtCoPmGQVtFWG8GLgGkWENZzvpeEdEtuWWlPj7sNIrPn/y4orB+6hDqQP5+9AQtX/Cq6jwefuJRWvfKezSCssYaa+k+/m6tyx6yQssr/z0XAFC/4EOt493Oi7aqyqXMGPqVH9aVRmDWlinq6pQoRtX85xItLy87EgBwBqsWikminMOavAoAgCuL1fMza7qKWfndUpRnUMHi/LUlVFllD8qp9TjnDOwqpIY6uhcfZGP3GUQlUE5Deuj60uXrGTTKADkDly+FdYTfnIEV8/fqQPaNurdE7NdY3PsZhP+ek3BHWG8GBgYGBq0JASAqiJGW4QizGRgYGBgECRMmaqNIHzoQ5y/4DgBwXdIwn+OLvx6k5cRxao7BYbPf1rqnd9Fc39MvVjOQl37yH60bcfKNtL5TDwDAlF92aN0ZBwzUsh3m2PDpQq3L6PcPLX+6TK07g5G39Ukluga7HDOHhYFK/6AEcm7MCABAx470zDEp1LT2V4EKE3mKaFKaizV38WRwxwR1XxlJ7+raEpp9kDxEha84KV5lPZWR2tUU7noKE0Wx59alm36azhBkaWm9219pqU1HQZf0ChM1d9KZE8VDSJPK2umnhQEghAkTGRgYGOzvEGi/m31YbwYrtu5C38mqJHP+tYcCAGqKK/TxuUPHa7nTscrKP/8rKqF8szclaO0EZe81n2tdZBxN9+o7+jAAwPTPqZyUN131TlBW8Pr5W7Su331ZWl63RlnsPYeQNR+bR8lum1qiG6OgKF5JFNi7uqkGtNRelHRO7EAzkkuLFUW1u4DKZSNZUxq38rMTlWdQy5LVPIEckZTqs6asjnsG1jWZLiqBaDpsL8cV5ztrGQBklC+Fdb07eM9Al5ayf0peRupkuTk2hzmQ1/H1kiWw/SFQzbnthYRCVOevwS3Y9QZ7FyZMZGBgYLCfQwggqp3ONQ3rzcBTX4eK3PUAgK+ufBgAcExPGgjzVc5QLb90h/ISzrn8Ea2bX/i9lkdc9gQA4Ofb79K6fseQfN0ElR+46obntO6zaHpTvDBMNZDNXkqjSM8f1U3LN877DQDQ7egBWlf3B+UX7DLRjEwqFy1aRfH/yvhSAEBqH/IskpgXUVGqms7qcimnEcWa0riVnxKrrOwEVg5au4uG30SkqN8hL/esdBhk42blpry01LaoRYyvBwBQaal3zsC3jNSfZ+B2KC1t9BqEE1zT2Z4MvFHXDTAvOcBnxZ6Q4okQXIBQzt0b2NeG8958+SZMZGBgYGAAwISJ2iQG9OyED99UuYBhJ6nvGf0O1scX33y0ljv+8ToAICqWqnE++atUy2//Q617+R6ik7j/tRFaPrGz+lVdXEY5h23Lftfy4EvUvd5c/JbW/bNXqpbL8xRpXua1Y7Uu94uvtJzYSVFxd+yRp3Url5CXUV2iLP60I3toXZqHBt7kb1ZEc5U7GK1EUlctcwbqlBhlZdvjLQHv4TewYv3cm6jglUN2NVGDczWRp1LlDAT7XXM4EdXVOXgB3DOIiPDND0QwCoz6Ol/yuFAscC/yOVM5ZOAHAqLF/uZCiBMBPAM1x/hVKeUjux3/O4BbrR8rAfyflHK5dWwzgAoAjQDcUsqRzX2esN4MDAwMDFoVLcRaKoRwAXgBwDgA2wH8IoSYKaXk9MWbABwtpdwphBgPYCqAUez4WCllMVoIYb0ZNPy1DrnjjgMADD9TUViv+IqqgTY+8LiWPztQDZf5v3c+0bq0n97XcspHDwEAypi1e3I8eQkV76lzk7J7ky6PKLLjT74TABAdQX0MGUVEV9FQZVXrDB6jdXn3vkTPMvJsAED6ULr+jppFWq6vVMNv4noR0V6vMqp2+s2iy6gEeQbRjK6CIylSWdbcM6grr9OyJ0Zdl3sT5Q4jLr36DFj/hKfcqqBxGHwPAB7h8tEF6jOIYIF4m2bC33AbJ8/A5VCh4++fOthRlhyBvJB2WprebISbZ6VyBi1yqUMArJdSbgQAIcS7ACYC0JuBlPIHdv5PALq0yJ39IKw3AwMDA4PWRIh0FBlCiKXs56lSyqmW3BnANnZsO7yt/t1xGYCv2M8SwFwhhATwMrvuHiOsN4OyGjdmrVZe0qLxKuZ9/8H/p4+fc+M0LZ9bpeLYT/fdqXU7rj9Syx/d9jEAYHwW1e6vvYdI6Xb8ouL3/S54TOuWfUYx/WX1ygq3+w0AoHTWB1q2rd1NDSzOz6qFcs5U/QMxgw7RuuJ6GuHZUFMJAIjqNUTrBhTQs9ZVqPxHZQ31WcT19x0vCQAR1ep3EJtKlnvxWhpuI2OTfNaUVZNnkGlZ6TxnEM36DOya/Yg455yB7QVw8rlar/yA0tf5yRk06g7kpj0DjmB1HN6ehfM5TpVDoXUr++pa01h2vn94WeutChG4WoyhuIlYvtMv2ZcnHoAQYizUZnAEUx8upcwVQnQE8LUQYo2UcoHT+mCxTwpmhRAdhBAfCiHWCCFWCyEOE0KkCSG+FkL8ZX1PDXwlAwMDg9aDXVoazFcAbAfQlf3cBUDu7icJIYYCeBXARCmlttiklLnW90IAn0CFnZqFfdU98QyA2VLKAQCGAVgN4DYA86SUfQHMs342MDAwaENQk86C+QqAXwD0FUL0FEJEA5gEYKbXnYToBuBjABdIKdcxfYIQIsmWARwPYCWaiVYPEwkhkgEcBeBiAJBS1gOoF0JMBDDGOm06gO9AZVWO6NwvBw++dB8A4N4jbgAA3P3Bdfr4cxUUErr8bNU09tWxFEY6ai3RUfzyoDr+6Ktnat3U6yjBvNMKidw0gcjprl1B5HgvLNoEALioN9FF/PUphQvjUlVI6rtN9EzllRR6GTtQTTLzdObkcuQ12hQP7jRqZOtRR8frqxXRXGUtNY8lHEwJYh7mcFUpAyM+g+5Vt5QmkXlifMNEu1iYqJt1scZ6KkeNjKPwmKZgiKWQGEe99boieNNZo0NpqUMjmbq+r84jAySQHZrG/E0/a6915AbNR0s1nUkp3UKIyQDmQJWWvi6l/FMIcZV1fAqAewCkA3jRCt3ZJaSdAHxi6SIBzJBSzm7uM+2LnEEvAEUA/ieEGAbgVwDXAegkpcwDACllnhUL84EQ4goAVwBA147pTqcYGBgY7BUoOoqWsRaklLMAzNpNN4XJ/wDwD4d1G6EiKi2KfbEZRAI4EMA/pZRLhBDPIISQkJU1nwoASV36y+N+VWG3O7uoRqn/nkMzjJ/4fp6Wew5We8sLCWTZvzJjuZbPzbSs2El3at2Gf1ADmY0JnUl++sC+Wv7xR0Uq98DJg7Vu7ouLtZxymqLT/ux3oovoz8o1/9ZdJaB3OZC4cewSlJTt2YGs2cY6ZaVXFVRpXWIHulY0t4J3FQAAYjpQaWolS9bWNvrmscpqyHOwr2VPNAOAyAw+1azpBLJbzzBmTWdOnoHb2TOwqSf8TTrTngEjn4sMIYHsRHTnzxp00jud2lxj0l+uO9g4b3M/vkxpLKG95tf3Rc5gO4DtUko7RvMh1OZQIITIBgDre+E+eDYDAwODJhEBEdRXuKHVPQMpZb4QYpsQor+Uci2AY6EaLVYBuAjAI9b3zwJdq6q0BEtmvAkAGPizIn1b0f1AffzMZS9r+YRvVXPafQMotHTWu19oecYLFwMAbp+7XuuGpZBlbQ+HKX/rCa274tgbtDz5phcBAFmPTtS61Y/O13JOH0Vkt2ktlZN2Yxb4oExlWf9VWqt1ccwdtSmuc1meISfRlza6tIYdT6eYfRyrh2ssUQ1qsekpWlfGSOeqGnw9A54zSLRoIHjOICrBwaOJjPHVgZWWeuUMnJrOGK20Q4OZ4I1ozDOIDJAz0E1nIVj7HNzjaC7C7yNj/4ZA+/UM9lWfwT8BvG1l0TcCuATKS3lfCHEZgK0AztpHz2ZgYGDgF+01ZLZPNgMp5TIATs0Yx4ZyneyuWbj6cZVuOPCCZwAAeXNobOXdYykVsXSIomk4ZvbrWld7JtFBbDriFgDARzdRo9jCfxDp3a71Ks7+6/Nk7Z/2D7rXZRYdRMPg47WOx+GPHKIG3Ux7fY7W8TxUSrkairM8n6z5VDZ8JjpBWfFriiknMDDZNyZfWk/37J5Ox2N5zN3yDOKYZ8DHYlZbXgJ/vmo2yCfWssgb3ZQz4J6B9CjvRkaRZ+BFQa09A7Ls65gX4EhHEenrBfCxl/7oqpvScTjRSfjzLLyuuxfMxObGbtvrh1WbgGi/nkFQ7zshxN+sZrAyIUS5EKJCCFEeeKWBgYFB+4FouT6DNodgPYPHAEyQUq7emw8TKtLL83Hh/EcBAC8kKUfj/7ZRhc9oRp6W1ktVYk3+hdb3P/ZULV805UcAQOEqqgDq8tQDWs6x9O8eTxQVY37+WMtRluW+cCvtkSlRtNeeYPURPJ+/mR1noxxX/wQA+Ll4OD0/GxgTbY2iXJlL1z8zfZeWbWuaE+11YzmDGmZZ1xQoLymuEw3K4V5Mjdv2DOgNXVdDnoE9yMbDPIPIeO4ZKO9FRjpXRjnmDPjYyij1d/NLR2F5Ma5IX/I6AIiOdFnP0XTOwF9uwKlyMBRr2yl5GI4JRQNntFfPK9jNoKCtbQQGBgYG+wLtdC8IejNYKoR4D8CnADTXsZTyY/9LDAwMDNoXzNhLIBlANRQHhg0JxZuxz5CXX4EHH1SNZeuqngcAZBxN5Z6v/PKGlqPLcgAA1/yLZhjPnU5sF2PPVuGf+PQcrZu+mf7o44edBgAoa6CmtI3T39Nyx0EXAgDe+XW71o1JogTqQdmqwctdW6l1XeOoNLTkp58BAGsiu2vd2RkU5olPV91uK3eUaR2yKEzkilalqVUsETw8mcI0RSxkVVOk1iX0o3nMtSxMs9MqT+WNag18BrLFUMqbzqK9ktmK7kJGOieQbToKrqtp8E0gN3rNOKCr6wRyKPMMnJLK/hq5zKQzgybQXv/kQW0GUspL9vaDGBgYGIQD9hW7595GUJuBEKILgOcAHA7lESwCcJ2UcnuTC/cysjom4OZJhwIAZnY/CACQffq/9fHjvyLL9Ys+igLkGrZ++OoPtRwVpyz3A447RuueeIfoKtYdoyaQDWbTwf74kjWoPaSGEC39jVhorz44W8tJ+Wrqmd08BgC9OpLln7dEXaukF80j6DgoQ8vJGSqBnJ9PnkXDNk1kqJ+fk9t1Y01z9SxZXVOoyPJcqZRA5vOOyywvgHsG9cwziLI9A5ZAdllzkzmkH2qNerevZ1AfYAYy9wL0DGR/5Z4BSkv1vAWvpLLHZ43X+hAmnYVCR2F7FoEmpbVXa7S5aO3fi2ihsZdtEcFucv+DolfNgZrQ87mlMzAwMNivIERwX+GGYHMGmVJK/uE/TQhx/d54oFBQltYZX016WMmvqHTGokfH6+M9j/mnlt/eosaJTpryrtZ9c+kFWj74NkUn8eBEmjFs5xEA4JN6RWfx4Qm9tO7NT9ZqefLRynOY9Nl3Wtd3Is2bqFqsPBOek8jpk6Xl7T8qArvyePIsMgbT7IsOSSomX8xKS6s2b9FyVIKy8rmFnxpLf95EVmZbVai8D1cqEcPWB8wZsKlm1rU8DURRIRxI6fzlDOosy5xPOqth85Rdkeq53cwz4GWk9rN4UVjvUc6gac/CE8BaV+c2fTyQxe+EUCaN7eupZGH4mdcsCLTfMFGwr6tYCHG+EMJlfZ0PO0toYGBgsB9BCBHUV7ghWM/gUgDPA3gKKmfwg6Xbp9i+NR83Tn4IAFBtxf9/PXeCPt53LFUW/fb0twCAlw4ja/SaPIrPv3ORIrhL20xNZxy5y9V40YH/JXrxyg9u1PJRmcoyreRNZSfQuWvuux8AkNrjYq3L6k+Dbr79coN6HSVEcZ1+Zn8td6tROYHNf2zUuvKIPC3HdaDZyDbS4ui18nnH1cWKYM4TR3QUjaxpa5dFrR3F3tBO845lKSOSi/cdiOPx4xnU6hnG/nIG1vAcP8NtbC8g2ouigg+nCY6ozh+cKoS8ifICrW/ysEE4Q7Tfv2+w1URbAZwa8EQDAwODdgyB9jsJr8nNQAhxi5TyMSHEc1AegReklNfutScLAvEdUjH49HMAAMOe2wYAOGIOWc6/fDJOy6vWqJj+/BPJoeGVQfJlRWq37LtVWtdjNI3Q3PDdpwCA/P4naF1OLPUpuL+eBsC7wqYgjQbpbF2sCq9yrkrTupRDDtNybq1i7K5jozpjBo7V8tAiNU5zjkWIBwDltRSpi8+hcZt6fT15PnzEZfk2pff2DGhdaaV6DSkunjNwqCYqZhU0DjkDN+8TYPmBOgfPoM6hcoj3DjhRWPsdWxkkUV0o1r6/3ECwfQb+rMnWiib4v3/rfbK1l56McAwBBYNAOQObgmIp1HjK3b8MDAwM9huoDuTgvgJeS4gThRBrhRDrhRA+0x6FwrPW8T+EEAcGu3ZP0KRnIKX83BKrpZQf8GNCiH0+b6B/sgcLT1AD4FOu/QYA8MpY6uBdMvpoLY9eoKijr0smIrj7H6X8wgcPzgUAFLOqlpv/TefesVUNvH9sPnkeE3uQZb3qf+r6camjtG7uhlItlxQq8rZxw6maSPTpoWV7uAwfGNOYQ57FIIvuub6SPIfyErL8EwepnIDX4PuKAjrekSz3wj8UUZ0nlp6fo8TyDLKYOeyu1ywkiElWXgbvQHbKGdQzdyPCIWfgYj0XvAPZrhxqdLP1LD+gieoCeAaegMNtfB65Sb2BAdAyFVRCCBeAFwCMg5r++IsQYqaUchU7bTyAvtbXKAAvARgV5NqQEWw10e1B6gwMDAzaMQQiRHBfAXAIgPVSyo1SynoA7wKYuNs5EwG8IRV+AtDBGgkczNqQEShnMB7ASQA6CyGeZYeSAbidVxkYGBi0U4TWUJYhhFjKfp4qpZxqyZ0BbGPHtkNZ/xxO53QOcm3ICFRNlAuVLzgV3jmCCgA3OK5oRexYuw13Ha56395dpprKenamcMV9HaiB7N+v/QEAuDGbjosrH9Hyyls/x+64syuFGd48cgQA4Kuv/9K6O/5OTWWfPa4moKWdTTOY3/yRmsKGWI1c5/QjCojiGOcZwTZKIiiM0ydNPUt9FRHVlbPS2GRrdgFvFJOlrPS0IyWYd9arfby60fldXVqlQkKJkWyGQB2bd5ypQk48NBPhkECud5hrDAB1jaGXlnpTRzgkkNlc4shgE8h+AruBSks5QqGeCBZOjxVKo1NzwxjttXSyJSCkhAi+kbBYSuk00RFw/jPtXqTj75xg1oaMQDmD5QCWCyHellIaT8DAwGC/h5BN81gFie0AurKfu0AZ38GcEx3E2pARKEz0vpTybAC/CyH4ziMASCnl0OY+QHOQHOXC2C7Kes66+yIAwOGZV+jjn19AjVgHvaPopsd/QxTWx0//Tcs3WB5DOUtkbnuE6Krvm6yiZOPfIyK8rFsu0/Lqe2cDAHoN7qR1m1ZSArePZSUP60TkdAu2ELVEopUgtamoAWB9aa2Wh3RUem6NF9QSHUSvrCSv6wCAO2+zluOZZ2AnqysbnN/Uu6obfK7FE9tR1lQz/iwyOgTPwPICOGlfPZuB7LIS116lpVG8wUzpY/w1nTnRUYTQSKbXN4ZOJeEPxthuL5BAy2wGvwDoK4ToCWAHgEkAztvtnJkAJgsh3oUKA5VJKfOEEEVBrA0ZgcJEdqH9Kc29kYGBgUG7gGx2RAZSSrcQYjKAOQBcAF6XUv4phLjKOj4FwCyonO16qHkylzS1trnPFChMZAediwHUSCk9Qoh+AAYA+Kq5N28uYgcOwMB5iibi4YzBAIAdI7+mEz5/W4sJlysiunu3kOX+y8fTtHzc64rUruyPFVpnl5sCwFU3qXJMbiFvTCK6CJvO4dxDu2ndTTPnaDnOqleM3viT1v2wjYjqcixSudgUoq1eXkCew6FJVdgdpfVkofTtpOgqeHdUQyF5jvFZ6Vq25x1XsvW8nLLaym/EMqI7N3vd0cm2d0N02jLKl3rC2zOg53Kio6hmJb1OOYOYCHoWT4CmM5umwiun4eAFhFJC6o/UrrnzjptLembi+60M2WKeAaSUs6A+8LluCpMlvFn3m1zbXAT7XlwAIFYI0RnAPKgdalpLPoiBgYFBOEBIT1Bf4YZgieqElLJaCHEZgOcsiorf9+aDBYNVmwpw4AXPAAA2vKxCZlXxf9PHx91AdNX33aFoK+6+d5rWNdSQZbts4JkAgBFHnKN1vMIof+qTAIC0XsO0btpSmu3TL1FZxif2IQv86hKyzO0Rl+Xfz9a6BXUnaflGa0SmPd4SAJZsILqJ/0tVljnPKexk+Y3h6SpmX8KG2FRspZxF2oGU3rFHY+5iOQdehVRbZRHVJVBM32nEpfRQZZOMoudy9gx405k1XCaK5ww4XbXVNMZ0EczjkdL2DJxtmVBGXDrBsZqnnVIQGIQKCXjaZy1N0JuBEOIwAH8HYGdNg11rYGBg0D4g0WJhoraGYD/Qr4fqOP7ESnL0AjB/7z1WcIhwRSI2RdXtX5+ohtq83nurPp6yeaWWr4Cy2G+tpjj8AScSEeuVL6lY/tnj+mhdb2YZ/zb1R7Xmzou07vPvN2n5pREqF5FdTqMwuTU8yKKD2PINOVT53UZrOWu4GjSTnEV5hPVbyfKu77AGAI23BIAaFlPv2cGqNooma7lqR5GWM8dls3XKsi6udvYM6uzhNolRWudVTZToMMjGYcRlAxslySuH7EE2fvsMhE0nweksfPsM/FJYO1QDOY249GftN3fEpRNCIcUzTkhbhgQCjEgNVwRLYf09BKI+CwAAIABJREFUgO+FEElCiEQp5UYA+5Sx1MDAwGBfIBzzAcEgqM1ACDEEwBsA0tSPogjAhS1RztQcDO6RjgWvqdGVaYerpPvQ9Yv08Zs/nKnlTyaqXMKJj9D0zn+PJyK4g069BQDwaulxWjfrrAFanvqmqjL69ym05pSrXqBnuXgMAKBi7vtal9iph5a7HaCopzd+s1nrdkWQF5M9SnkkGYzwrZB5BpVxyuOISSGiuxoWk+9kWfGlHchCr8wjL8iVQevsEZf2eEsAiONVSNZYyRhG8c1HXEYk+dJlO+UMqllOw2nEpT3eEnAecclHbToNtwmJwtrB3HZ5XXPPKKyd7h+KZR8sHfK+pk1uC85Km/GY9ufNAMDLAP4lpZwPAEKIMQBeATC6qUUGBgYG7QpSAnsw1zocEOxmkGBvBAAgpfxOCOEbODYwMDBo59ivw0QANgoh7gbwpvXz+QA2NXF+q2DXitWY2f0gAMCER1X4p/yaBfr4bXXUNHZDkZp78O4p1HTW8L3u79DNXrm/UqNYv48f0HL9dMUQO0pQOWllwWYtx518DwBg+cVEUZE14Hotdx2uwiwzP1qjdVVFRDyYPkq9jsGVNAnt058pCrfLrc6NTz8YTsiMV3/KLekUrqksoEY1T3wqvRYrzFJYSTMKYh0SyDHJlPRtLKDSUhGf7HP/Rpdv01l1Q6AEMiPCY8lwm46ijpXw8WSxPU3OFYhOwk/TWVM6rg+U6FXnBjxlv0P7LsNtuaaztoZgm84uBZAJ4GPrKwNWa7SBgYHBfgXpCe4rzBCIqC4WwFUA+gBYAeBGKWVDU2taE3WNEpusBqnpMcqij3jrKn380XNoBMPfD1XNXD9N/LvW5W6gqWFjHnodADDvjQ+17tNyops+OFVZ3DumPK11rmginfu1RlnLf/1MjWbDJlGZaMLoE9U9a6fT81fQJLTIwReq++RTAnkGa1or3aUa0JJHOE8niyxTzCFJOVR6mr+sUMuNCek+a4rKyTPoFskTyMryjk0lL0PmMis7wWmqGSsNdfkmkCMcqCdcLCvb6EBhzZvOnJLFXt5CwAQyyQEnnTmYSKFYu07eQmsay873b8/WeiuiBeko2hoChYmmA2gAsBBqBNtAqJ4DAwMDg/0OAvtvzmCQlHIIAAghXgPw895/pOCRPbg3bvt8BgDg2p4qpt9pDsX8kyOnavnQme8AAP7Z8SjHa715nqKZGLNll9Y98M4yLX9wthqU8/Pr9CtIO+Y6Lb+wUM1GzmHW9nkHE2ldRbbyHOpZIxWPSVd26AkAGCYpNs89h52l6rk6HE/eCG8Ui9ipchmJ2eQ5lC2knESti6x8G4UV9KyDOPW1RdMR3ZHu5RV/T6L8g74+K3O1ZxvXup1zBrYXwecae7zWK72HsUN6NZhZzWROQ2wAPzkDp+E2AXIGHP4M671hcJtBNm0ZEmhBavO2hEDvOx0SaunhNkIIlxDidyHEF9bPaUKIr4UQf1nffT9xDAwMDPYlbDqK/S1nAGCYEMLuXBIA4qyf7eE2vmUlweM6AKuh5ikDwG0A5kkpHxFC3Gb9fGtTF/izoB5Dn1GNW/87vhcAYOz9VCG0c+7DWj7rMxV/PzeTrN1KZrnuemgyAOD+vz+kdedcTmMxe72qvIDnpl6odX1G9tPyTz8qK/xYVkEzpjv9en7OVdY2HxjDSefWlKhBNn3TqGnMrpoBgPxd6ngPNraTX6th6zoAQEJnynNwiuuKOl9rprCchuc4DbKJ6UD34s+CGN+q4jq3LyldDW86i+BNZ8qu8JcziIpxWfdkFNYO1BN+Kawdgv5O1r6/sZnNRSjGtrHMww/tNUzUpGcgpXRJKZOtryQpZSST93gjEEJ0AXAygFeZeiJUjgLW99P29PoGBgYGewdyv/UM9haeBnALAF6W0skepmONduvotFAIcQWAKwBARCdh049qmE35G2qsZcpdVA30f1t6aXnua6oP4f3PqHegesWvWn7lX2rdjZdSnJ0PgV+TQtTVNq48hkjtrvpcjdPkFnbs6m+1PDu3BwCgRzyRv8WlUs/DD1tVZdNBsc77c36tsqaHdiUqiEZ2r7ptqu0jqRtds5QNjCmrozennWuoqiJrPyHOd5BNTCpVJgFEbSEtz8CLaM5hkI1XNRHLGVQH6DOwB9lworpA1UROPQXeYy/heK4THEdk+rH3gx1kY4bYtCOE4Qd9MGjuezRkCCFOAVAopfw14MkOkFJOlVKOlFKORJRvUtTAwMBgr8GmowjmK8ywLzyDwwGcKoQ4CUAsgGQhxFsACoQQ2ZZXkA2gsMmrAOjdIwtPva6G1k+8RMX6f/rgPn384FMp5WBb4S95RmjdaWecoOUNV6qqpFV336t1XQ6mbuJ/z1Gdw6ekUkz/1P5Uu39xmaKL7pdIFnDBF0SUtzDqbHUd1geQlEOexfzV6uVeG896D2LpXHuQzZEdyZkqiCHLvHyT6jPoOOYIrati1nZxNRtOY5mZ9hAbwJuUzvaIeM5Aeqgnwyal8xpYw+5lewE8Z+A0yMYeYgM4D7LxsGtGR9K9nHIGHE79A84dyI7L23kHrUHzICHde7/VSgiRBuA9AD0AbAZwtpRy527ndIUiEM0C4AEwVUr5jHXsPgCXA7B57O+wRmX6Rat7BlLK26WUXaSUPQBMAvCtlPJ8ADMB2MMCLgLwWWs/m4GBgUGTkGgtz8AuqOkLNWr4Nodz3FCNwAMBHArgGiHEIHb8KSnlcOsr4LzkVt8MmsAjAMYJIf4CMM762cDAwKDNQEJCNjYG9dVMBCyokVLmSSl/s+QKqOrMzrufFyz26ehKKeV3AL6z5BIAx4ayPmr7JnS+Xc0z6HqQmmcQeccF+nj2CCoDvfCMwQCABx6heQMfHDFKy3f3VQRxX3y6TuuunEVzg599Vc1JeOjvlEiO+ukDLcckqfWDh1Le+6/P/9By7tCjAQDdjuyudZnZFGbaulF5gDWxv2idTZ4HUBlsnzSWJ4mlZHTZJjUvIWcSNbrxSWh5jJQuzoqj1LJJZ7Es/NVYr0pOI5OpTJXDE6PCV15hIlZSa+sra6k1xYmozqu0lBPV2U1nARLI/kpDI3TTmTOdhda1wEQzJ324TzXb14/UFn8nGhKhTDrLEEIsZT9PlVJO9Xu2N4IqqLEhhOgBYASAJUw9WQhxIYClUB7EToelGmaOsYGBgUHQCGmeQbGUcqS/g0KIb6Di/bvjzlCeSAiRCOAjANdLKe2yv5cAPAC1fT0A4AkowlG/COvNoLisDq/OVJb8yu1qKtnkzmSNLyodr+WsPGVxP1ZIpaO/f16g5bGvqUlnnxxFoblnh1MZ591r1Izknu/+S+vW3E9lqhkDVLqjTz+ytt9/YLaWyzqsAgB0vZwoqPvVEV31gjnquYvdG7QuPn2Mlu3KzWw2lzifJaPLt1cAAGQHem9x6osCL7pqiyKaTzpjnoGnVCWbIxJ9J5oBzgnkinrmBVjJ4hpW2uridBdWYpnTUdSzpji7ZNRfI1mg0lLH0lCms0nt/FnwzkR1cDw3WIRCFGdI5dowZMslkKWUx/k7JoQIqqBGCBEFtRG8LaX8mF27gJ3zCoAvAj1PW8oZGBgYGLRxSEhPY1BfzUTAghqhrIbXAKyWUj6527Fs9uPpAFYGumFYewadu6fhobvPBwA81W8CAODMQRTn3nHuBC3//oeqsBp924ta9+M7FPOf3/EYAMCBbIZw0dPcW1PW2sq4/lqz8iuy4g98tCsAIH30mVq37lYqLa3ZqTbqmEMna93R+WR5z3pDNY0VlezQug6DqYHMRnwVeTMpXViZ6Qr1+jxJtIb1gWk6CwDIivT1DLxyBtYgm4gUX9prgEjp+FxjPsjGJqqr5p6BA/UE1znRVXMKDMeYv5/a0CgHojqnclN/c40NXXUo99/PvBi7mmjv4xEA7wshLgOwFcBZACCEyAHwqpTyJKgy/QsArBBC2KyadgnpY0KI4dYTbwZwZaAbhvVmYGBgYNC6kKEkkPf8Ln4KaqSUuQBOsuRF8JPvl1Je4KRvCmG9GWyLSMV1cX8DAIyJ+QgAcMRSGnt5ffxAnzWzLj5Ay8ewaprrp6icwFdXUIXR18/StTpOuAMAcM+s1VrXh1nb1xylqC92dSISNye66tLUvlo3OpruX1tWDAAoKSrRuvRxlBOwK4AiCskbSelJ1UarFysK62oXEfFxbN9ZreX+Ucokbqgq07pYRn1tW+ROVNUAUGOR0rlYhVBlvW/lUGUd1/Gcgd105lxNpHMGrEIoxmGQjV86iiDpqv1ZtU7q5hrAfhvcglxvqKrbCGTLkhq2JYT1ZmBgYGDQugipmiisENabQWl+IWb893kAwJRtqpx35N3z9PHXjuup5QqLQnr52RTTf/MtyskMPEFVCXWeTRVCPz1Gg3COOEZVK9lVPwCQxeLcR6SrN8jHG6mUN5PRRUQlKMt7yY4KrTu2p681vnkXVf0M6UHVRilR6lr1G//UuuSelCMqqlNUTztrnePkecyLSYtW13LXVmpdbDqR0EqPFauPI51XT4H1urnOiZSuhnkLTvmBSEaOJ5kXFeNUTeREYe2nGsiRqM6hJyFQn0EwcLqCqcpox2jBaqK2hrDeDAwMDAxaF8YzMDAwMDBovWqiVkdYbwYdOmVi3HVXAQD6XqnKRAtWUtK323zqs8is3AwAuK7byVr38Of/1XJCpioNfX0jhStyYunXc+3xqqR0xDPUTd41jhrA3HPVvIS3y8Zo3UUpVK6ZnN0bADB7FZWGnhRDDXB2aKWAJV0P7kEJ3AortFOxjhLISf16a9lmNS2upvV8RnJNJZVpxnVQDKUNNRQmikunkJX0KGoLT6zz/CI7gRwRSa+/2oGhtNpP05mdLPafQHZZz+EnzOPQNMbhzFrqeKojnGYUBDu3oCVgkr1tFxLSi+akPSGsNwMDAwODVoXxDNomeohyvBalKB+6Flh8/5dcoo8fcRvRQYwZo0o/R7Omso9voqazU597GwDw1IxlWvfOCTQprX/pbwC8CddG9aSmsZX/UxPX1vQeoHV9jybSuIweKpm9ZGW+1tWCktE20R2fTva3bLLMN1jJ1p3ryJtIG3u8lm0iu+1srnEcS9pWVzA6Cut3YM86BoDoVE49oZ5RslnHXkRzVmko15WxMl1KINNriYzypaPwNwPZkY6CJ5CtBHCUi3sLbB6C07zjAKWlgUjp/OWanZLVTtjferOCRdj9XqSEbKgPfF4YIqw3AwMDA4PWRes0ne0LhPVmsH1TMW694HUAwM/5KwAAvar+0scT3vxayx+sUTHxF2YQ0dz14x/S8pTTVE4g9ZnXtG7o/+7Q8vqnnwUAdBpyjtYdQCF7fPqoKmkt9pBn0eeSw7Xcr0Y1iP08n5rWCqtXaDkpW5Wx8ka1bskUky/toryE0vXUlNa7Uw8t23TVOyrIM0hgljfPGSRajXGNpXSuK5W9GKipbp4YorvgZaRldcoL4NPLOF11ZLR67vp6Xm7qS0oX6adpzCalC6W0lCPKnpTmUG7qtd4vHYXvM4WCcCKlCzfDvE3AhIkMDAwM9nNI2RIkdG0SYb0ZZCTH4JLDlUW7ZawimlucTxUyk6w8AAB88uqHAIAPOp6kdUdlPK3lHbereccigmZILIoZrOX1HyoK6mNf7Kd1HUcRtcTqu9RUueqSXK2LOYq8kFPzFbXEN6zRLW/HZi2njjzb5/UlVObR8V4qpp/7M12/MYWGGtmkdBsLq7SuF4vT11axXEKGoqywCekAwJXqOzuj2k1eCielq7SH07CcQUWtb4OZm1UYOTWdxUW7mI7NaLa8gMYARHVRDhVGQPCkdM0lpANaL+bt//6t8wD7HSFdEzDVRAYGBgb7O6SEbDSbQZuDu1sv7HrmXQDA7EGjfI6/0nurlhOvmQQAuOXhT7Vu/bPnafnpy9S40V5XP6V1t75D8f+DLDqH247po3Ub2b0aJeOLtrA5uouWj7KmXdaWFWld/o5iLXc9W/UUJLLYOLZQTiGtXw4AYOk3m7SuzEM5BRs7GCHdSGZ5c1K6eKt/gVvjER18R1zWMM/Ai5TO6oVwRcf56AAg0qLOcDNaa1sHUE9BjEOFENcHyhk4EdIBrMKn0TlnYCOUUZahwOmxQqGoMKR0bRdSSnga3IFPDEOE9WZgYGBg0KqQMJ5BW8SGTXmYeImqCNr5rfpesfxXffy5427R8jMr3gcAvPLQcq3bdswTWs6tfRUAcOt5w7Vu8k00COcwy9zquWOx1t2dS30E/RJVV298eo7WfbmOvICr+/pau2sryDIfO1DF7BuZNV+z6jctpw5UrkVRHXVYF1Uzy9l6vvwS8gxSE8hzqK9mnkF2mvUs5VrniaMOZLtyiA+s4d3GFfX22EqeM+B9Br45g2hG2mf3FHBr3ylnwH9XURG+noHfnIHTIBynPgM/NriT3p+x3VxSOmPFhx/MZmBgYGCwn0NKCY+ZZ2BgYGBgYKqJ2iCiEpLQ9aAxAIBjflBNWZcfd60+Xu7+RMvfHH8pAGDkpY9r3UUvL9HyI8PU7OAjs4mi4bKdRB0xNlM1av31LIWOvsiksaJTR2YBADr2H6F1H/6wRctXVql0s007AXiT0p3VXSV1N8dTOKZo2Totdz1bTXTbyUIvW8voWe1JaJVsbkECm7rm5qR0mepe0kOzF5zCRFUOMwoAYGe1Cum4YlgCmTedWSWtbi86ClZG2miHiUjnlCzmCWAn6gl/5Y5RASad2fcKJVHsj7rCYD9DK1UTCSHSALwHoAfUDOOzpZQ7Hc7bDKACQCMAt5RyZCjrOcwcDgMDA4MgYVcTBfPVTNwGYJ6Usi+AedbP/jBWSjnc3gj2YD2AMPcMDsiKxZJbFTFcwgRl8a+YQyWS+e9co+UbTn8GADDn2sO0LnPsTVo+8vX7AQCbH7xT67KGTdDyQacpi/3Lp7/Xuh2jKFk9+OKxAICB9Vla99siosYoLF0EAEjuTPevaaTSzQHpysqu60IUEIUrqOmsx01qnnMlI3TbvIs8g2TLyq4qJ0K6hI7MMyijZrTITHsONBXHemLJM7C9gDLeSMa8AJuUzqadALypJyKtJHhtFSWVOfVEo1tdNy4qgGfg1Ujm0HTGG9kCTjrzWe6FQER1/uDU9OVMdGdoJ9pL35qndRLIEwGMseTpAL4DcOveXB/Wm4GBgYFBqyK00tIMIcRS9vNUKeVUv2d7o5OUMg8ApJR5QghfigD9RJgrhJAAXmbXD3a9RlhvBgUr1+Opfsp6v//zLwEAjz/5kT7+3wSinpjYfRoAYNNlZ2hdZMwgLX/sGgoAKHyTSk8nvXWvljsddAgAYPn9c7SusmCzlmPH3wcAOC+PfqXz3vhQy1vWqvh/x2OIdoLTJiSXq/xC5qAMWrOA6KrdaT0AEO0EAKzJo3nKAyxrvKaCPIDEbPIy3IXkRbjSaXayjSreYGYR0JXWMMueNZjZ1BM8D9BQ55sf8LjJS3Ginoj28hYClZY6EM35sTSDpZ7wV9bppG9Nq9b5/q05XKedmPB7A6HlDIp3C914QQjxDYAsh0N3Ouj84XApZa71Yf+1EGKNlHJBwFUOCOvNwMDAwKA1IdFy1URSyuP8HRNCFAghsi2rPhtAoZ9r5FrfC4UQnwA4BMACAEGt5wjrzSBKCGRazUxjZv9/e+cdXVd15eFv2+rFKpZ7wQ4uGIw7HYfeHAZSJhNgmZSBMIRAbCeBwUCyIKGNQ4CQBAMxJo1AAqGYMjC0xHQwJe4CGxtXhHu3JVln/jj3nXNk3Wc92dKTnry/td7y0b733XevJOucffbev30jACNv860sv3rRLW685cV7AZg4+HxnG32b30K75p63ADg12HO//VQv6zxzpV2Fhw1jQjmGd7fZ/flT+3vbri3r3XjxMjsefkU3ZysJVta1894AoMtw31DnH08vcuOqXQ1Xax9Xec9gbPR9qA4+s2ioj5/UvedX3lLa0GPcWh0WmFnPYHOQ7RQ+66Yd9lqhZxAWmGXndmxgKwg9g6g5SG4jEtYhWTG2MGawL9ITSeUo4s2xqPTEAYYx1FWnRY5iBvAt4Nbo3yf3PEFECoEOxpgt0fh04Gepvn9PMnoyUBRFSSsG6tJTZ3Ar8DcRuQhYBnwdQER6AtOMMeOAbsDj0RZiFvAXY8xze3v/3sjoyaB82BDOe+1VACYW2gyZa7Mnu+Mlvfx23bWVtg7h4EKfLz/5O/74QSddAcCgoqC94/0/ceOf1ZwNwJXdi5zt4QGj3Hjqa1ZA7vejfD5/uJr+KGouM26o3yJclx80r3nHxpnKRw1ztqpdz7rxishjyQ82ytev9dITpZEsdeiNFPX2noGp89LXdYW21iGsHajvGdj72hhITITZRBtdNpFfA9cEXkR+9D2Ma3Jv78Wu4sM4Qlx8IJkstROqS7IqDqUr3DPtZ5N7lZ1QAAzpqTMwxqwDTomxrwLGReNPgOFNef/eyOjJQFEUJa2Y+luS7YmMngzmfLqeAd+1EtYzJx4LwC9uedkdf2GFl6M+7sIpAKy6/yJn2/bAdW5c0tfm8Y8b7VfWM2/0K/OFx1k56lGXjXW2AXlezvrNN6xc9tpVrzhbcU8fc1j/jv0FOq6Pz+ef38PXAax+28YHul/g729TIBS3cK3NEgrjDFvWB3UGfWzmUO1On02U26MfntAz6AzUb2UZZg4lvID1QavMrJxcN97eSDZRTrTir63212wsm6h+tXEim6iugQ18TUGcBwA+myhZ7UBjFchxdQrJsnm0piDm81v7BloU027lKNJegSwifUTkFRFZICLzRGRCZC8XkRdE5OPo37J035uiKMpeieoMUnllGq0hR1EL/MgYMwQ4Gvi+iBzKPpRPK4qipBNjDLura1N6ZRpp3yaKquISlXFbRGQB0It9KJ+uq61h+7qVADxx6c8BOPXhc93xDteMd+OanXbL5uWhVzjb8u+e4MYXPPAIAAOO8IVqv57+JTde/4ktRqu42weVL/3MB2AvedJuKS2a7buj9fziWW7cMUrs6rZjhb/mCJ9muuw1a6/p6vsqV9f5QrC5K23vgS8E2y1bN/rU0pK+tkdy7Ry/dZTVzfdbgHfdaFudvUZWEBTesLNhgdm6cJso2BLatcP+omfn+l+fHUFvhsSWUNijIJSeSBSYNSWAHCc+F2abmhg5ipDYFNBW7muc7B60wKwt0363iVo1ZiAi/YCRwNukWD4tIpcAlwB0CPR0FEVRWhztdNb8iEgR8HdgojFmc6qroUh74z6Aw0eMMo/9cRIAQ8/6IQBb//W0O/eKbie68bl/sCv6S2/0x8/e5OWe7zrJrtIfW+bduzBYm11oJ54XNxQ42zmD/GT0zai38ewlvrvZ8ZN8D+SKaBVdM8vLWfQ4Zogbv/zMYgAO3RkUtQXfktnLNwJwYiBxvSvop9xpjJWY2P2e9wykwn9+GCzetKthp7K12wMvIM+mz67f5gvwQi8gkUaak+9tcQVmieKy0AZ+FR8Wl4VeRFbHGDmKmAKzZAHkVAvMmtK9LKl0RZJrpPpZqaKpqW0EA2Z3w37n7YFWmQxEJBs7ETxojHksMje5fFpRFCWdGEy6VEvTTtonA7EuwP3AAmPM7cGhJpdP76qsZOkJJwIwKkodHXufl3C4+XC/0zT2dFtoVXiH72E8LiggWzzpYgBu6naps009upcb/32YjS/c/NR8Zzt55FI3TjSt+WirX01/Y6R///KoEGvVS284W6JhDcCqnbYRz8KgkKwoSL3cuMamjHbq5cXnwgKzTv2sZ2DqvCz17mJfdBbnGXTMyXO2dduClXm+/b6EMYPsoIdxdeQZFJX694eeQX6O/bWqFzOIkaOoLz4XpJHGLINTbVgDqfc7bqmGNbqKb8cYMHXqGTQXxwEXAnNEJBFtvYZ9KJ9WFEVJJ8bA7motOmsWjDGvkXwLtUnl05t31vL8ItvJ7Z8n2T31osueccePfds3onl97MkADDvnemc77YjRbvw/X7cFaotGv+BsY34xwY2/un4QANPu89dfPMcrxXY99NsA7HjVy1Yf2dV/e7MOt6v0ZTOXOFvfa49040SB2fsrNzlbRY5/f8Iz6DzQl19UL9/sxtl9E01zvGdQnefPDeUkqqIVf8IDAFgdtMvMzrVezM6gEC0niBkkGujkBqv92mrvERXlNvQMcmJE6XKTtL1MxAfiGtaE52Yn0bBOrMxbumFNsnP3dk/7+v5kpPrulsoaOuCSkYzRmIGiKIoCdToZtD16De7NzfdbmeqfnHAlACOvu9cdH/OTl9z4+PdsC8lnXzre2eZvPsqNN9XYtpm7d/lsnMp+Xm78B0Pst+qX1y1wtg9XLPaf9Ys+AJRMD/JLZvnMpb4n2PacYdvMQ3L8nn6Cd5f4OMD5Bf7Hs31dFQBlh/g4xO5FQU1BDy99nWD9zrAVpfcMPo/iA9l53jNYs8V7BgkvIFFPAHtmE1kvpTjP28LvWyJzaHeSmIH3DOJzcVydQT2Jiqas1hva4pyI5HIUMbb4U+M/vwnnpvr5ShtBU0sVRVEUA9RpAFlRFOUAxxgNILdFKrdmcdKrtmfwT/tZOYYbvzfYHS88/Xo3vr6rVQjdcOWFznZRxcVu/NujbYHW08PPcLYrHvbSEs8Ptz2KEymkAO8HQdfLxtptmpUlPt1y5ZM+2Nz7q+cAsPgmH6CeXeUVRkuy7UbErOU+gNytry9q27HhMwDKDjnI2epm+GBxbWlPoH4h2cZgmyhMI/0s2hJKFNJB/TTSRDHZriCAXNjJq5Ym0kiL8nwBXFwaaVh0llevwMxeN66vMUBezPZRWGDmeyAnCSCnmEbaHMJcuqVzYGG06ExRFEXRCuQ2yvYN65n1yEMADHzDBmafH3asOz726rvd+NxjvgzA5GMnOtvCETPc+JiHbO/ky6t8odoNP/+zG8992XZU6zXycmfbGqTzyMdJAAASaklEQVSRHlNmV8FzgkK1xc9VunHvn54I1O9R8Oon69y4Z7TKXrfai891PbTCjXd9bFNoc75wMh7vGezItWmkYQrpskBuIyfwAlZEfRBy8/1qf8sWnxqaG3Vg2xp4PgVBsDiRRloUBJVDz6AoOjdc7celkYYeQF1dw2CxibGFhKvyekVnMUv+pvQd2N8eBS0hPte0vszN764ccCmkSWm/FcitIWGtKIqSmUQVyKm89odU+ruIyGAR+TB4bRaRidGx60VkZXBsXMNPqU9GewY9e3dnwu1W5XrMeFs0Nr7K9yB+4Wy/Sv7lcivXUBGkOIb7/7/faPfcv3eY31u/smqpG/9zkY0ZfONyH5PI+Z3fM695fjoAA77s01Xvm/A3N+5bYwXuwgXu65VeaO7HJXaVvm3NMmerOM6ni+6eZ2UqpJeXuA4lJtZsj2Sl83z3tJWbg0KyAu8ZfLbJegY5QQ/mndt9fKAgks5Yt9rbSgOBvEQaaXE9b8GnlhbGyFHkxhSdxYnPQbwAXVx8IFnMIM66v+JzSYvG4s0pozGHzMKQtjqDRH+XW0Xk6ujrepL+xphKYASAiHQEVgKPB6fcYYy5LdUPzOjJQFEUJa0YQ116soma2t/lFGCxMebTff3AjJ4Myjet5uvP2qY2d5YdB8CPrzrRHf/jET5z6NajbKObxXd4yaOZdWe78Q13vwbA6X18D+OKQUe48Ucf2t7KPz/GN4yZM8B7bnOn2wK3MdPudLal233M4R9L7J5/t2Cf/eVPN7px96jRzY51vldxp2HD3Ng8aAX2asv7OVvHoJBsZSQREWYILV/vRe9yCr3A3edRLCCvMPAMtnkvoLyL9S5qdoVxgMAziLyAosAzqJc5lNWwh3FcfCA3bnOf+JhBS/Qw3l+JiZaitZ0FjQ8kx5i0eQYp9XcJOA94aA/b5SLyTWAWtrvkhr1dQGMGiqIoTcDU1aX0AipEZFbwuiS8joi8KCJzY17nxn9yPCKSA5wDPBKYpwIHY7eRVgO/bOw6Ge0ZrK7awq1TbBbRoh1WhuL15V9xx+fe5vfvt0cr7vmn/8zZ/lLhV9Zd750GwItPvulsp//6e27c5Um7P1/8lp98Dxvvr//ITf8HQI/ig50tXEA8+YFtz/mfQb7+xhU+PtB9jI0PVM/wdQbZg7yQHljPoGqHv2h2IDS3dGOUIVTk4yCfrvN1DPlF/nN3RDUFeUEcYHMgnd05ihnU7vDxl9IgvpCID5QEtnAVnmhxGcYMCoJGQa45TRBAqZ8NtPcWl87WBDmKlpKYSDVzqKViA9q2Ms0Y0xTPYK0xZkzyS5lTkx0Tkab0dzkLeN8YUxVc241F5HfA03FvDFHPQFEUJVWiOoNUXvtJor8LNN7f5Xz22CKKJpAEXwHmNvaBGe0ZdO9WxH+PHwvAo71HAjDpmG+545V3fM2N53SwdQYX3Piis73e52U37jxgFABvfehtU77kM4dmH2xX3LNvf9DZRt/je/MsuPopALI/DmsH/Lf3n5VrAeg3uruzbVuz3H/+UdYLME+85Ww1XX3mUKIV5bJNvh4gp9jHLD7+3K7ic4tLnW1FsNoP4wMJz6Ck3McXdu3w1y0psJ7B7iBDqCTwIhLxgXC1n6gqBh8fCGsH4uIDeTEZRuCzhBptWNOE5jZNqR1IZ4ZPa6/r1bFoGoa0CdXF9ncRkZ7ANGPMuOjrAuA04L/2eP8UERkR3fLSmOMNyOjJQFEUJa0Yw+7qlp8MjDHriOnvYoxZBYwLvt4OdI4578I9bY2hk4GiKEqKGAN1RuUo2hwbS3vy+LnXA1A99UwAarb57l9PHn2FGz81wBaTFf3ufmf7619fc+NLHp0EQM4zv3K2wmfvcOMjfmgF7MJCspKChj0EHn7Tp/n+uKLAjX+z1PZm7nvyUGfb9ZDvXZA19JsASId3nW3FNr8CySnsBEDlWh/Uzevk+yF8XGVlLAo6+aK5rYEcRSg0t+oTm2HWNbDNC4LFnQujAHK4TZTfMLU0v942UcNgcVhIFrclFFdcZu1xBWYNz0sWQE41WNyUQrKmdCpriW0mDRS3HXbrZKAoinJgY6ifJdieyOjJYOXyz5g88VYAtlU+AUDXz3xx1aSr7nHjIVFq5uDTrnK2j4Jg8a9H2FXyvLP8av/V6/7uxid9YIvKFl/sC8nuf9sHgIcU2/c/M3e1sx18hr/W1nlLASg7wWeTmQef8MdL+wP1+xLPX+NTQ/NKrBfwYSBxXVDqg8WronNDD2DLer+y79Wrkxsv2Wa9gC7F3ouo3u6vWx6llobdy0rqidLZYHFxIO0RBosLshNFZ6FQXYwEdbBcb0xormOM3LUKzTXh89WxaDbUM1AURTnAqTNQrZ3O2h4FpeUc/rXzADj8l7Yf8RuD3nDHryr0K+c/P2P38mf87YvOtvBvPrVy/hWXATDslp862/Qh4914zSKbptkl16+Gn3h1qRtPO9IK3W345F/OdtAPT3Pj2vfeA6DuEP/5HXOe95+/NtqHL+vmbB+u9Kv1/M5WGntBYCsu90Vzm9fZ93fp7T2AqmVe7qJHqT+3Zpu9RhgzCL2Azi611MccQrnqRG/juEKy0B6u9rNiVvZZSTbXU00tTaJmkSQNteF5zbFYbqnr7g/qBbQsuk2kKIpygGMwuk3UFhncqZZ/nmRXv2VX2sygm37jC/UenOsLuFa/8QcA5Be+Oc3J03/kxjecdQMAa24a5Gw5wbLvrqfmAzBliG8485sP33Hjw759EgA7blvqbB2PvsyNO2TNAeCjrf6aoRfw+qc2w6ewixfCe2uxL2AriTKT1gcS3aVdvFz1soVWDnvMCF94OH+Tl8juUXK4G9fstNeoKPQtMsMCs065DSWoi3IaegFFOf7Xp14jmpgsoVQzhJLZmyInkepqvbUzhOx1dRmfSWgAWVEURQF0MmiTrKxcwU9OuBKAR2dbgbmq455zx4f83st/n/jIZMB7AACnftf3faiuux6AyX/5wNmmDPV5/NPftF7GmElnOdu2O7zQXM4ZVgCvw503Oltljc9sKuhsYwovLFrrbMU9B7jxKwusDlVZdx/nqFrpaybKukZyFAv9an/U4V7aovLNeQAc1NnXMYQZQj1LfOZQQoAuERuA+vGBkkhGY3cgS12S21CULpmcRE6MBHVMj/t63kKjQnVxdQZJluupegHNIUeR6qkt5QGoY5FejNFsIkVRlAMeg2YTtUk65WZxaj8r1lZ+pc38OePdP7njEwf6RjYjvmY9gpwOXsL6irt9TOGBU/oBcN8rLznbMTd92423Xmsrg3O+5iuUO/72ajd+e7PN1inq3s/ZHp3taw7K+h0GwDMf+OY1Xfp4SZGVS23so3tfn+FU+f4KNz5yuI0FLHx9trMN7Oab3zyxxVYzH1Tuq57DauyugYR1zU5bk1ARegaBF1AeVRuHMYP87KA5TWTPzYpf2cd5BjkdG3oRWUmCBnFZRnFeQEvJSseZ01knEF8nsV+XVJoJjRkoiqIogG4TKYqiHPDYmEFr30XLkNGTQe7gwfR/4R8A3NHNpk4uPM2nSE7o6lMvL77pUQDm3OADwLf8dYYbj37Cbv/s+vJdzrbqyOvcOL/MSk/8aZ4Xl0v0QACY+tonAHQf7AO4zwVyFT2jfggrgn4Ho4/o5cYzn7fbP/92Qn9ne/95X0A3rLftqvbQhs+c7Qud/fNVb7Wpqb0CobqaQHyuR7BNlOhHUJofSkw03BKq170sq+E+RV6STmU5Mds/2bGdypIVnaVoS1a0FmPf3+KwZFs/LbElpLRt1DNQFEU5wDFAWlrbtAIZPRksWFLFkRdameml91sJ6Iq7HnDHn5v9rBtvHnczAGsvnOJshS/f6ca/WmoDr92Hn+RsEx7zneL6H308AFOfWuBsA4/wncjeect6ASd80a/s//dxH6C+9OKTAbhr5tvOdvKFI9342T8uAWB0H9e3gp0bXBtTBldYL6A6CAofXOaDxbWRnEQ9DyBY7ZfmNfQCirLjU0MLsxp6BnFppHEeQDJ7XFA4zluAZD2Qmz8AnM7VvnoA7QOD0WwiRVGUAx2bTaSTQZujQ1Y2BZGA26UdbZrloFP8avr4qZVufMR/fAOAr97sZavPvMDHD26/5x8AfHv8sc527z1Pu/HPJts01Wt+6j2Pe2/zbUUveszGH86fNNbZHvqV9yJOH2Q//5bPfRzhqD6+wGznJluMdmgQ56je5ovG+pfaWEAoG9GlMBCPi+zl+fENZ0pyG67s60lMxEhQh8TFDHLjKslIEjNI0VuAZDGDuEK01Ff2cbZkq/WmrOx1xX+A0Y4DyMnkXVoNETlTRCpFZJGIXN34OxRFUdJDwjNI5ZVptCnPQEQ6Ar8FTgNWAO+KyAxjzPy48w8/qJzXp10AQKdjvw/A5jd+644nbKE9tP1rSnDuHVMBuO7EC5xtyjV+Zf+d4VZUbsIav7I/Z2CZG++Kir6O7+2b09Tu9Nk8I7raorRwZT+w1Bd9JVbxBxU3lH0A6BF5AaGta+AFJCjLjZ/fO+U0tBdnxy9rCxrJHEqQ5KOIu2ycLeZjkto70PA/V5ytKedKkv+wcfb9PfdAf386PyvZ+5uL9uoZtKnJADgSWGSM+QRARB4GzgViJwNFUZR0Ukf7laMQ04bcGRH5d+BMY8zF0dcXAkcZYy4PzrkEuCT6cigwt8GFMp8KYG2jZ2UW7fGZQJ8rkxhsjClu/LTkiMhz2O9NKqw1xpy5P5+XTtqaZxC3cVBvtjLG3AfcByAis4wxY9JxY+mkPT5Xe3wm0OfKJERk1v5eI5P+uDeVthZAXgH0Cb7uDaxKcq6iKIrSTLS1yeBdYKCI9BeRHOA8YEYj71EURVH2kza1TWSMqRWRy4HngY7AdGPMvL285b703FnaaY/P1R6fCfS5Mon2+EzNRpsKICuKoiitQ1vbJlIURVFaAZ0MFEVRlMydDNqDbIWI9BGRV0RkgYjME5EJkb1cRF4QkY+jf8sau1ZbREQ6isgHIvJ09HVGP5eIlIrIoyKyMPqZHZPpzwQgIpOi37+5IvKQiORl4nOJyHQR+VxE5ga2pM8hIpOjvx+VInJG69x12yEjJ4NAtuIs4FDgfBE5tHXvap+oBX5kjBkCHA18P3qOq4GXjDEDgZeirzORCcCC4OtMf65fAc8ZYw4BhmOfLaOfSUR6AT8AxhhjhmITN84jM5/r98CedQCxzxH9PzsPOCx6z93R35UDloycDAhkK4wx1UBCtiKjMMasNsa8H423YP+49MI+yx+i0/4AfLl17nDfEZHewJeAaYE5Y59LRDoBXwTuBzDGVBtjNpLBzxSQBeSLSBZQgK3tybjnMsbMBNbvYU72HOcCDxtjdhljlgCLsH9XDlgydTLoBSwPvl4R2TIWEekHjATeBroZY1aDnTCArq13Z/vMncBV1G8MlcnP9QVgDfBAtPU1TUQKyexnwhizErgNWAasBjYZY/6PDH+ugGTP0e7+huwvmToZNCpbkUmISBHwd2CiMWZzY+e3dUTkbOBzY8x7rX0vzUgWMAqYaowZCWwjM7ZO9kq0h34u0B/oCRSKyPjWvau00K7+hjQHmToZtBvZChHJxk4EDxpjHovMVSLSIzreA/i8te5vHzkOOEdElmK38E4WkT+T2c+1AlhhjEn0LX0UOzlk8jMBnAosMcasMcbUAI8Bx5L5z5Ug2XO0m78hzUWmTgbtQrZCRAS7B73AGHN7cGgG8K1o/C3gyXTf2/5gjJlsjOltjOmH/dm8bIwZTwY/lzHmM2C5iAyOTKdgpdUz9pkilgFHi0hB9Pt4CjZ2lenPlSDZc8wAzhORXBHpDwwE3mmF+2s7GGMy8gWMAz4CFgPXtvb97OMzHI91TWcDH0avcUBnbObDx9G/5a19r/vxjCcCT0fjjH4uYAQwK/p5PQGUZfozRc91A7AQKwf/JyA3E58LeAgb96jBrvwv2ttzANdGfz8qgbNa+/5b+6VyFIqiKErGbhMpiqIozYhOBoqiKIpOBoqiKIpOBoqiKAo6GSiKoijoZKC0Y0TkehH5cWvfh6JkAjoZKIqiKDoZKO0LEbk20qd/ERjc6BsURQGs+JaitAtEZDRW/mIk9nf7faA9ieUpSouhk4HSnhgLPG6M2Q4gIhmnV6UorYVuEyntDdVXUZR9QCcDpT0xE/iKiOSLSDHwb619Q4qSKeg2kdJuMMa8LyJ/xaq/fgq82sq3pCgZg6qWKoqiKLpNpCiKouhkoCiKoqCTgaIoioJOBoqiKAo6GSiKoijoZKAoiqKgk4GiKIoC/D81CPTumjnuAAAAAABJRU5ErkJggg==\n",
+ "text/plain": [
+ ""
+ ]
+ },
+ "metadata": {
+ "needs_background": "light"
+ },
+ "output_type": "display_data"
+ }
+ ],
+ "source": [
+ "EMBEDDING_DIM = 100\n",
+ "MAX_SEQUENCE_LENGTH = 100\n",
+ "MAX_NB_WORDS = 64\n",
+ "pos_encoding = positional_encoding(MAX_SEQUENCE_LENGTH, EMBEDDING_DIM)\n",
+ "\n",
+ "plt.pcolormesh(pos_encoding[0], cmap='RdBu')\n",
+ "plt.xlabel('d')\n",
+ "plt.xlim((0, EMBEDDING_DIM))\n",
+ "plt.ylabel('Position')\n",
+ "plt.colorbar()\n",
+ "plt.show()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "You have already created this visualization in this assignment, but let us dive a little deeper. Notice some interesting properties of the matrix - the first is that the norm of each of the vectors is always a constant. No matter what the value of `pos` is, the norm will always be the same value, which in this case is 7.071068. From this property you can conclude that the dot product of two positional encoding vectors is not affected by the scale of the vector, which has important implications for correlation calculations. "
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 5,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ ""
+ ]
+ },
+ "execution_count": 5,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "pos = 34\n",
+ "tf.norm(pos_encoding[0,pos,:])"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Another interesting property is that the norm of the difference between 2 vectors separated by `k` positions is also constant. If you keep `k` constant and change `pos`, the difference will be of approximately the same value. This property is important because it demonstrates that the difference does not depend on the positions of each encoding, but rather the relative seperation between encodings. Being able to express positional encodings as linear functions of one another can help the model to learn by focusing on the relative positions of words.\n",
+ "\n",
+ "This reflection of the difference in the positions of words with vector encodings is difficult to achieve, especially given that the values of the vector encodings must remain small enough so that they do not distort the word embeddings. "
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 6,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "tf.Tensor(3.2668781, shape=(), dtype=float32)\n"
+ ]
+ }
+ ],
+ "source": [
+ "pos = 70\n",
+ "k = 2\n",
+ "print(tf.norm(pos_encoding[0,pos,:] - pos_encoding[0,pos + k,:]))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "You have observed some interesting properties about the positional encoding vectors - next, you will create some visualizations to see how these properties affect the relationships between encodings and embeddings!"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "### 1.2 - Comparing positional encodings\n",
+ "\n",
+ "#### 1.2.1 - Correlation\n",
+ "\n",
+ "The positional encoding matrix help to visualize how each vector is unique for every position. However, it is still not clear how these vectors can represent the relative position of the words in a sentence. To illustrate this, you will calculate the correlation between pairs of vectors at every single position. A successful positional encoder will produce a perfectly symmetric matrix in which maximum values are located at the main diagonal - vectors in similar positions should have the highest correlation. Following the same logic, the correlation values should get smaller as they move away from the main diagonal.\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 7,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXEAAAEKCAYAAADkYmWmAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO2dfbBkZX3nP9/uO68MCANCRtCAAV+IUcxSCYZswgaTKLJgtmJCKhhi2CVbpRETtxRN7cbEsoqkjC95szImKr5EJcYXwkZXapQYNysGFRFEAwFEYMIw4MsMzNu9/ds/zjn3nJnue8+508/p22f6+6k6dbtPn/P083T3/fW3f+f7/B5FBMYYY7pJb7U7YIwx5vBxEDfGmA7jIG6MMR3GQdwYYzqMg7gxxnQYB3FjjOkwrQVxSe+StEPSbZV9myXdIOnO/O9xlcdeJ+kuSd+U9PNt9csYY6YFSfdK+pqkWyTdnO9bMk6Ook0l/h7gBYfsuwrYFhFnANvy+0g6E7gE+OH8nL+Q1G+xb8YYMy38p4g4KyLOzu+PjJNL0VoQj4jPAY8esvti4Jr89jXAiyv7PxQR+yLiHuAu4Mfa6psxxkwxS8XJkcy13p2DOSkitgNExHZJJ+b7Twa+UDnu/nzfEJKuAK4AWK/ef3jyuvXZA/nM0xhUZ6CqvNXPbvf61X3ld5h6ytsf3fHB/ACA+X0Li/v25PsA9g2GZ76u75WNbViTPVd/XfmSV59qsDAY6n9vruxfcV5/TXl+dbbt/J79AOx/7MDivr2VtqrPtS7v19y68sdO9bkWz6n0v/q4+v388XLfYL58XRb2Zn2Y3ztfPl4dV95uf305lrnK69Kby/vVW+rNiOHnPFDeHizeLs/vryv7umZj9pnprV1btll5qsG+7LWc37Ov0mb5Xldfi7mN67J968q2pMrrciB7LQb7y/el+h5rrvhcVM7vV36E5ofGQvlaxqDsS/EeqF/5V66eXwwsynOoztIu+lo5v/q+Lv5DLDWzOz8/ev2hfdWnP+SksiuV8xaK97XyVKNmlC9U9j22v3zfH92VvW97d+9e3Dc4UL6HseeRnRHxxNEDaUbvmFOC+b2Njo09j9wOVA/eGhFbDz0M+LSkAP4yf3ypODmSSQfxpRj1Vo/81OSD3ArwtA1HxV889Uyg/Cc+sLv8Z1ElYK/dtAaA9ceuX9y37gnrysePyh7vry0/VIOFsgt7v5O9F4/eWf64+OrDjy/evvfx8nkLnrax/Mf84S2bADjuqccu7utVgtSevP1q4Ft/XNnXzacfD8Cmk8vP4MLe/Yu3H7njgawfX3hgcd9dldei8lJwWj7WE04r+7LxhI1lv4ovvMprsfH4o8p+Hf+ErM315fj2PvK9xduPfvMhAHZ+s3yt9jxW9nXDUdl5m88oU32bTz+hfPyJ2f65DZUgW6H4wtrz8HcW9+3eXj7/rgd35+Mog8kTfvCYxdsnPvd0ADad9uTFfdXAufue+7L+3/pvi/see+ixxdvV1+qEZz8VgKNPP7Vsa235vu1/aHvWp/seWtx34PHy/3rD8Vm/jv7BJy3u6z3h+HKwg+xzvfC9RxZ3zVfOL96DuePKz0Xv6EoKNQ/Isa88J+Yr/yN5X/uV5+wddXTl/P5B/TiUWLMh+7u+PGewrvysHBTQF9ssw87Cuk2Lt7+7N3uO3fvLL5yFEUH8u5X/kZvu/+7i7Q/94z0A/OvnSz24a3v5Hh645d3fGjmIlTC/l7mnX9To0AO3vHtvJUWyFOdGxIN5oL5B0jdW2qVJu1MekrQFIP+7I99/P/DkynGnAA9OuG/GGLM8Eur1G21NiIgH8787gI+RpZGXipMjmbQSvw64DLg6//uJyv6/kfQW4EnAGcAXa1uLGFLgo9Q3lAp8lPqGUoGPUt9QKvBa9b1pWH1DqcBHqW8oFfgo9Q2lAh+lvqFU4HXqG0oFPkp9Q6nAR6lvKNXfKPUNpQIfpb6hVOCj1DeMVuCF+oZSgY9S39lYMm0ySn1DqcBHqW8oFXid+oZSgY9S31Aq8FHqG0oFPkp9Q6nAR6lvKBX4KPUNpQIfpb6hVOAj1fchfVlsM1ffUCrwWvUNiwp8lPqGUoGPUt/Zsdn/yCj1DaUCr6rvo7f80OLtR28Z3a2VIXpzo38hrrgl6SigFxG78ts/B/wBS8fJkbQWxCV9EDgPOEHS/cDv5Z26VtLlwH3ASwAi4nZJ1wJfB+aBl0fE6N9vxhizWuRKPBEnAR9Tdt1hDvibiPiUpH9hRJxcitaCeET8yhIPnb/E8W8C3rSi5xjEkAIfpb6hVOCj1DeUCnyU+oZSgY9S31Aq8FHqG0oFPkp9Q6nAR6lvKBX4KPUNpQKvU99QqspR6htKBT5KfUOpwEepbygV+Cj1nY0xU+CHo76hVOCj1DeUCnyU+oZSgY9S31Aq8Dr1DaWqHaW+oVTgo9Q3VBT4CPUNpQIfpb6hosBHqG8oFfgo9Q0VBX4Y6hsqCrxGfUOpwEepb2ie/x6lvqFU4FX1/bSfPGfx9hc+ObqLK0EccuF5DCLibuA5I/Y/whJxchTTcmHTGGOmH4leOiWehI4HcQ0p8Im6T0bkv1fbfVKnvsHuk6bukzr1DXafTJP7pFDgVfV9yU+ftni76mEeh4TplCR0PIgbY8wESZsTT4KDuDHGNESI3tya+gMnSKeDuPoaSqPMuoWwLoUCthA2tRDWpVDAFsKyzeVTKNC+hbBIo1RTKD9+Svn/kAQrcWOM6TYO4gnp9TWkwGfdQlinvsEWwqYWwjr1DbYQNp3AA+1bCAsFXlXfx65PHOKkZBbDVHQ6iBtjzCQRVuJJUb83pMBn3UJYp77BFsKmFsI69Q22EKaawAPjWwgLBZ5cfVdRj36iafep6HQQN8aYiSIr8aSopyEFPuvukzr1DXafpCofC3afNJ3Akx3brvtklALvL7VAwGEi7E4xxphO4yCeEGlYgc+6+6ROfYPdJ6nKx4LdJ0293zA590lVfW9am3jJBPvEjTGmyziIJ+dQBT7r7pM69Q12nyQrHwt2n0yR+6RQ4FX1fez6tAFXEr01dqcYY0w3cTrFGGO6jYN4Qgbzg6E0yqxbCOtSKGALYaoa4GALYXnc8ikUaN9CWKRRqimU/r7dQ+eMSzVGTAOdDuLGGDNJJCEH8XTM71sYUuCzbiGsU99gC2Gq8rFgC2HTCTzQvoWwUOAHqe9B+T+cin4/sW1xTDodxI0xZqIIK/GU7JkfLKvAZ9FCWKe+wRbCVOVjwRbCphN4oH0L4aICb0F9F2RVDB3EjTGmo4he4nos49LpIL5vEEMKfNbdJ3XqG+w+SVU+Fuw+aTqBBybgPhmlwGMwvG8cnE4xxphu4yDeEnafNPN+Z2O0+wTG936D3SdNvd8wQfdJRX339j02/PgYSNCfSxfEJfWBm4EHIuJCSW8A/hvwcH7I6yPiH5Zr44gJ4sYYMwmUNid+JXAHcExl31sj4s1NG+h0EF/fE0/beLACn3X3SZ36BrtPkpWPBbtPGnq/YQLuk1yBV9W39u4afexhIinZjE1JpwAvAt4E/M7htjNdrnVjjJly1FOjrQFvA14DHHr19RWSbpX0LknHjTjvIBzEjTFmBawgiJ8g6ebKdsViG9KFwI6I+NIhzb8D+CHgLGA78Md1/el0OmXDmt5QGmXWLYR1KRSwhTBVDXCwhbDpBB5o30JYpFGqKRQd2DN8zjiIlfjEd0bE2Us8di5wkaQLgPXAMZLeHxGXLj6V9E7g+ronsRI3xpiGCNGb6zXaliMiXhcRp0TEqcAlwGci4lJJWyqH/QJwW12fOq3E++vmhhT4rFsIx1XfYAth0wk8YAthqgk82bHjWQgLBZ5cfVdR66Vo/0jSWUAA9wK/WXdCp4O4McZMmsQWQyLiRuDG/PZLV3p+p4O4GFbgs24hrFPfYAthqvKx1b7OuoWwTn1D+xbCkQo88So8WQGspE2Ozap0R9JvS7pd0m2SPihpvaTNkm6QdGf+t9ZaY4wxEyVPpzTZJsXElbikk4FXAmdGxB5J15Il9s8EtkXE1ZKuAq4CXrtcW4OFwZACn3X3SZ36BrtPUpWPBbtPmk7ggQm6T2pe//HQQb9Qp4HV6s0csEHSHLAReBC4GLgmf/wa4MWr1DdjjBmJrMQhIh6Q9GbgPmAP8OmI+LSkkyJie37Mdkknjjo/N8xfAXDSmrVDCtzukxK7T9otHwt2n0yV+2TE9YfBY2mn3cP0VTGcuBLPc90XA6cBTwKOknTp8meVRMTWiDg7Is5+wlynr8saYzqGBP2eGm2TYjWi4POBeyLiYQBJHwV+AnhI0pZchW8BdqxC34wxZlkmGaCbsBpB/D7gHEkbydIp55PV030MuAy4Ov/7ibqGenO9oTTKrFsI61IoYAthshrgYAthTl0KBSZgIczfg2oKpfq5SIGYrMpuwmrkxG+S9BHgy8A88BVgK7AJuFbS5WSB/iWT7psxxiyHBGtrptRPmlVJKkfE7wG/d8jufWSqvDH9dXNDCnzWLYR16jsbiy2EkGACT6UvVWbRQlirvqF1C2GhwKufi9hfvscpkGBu1pW4McZ0FeGceFL6a+aGFPisWwjr1DfYQjip8rEwOxbCcdU3jG8hLD4XVfWtufJ/PAlyTtwYYzpLpsSdE09GRAwp8Fl3n9Spb7D7JFX5WLD7pPEEHmjdfVIo8Kr61rryPU6FlbgxxnSUnmR3Skrm9+wfUuCz7j6pU99g90mq8rFg90lT7ze07z4pFPhB6ntw6BrE49NPXE98XDodxI0xZpIU0+6niU4H8f2PHRhS4LPuPqlT32D3id0nR6b7ZFGBV9T3YFf5yzUVDuLGGNNRPNnHGGM6jPCFzaTsHcRQGmXWLYR1KRSwhTBZDXCwhbCgJoUCE7AQ5mmUagpl/jsPDz/nGDgnbowxHcbT7hMjhhX4rFsI69Q32EI4qfKxMDsWwnHVN4xvISwUeFV9VwviJcFK3BhjuovriSdmXU9DCnzWLYR16htsIUxWPhZsIWw6gQdatxAWCryqvuc2etq9McaYnJ4XhUjL3Lr+kAKfdfdJnfoGu08W2xlzAg/YfdJ4Ag+07j4pFHhVfVc/F0lInBOX1CdbnvKBiLhQ0mbgw8CpwL3AL0XEsjOWpusrxRhjphgh+mq2NeRK4I7K/auAbRFxBrAtv78snVbivbnekAKfdfdJnfoGu0/sPjky3SeFAj9IfVf7koheogJYkk4BXgS8CfidfPfFwHn57WuAG4HXLtdOp4O4McZMkqqtuQEnSLq5cn9rRGyt3H8b8BqgokA4KSK2A0TEdkkn1j1J54P4oQp81t0ndeob7D5JVT4W7D4pD1xefUP77pNFBV7py6Ci+pMg6DXPie+MiLNHNiNdCOyIiC9JOm+cLnU+iBtjzKQQsCbN8mznAhdJugBYDxwj6f3AQ5K25Cp8C7CjriFf2DTGmIYU6ZQm23JExOsi4pSIOBW4BPhMRFwKXAdclh92GfCJuj51Womrp6E0yqxbCOtSKGALYaoa4GALYdMJPDABC2Hel2oKZde3Hhzd78NFWkk65XC4GrhW0uXAfcBL6k7odBA3xphJItK5Uwoi4kYyFwoR8Qhw/krO73QQ7831hhT4rFsI69Q32EI4qfKxMDsWwjr1De1bCAsFXlXfex75/nD/x2QF7pSJ0Okgbowxk0SCNf3pupTY6SCufn9Igc+6hbBOfYMthGWb403gyY61hRDq1Te0byEsFHhVfa9JXACrjXTKuHQ6iBtjzKRxOiUh6vWGFPisu0/q1DfYfZKqfCzYfdJ4Ag+07j4pFHhVfR/9lJNGj+swEbISN8aYzuKVfdIymF8YUuCz7j6pU99g94ndJ5NfvBjad58UCryqvteetGV4fGOQ5cSTNjk2nQ7ixhgzSRJOu09Gp4P4wt4DQwp81t0ndeob7D5JVT4W7D5p6v2G9t0nhQKvqu9qX5MgmDKH4erUTpF0rKSPSPqGpDskPU/SZkk3SLoz/3tcfUvGGDM5Cothk21SrNZ3ytuBT0XEM4DnkK1sseIVLYwxZrIkX9lnbCaeTpF0DPBTwK8DRMR+YL+kFa9oMb93fiiNMusWwroUCthCmKoGONhC2HQCD7RvISzSKNW+7rrrXlLiyT4ZTwUeBt4t6TnAl8jWmWu0ooWkK4ArAE6sfJiMMaZtsmn3DuJzwI8CvxURN0l6OytIneTLG20FOH3NhjhUgc+6hbBWfYMthInKx4IthE0n8ED7FsKir1X1vfPWu0nNlAnxVcmJ3w/cHxE35fc/QhbUH8pXsqDpihbGGDNpeqjRNikmrsQj4t8lfVvS0yPim2S1c7+eb5eRFUVvtKJFr6chBT7rFsI69Q22EJbHjTeBB2whbDqBB9q3EBYKvKq+H9/5OCkR06fEV8sn/lvAByStBe4GXkb2q2BFK1oYY8yk8YxNICJuAUatAr2iFS366+eGFPisu0/q1DfYfZKqfCzYfdJ4Ak+lL225TwoFXlXfR51U+TWXAk2fEm+UE5f0X/JJON+T9H1JuySlXzLDGGOmGHXYJ/5HwH+OiDva7MxKmVs3N6TAZ959UqO+we6TVOVjwe6Tpt5vaN99Uijwqvo+4dnle5iKrqZTHpq2AG6MMavBlMXwxkH8ZkkfBj4O7Ct2RsRHW+mVMcZMIV2esXkM8Djwc5V9AaxqEO/N9ZdNo8yihbAuhQK2EKaqAQ62EDadwAPtWwiLNEo1hbLptKeQmimL4c2CeES8rO2OGGNMF5iySrTNgrikU4A/Bc4lU+CfB66MiPtb7Fs9PQ0p8Fm3ENapb7CFcFI1wLNjZ8RCWKO+oX0LYaHAq+o7FkZfXD9cNIXLszX9Unk3cB3wJOBk4O/zfcYYM1NIzbb6drRe0hclfVXS7ZJ+P9//BkkPSLol3y5Yrp2mOfEnRkQ1aL9H0qsanjsRbCFsOIEHbCFMVD4WbCFsOoEH2rcQFgq8qr533/NtUiKSplP2AT8TEbslrQE+L+mT+WNvjYg3N2mkaX92SrpUUj/fLgUeqT3LGGOOMCQ12uqIjCLPtibfRucIl6GpEv8N4M+At+ZP8s/5vtVlEEMKfNbdJ3XqG+w+SVU+Fuw+aTqBB9p3nxQKvKq+d3zlLpKiFU32OUHSzZX7W/NS2mVzUp9sTYXTgT/Py3O/EHiFpF8DbgZeHRHfYQmaulPuAy5q3HVjjDkCEbCCNSF2RsSoGlGLRMQCcJakY4GPSXoW8A7gjWSC+Y3AH7OMaF42iEt6TUT8kaQ/ZYTMj4hX1g6jRQbzC0MKfNbdJ3XqG+w+sftk8ivQQ/vuk0KBV9X3976VvsRTk1TJSomI70q6EXhBNRcu6Z3A9cudW6fEi6n2Ny97lDHGzADZjM1EbUlPBA7kAXwD8HzgDyVtKZaqBH4BuG25dpYN4hHx9/nNxyPibw/pwKrX+144sDCkwGfefVKjvqt9tftkPPUNdp809X5X+9qW+6RQ4FX1PVgoP1epSKjDtwDX5HnxHnBtRFwv6X2SziLLftwL/OZyjTS9sPk64G8b7DPGmCMYJaudEhG3As8dsf+lK2mnLif+QuAC4GRJf1J56BhgCelhjDFHKFO4KESdEn+QLB9+EZkNpmAX8NttdaopgwMLQ2mUWbcQ1qVQwBbCZDXAwRbChhN4oH0LYZFGqaZQjn5S+blLgSLQEmnG1aIuJ/5V4KuSPhARVt7GmJlHkT7PPg516ZRrI+KXgK9IqkolkU04enarvatFQwp81i2EdeobbCGcVPlYmB0LYZ36hvYthIUCr6rvTVueQFrioPd6GqhLp1yZ/72w7Y4YY0wnWEJ8rBZ16ZRCau4E9kTEQNLTgGcAn1z6zMnQX9cbUuCzbiGsVd9gC2HO2BN4wBbChhN4oH0LYaHAq+q7WDQmGTF9SrxpAazPAeslnQxsA14GvKetThljzLSiGDTaJkVTn7gi4nFJlwN/mk/F/0qbHWvCmo3rhxT4rLtPatV3pS9V7D5Z+QQesPuk6QQeaN99UijwqvoetWzjeMTSn4tVonEQl/Q84FeBy1d4rjHGHBkEU5dOaRqIX0U2Q/NjEXG7pKcCn22vW83orV07pMBn3X0yrvoGu0/sPklfPhbad58st2h6OuKg92IaaFqK9h+Bf5R0tKRNEXE3sKoVDI0xZjXolE+8QNKPAO8FNmd39TDwaxFxe5udq+/YsAKfefdJjfoGu09SlY8Fu0+aer+hfffJKAVeXTg9GV0M4sBfAr8TEZ8FkHQe8E7gJ1rqlzHGTB8RS4qj1aJpED+qCOAAEXGjpKOWO8EYY45EOplOAe6W9D+B9+X3LwXuWeb4iTDYt38ojTLzFsKaFArYQjipGuAwOxbCuhQKTM5CWE2hFCt/paO7k31+A3gi8NF8O4Fswo8xxswWMWi2TYi6Aljrgf9OthLz18hWXT6w3DmTZH7PviEFbgthiS2ELZePBVsIG07ggfYthIUCr6rvR+/aOfLYw2YKp93XpVOuAQ4A/wS8EHgmmWfcGGNmDtG9nPiZEfEjAJL+Gvhi+11qzuDAYEiBz7qFsFZ9gy2EBeNO4AFbCBtO4IH2LYSFAq+q70fvbCEnvjBd7pS6nPjib/vUi0JI6kv6iqTr8/ubJd0g6c78b+LyY8YYMybFtPuu5MSB50gqvjYFbMjvF4tCHLP0qbVcCdxBtl4nwFXAtoi4WtJV+f3XLtdAb643pMBn3X0yrvoGu0/sPklfPhbad58UCryqvvc8ln6yz7SlU5ZV4hHRj4hj8u3oiJir3D7sAC7pFOBFwF9Vdl9MloMn//viw23fGGPaITqnxNvibcBrgIrU5KRiEYqI2C7pxFEnSroCuALglGOPHlLgs+4+qVXfYPdJovKxYPdJU+83tO8+KRR4VX1vOKqFYlhdUuJtIOlCYEdEfOlwzo+IrRFxdkScffymDfUnGGNMKopp9022CbEaSvxc4CJJFwDrgWMkvR94SNKWXIVvAXbUNdRbt3ZIgc+8+6RGfYPdJ6nKx4LdJ02939C++6RQ4FX1fcLTN5eNlT82xiAO+jU9DUxciUfE6yLilIg4FbgE+ExEXApcB1yWH3YZ8IlJ980YY5YlSKbEJa2X9EVJX5V0u6Tfz/evyKk38SC+DFcDPyvpTuBn8/vGGDM1BEEsLDTaGrAP+JmIeA5wFvACSedQOvXOIFvT+KrlGlnVJdYi4kbgxvz2I8D5Kzlf6g2lUWwhrGAL4eKuNmqAgy2ETSfwQPsWwiKNUk2hbH56+brwyaGnWjlBspV9IiKAIie4Jt+CzKl3Xr7/GrIYuaTdepqUuDHGTDkrurB5gqSbK9sVh7aWT3q8hewa4A0RcROHOPWAkU69gk4vdjw4cGBIgc+8hbBGfYMthOWB403gAVsIm07ggfYthIUCr6rv9ceXz5+EWNGFzZ0RcfbyzcUCcJakY4GPSXrWSrvU6SBujDGTJYgW7IMR8V1JNwIvYIVOvU4H8cH+A0MKfNYthHXqG2whTFY+FmwhbDiBB9q3EBYKvKq+q7+AklC4UxIg6YnAgTyAbwCeD/whpVPvaho49TodxI0xZrJEsgubwBbgGkl9suuT10bE9ZL+H3CtpMuB+4CXLNdIp4N4DGJIgc+6+2Rc9Q12nzSdwAN2n6SawAPju08KBV59/fc+8r2h5x+LoKl9sL6piFuB547YvyKnXqeDuDHGTJburnY/lWiuN6TAZ919Uqe+s2PtPoHxvd9g90lT7ze07z4p3oOq+n78kfLXWhJW5k6ZCJ0O4sYYM1msxI0xprskdKekotNBvL9u7XAaZcYthHUpFLCFMFUNcLCFsOkEHmjfQlikUaoplMH+tAE3CCKdOyUJnQ7ixhgzUazE06J+f0iB20JYYgthuzXAwRbCabIQFgq8qr4HC6N/mR42EcSB9Ot2jkOng7gxxkyWpJN9ktDtIF75aWMLYXHc8uobbCFMVT4WbCEsqFPf0L6FsFDgVfVdfa2S4XSKMcZ0lGinANY4dDqIx8L8kAKfdfdJnfoGu0+SlY+t9GXW3Sd16hvad58UCrz6Wu28p/x/SIXdKcYY01UiiAUH8WTEYDCkwGfdfVKnvsHuk1TlY8Huk6beb2jffVK8VlX1fc9jaafIRwSDA0t8rleJTgdxY4yZKIGVeErU6w0p8Fl3n9Spb7D7JFX5WLD7pKn3G9p3nxQKvKq+U9vEwUHcGGM6S0QwSFRPPBUO4sYYswLsTkmI+nPDaZQZtxDWpVDAFsJUNcDBFsKmE3igfQthkUapplBO31R+LkhRWtzuFGOM6S52p6Sm3x9S4LNuIaxT32ALYarysWALYdMJPNC+hbA4tKq+Tz3n5HJgn/g6KRhYiRtjTEexxTA1GlLgs24hrFPfYAthqvKxYAth0wk80L6FsFDgVfV9/DOrSpzxcU7cGGO6S2B3SlpiMKTAZ919Mq76zo61+wTq1TfYfdJ0Ag+07z4pFHhVfVcL4iUhgsF+X9g0xphuEjCwEk9IxJACn3X3SZ36BrtPUpWPrfZ11t0ndeob2nefFAq8qr53P1B+LlIQTF9OfIncgDHGmCEi+8JtstUh6cmSPivpDkm3S7oy3/8GSQ9IuiXfLliunW4rcfWGFPisu0/q1DfYfZKqfCzYfdLU+w3tu08KBV5V34/eVX7G0hApL2zOA6+OiC9LOhr4kqQb8sfeGhFvbtLIxJX4Mt8+myXdIOnO/O9xdW0ZY8xEyX3iTbbapiK2R8SX89u7gDuAk5c/a5jVSKcU3z7PBM4BXi7pTOAqYFtEnAFsy+8bY8zUEBEs7J9vtAEnSLq5sl2xVLuSTgWeC9yU73qFpFslvatO0E48nRIR24Ht+e1dkopvn4uB8/LDrgFuBF67bGP9ueE0ii2Ei9hC2G4NcLCFsOkEnmws7VoIizRKNYWy9zvl+5aGFaVTdkbE2XUHSdoE/B3wqoj4vqR3AG8ks6W/Efhj4DeWOn9Vc+KHfPuclAd4ImK7pBOXOOcK4AqAp/zAyEOMMaYdEk+7l7SGLIB/ICI+ChARD1Uefydw/XJtrFoQH/Ht0+i8iNgKbAU4+1nPiCEFPuMWwjr1DbYQpiofC7YQNp3AA+1bCAsFXs6TVEkAAAtsSURBVFXfc0tc6D9sAiLRckHKgt5fA3dExFsq+7cUghb4BeC25dpZlSA+6tsHeKjovKQtwI7V6JsxxixFECmrGJ4LvBT4mqRb8n2vB35F0llk6ZR7gd9crpGJB/Glvn2A64DLgKvzv/XlaqRlFfgsWghr1TfYQpiofCzYQth0Ag+0byEsFHhVfW84rvIZS0FADNIo8Yj4PDAqBfEPK2lnNZT4Ut8+VwPXSrocuA94ySr0zRhjliQCFvbXT+SZJKvhTlnq2wfg/BU2NqTAZ919Mq76BrtPmk7gAbtPmk7ggfbdJ4UCr6rvQSLVvEhEspx4Kro9Y9MYYybMwEG8Hew+aej9BrtPEpWPBbtPmnq/oX33SaHAq+r7O3eXfUmCV/YxxpjuErSQohkTB3FjjGlKhC9sJkW9oTTKzFsIa1IoYAthqhrgYAth0wk80L6FsFDI1RTK7dsr6cUERMLJPqnodhA3xphJ4iCeluj1hxW4LYQlthAu7mujBjjYQth0Ag+0byEsFHhVff/r7vJ1SUPSGZtJ6HQQN8aYiZJwxmYquh3E1Vtegc+ihbBGfYMthKnKx4IthE0n8ED7FsJCgadX3yWBfeLGGNNdIg76kpsGuh3ExbACn3H3ybjqG+w+aTqBB+w+aTqBB9p3n4xS4KduLPvF40MPr5is0oeVuDHGdJaECyUn4cgJ4nafAA3UN9h9kqh8LNh90tT7DZNzn1TV93OeuLF84FtDh66cCCtxY4zpLPaJp0ZDCnzW3Se16hvsPklUPhbsPmnq/c7G2K77pFDgVfW9+YzytUqhxAMXwDLGmO4SwcJ+B3FjjOkkETBY4rraatHpIB69/lAaxRbCCrYQLu5rowY42ELYdAIPtG8hLNIo1RTK+tRrbLK0OWK16HQQN8aYSRIcfH1gGuh0EF8YxJACn3kLYY36BlsIU5WPBVsIm07ggfYthIUCr6rvXn+p5XwPHytxY4zpKIOA/S6AlY5BLK/AZ9JCWKO+wRbCVOVjwRbCphN4oH0LYaHAq+q7jVV4nE4xxpiOEoTTKSmJGH5BZ919Uqe+we6TSZWPhdlxn9Spb2jffVIo8Kr63l8ZawpSXtiU9GTgvcAPAANga0S8XdJm4MPAqcC9wC9FxHeWameJZXCMMcaMYiGabQ2YB14dEc8EzgFeLulM4CpgW0ScAWzL7y9Jp5V4FbtPmnm/we6TVOVjwe6Tpt5vaN99Uijwqvre9719Q+2PQ0Q6d0pEbAe257d3SboDOBm4GDgvP+wa4EbgtUu1c8QEcWOMaZtgRe6UEyTdXLm/NSK2jjpQ0qnAc4GbgJPyAE9EbJd04nJP0ukgvhAxpMBn3X1Sq77B7pNE5WPB7pOm3m9o331SKPCq+t773bKvKVhhTnxnRJxdd5CkTcDfAa+KiO9LK/O2dzqIG2PMpEnpTpG0hiyAfyAiPprvfkjSllyFbwF2LNeGL2waY0xDouFFzSZqXZnk/mvgjoh4S+Wh64DL8tuXAZ9Yrp1OK/HH9i8MpVFm3UJYl0IBWwhtIZz8CvTQvoWwSKNUUyj7d6e1GEJSJX4u8FLga5Juyfe9HrgauFbS5cB9wEuWa6TTQdwYYyZJkBm6k7QV8Xmy5d5HcX7TdjodxB/dtX9Igc+6hbBOfYMthKnKx4IthKkm8MD4FsJCgVfVd+ql1IJw7RRjjOkqmTvFQTwZe3fvHlLgM28hrFHfYAthqvKxYAth0wk80L6FsFDgVfW9pvJaJKH5bMyJMXXuFEkvkPRNSXdJWna6qTHGTJJCiTfZJsVUKXFJfeDPgZ8F7gf+RdJ1EfH1UccPDuwbUuCz7j6pU99g90mq8rFg90nTCTzQvvukUOBV9d1fU/kfS8S0KfGpCuLAjwF3RcTdAJI+RFZHYGQQN8aYSTJg+haFUExRkl7SLwIviIj/mt9/KfDjEfGKyjFXAFfkd58F3DbxjrbPCcDO2qO6xZE4JvC4usTTI+LocRqQ9Cmy16YJOyPiBeM8XxOmTYmP8kwe9C2TF5DZCiDp5ia1CbrGkTiuI3FM4HF1iUOKUR0WkwjKK2XaLmzeDzy5cv8U4MEljjXGmJln2oL4vwBnSDpN0lrgErI6AsYYY0YwVemUiJiX9Arg/wB94F0Rcfsyp4yszXsEcCSO60gcE3hcXeJIHNN0Xdg0xhizMqYtnWKMMWYFOIgbY0yH6WwQPxKm50t6sqTPSrpD0u2Srsz3b5Z0g6Q787/H1bU1jUjqS/qKpOvz+50el6RjJX1E0jfy9+x5XR8TgKTfzj9/t0n6oKT1XRyXpHdJ2iHptsq+Jcch6XV5/PimpJ9fnV6PTyeDeGV6/guBM4FfkXTm6vbqsJgHXh0RzwTOAV6ej+MqYFtEnAFsy+93kSuBOyr3uz6utwOfiohnAM8hG1unxyTpZOCVwNkR8SwyQ8EldHNc7wEO9XGPHEf+f3YJ8MP5OX+Rx5XO0ckgTmV6fkTsB4rp+Z0iIrZHxJfz27vIgsLJZGO5Jj/sGuDFq9PDw0fSKcCLgL+q7O7suCQdA/wU2XJaRMT+iPguHR5ThTlgg6Q5YCPZ3IzOjSsiPgc8esjupcZxMfChiNgXEfcAd5HFlc7R1SB+MvDtyv37832dRdKpwHOBm4CTImI7ZIEeOHH1enbYvA14DQcvhNLlcT0VeBh4d54i+itJR9HtMRERDwBvJlsGbDvwvYj4NB0fV4WlxnHExJCuBvHa6fldQtImshWvXxUR3687ftqRdCGwIyK+tNp9Scgc8KPAOyLiucBjdCPFsCx5jvhi4DTgScBRki5d3V5NhCMmhnQ1iB8x0/MlrSEL4B+IiI/mux+StCV/fAuwY7X6d5icC1wk6V6yVNfPSHo/3R7X/cD9EXFTfv8jZEG9y2MCeD5wT0Q8HBEHgI8CP0H3x1Ww1DiOmBjS1SB+REzPlySyHOsdEfGWykPXAZflty8DPjHpvo1DRLwuIk6JiFPJ3pvPRMSldHhcEfHvwLclPT3fdT5ZieTOjinnPuAcSRvzz+P5ZNdmuj6ugqXGcR1wiaR1kk4DzgC+uAr9G5+I6OQGXAD8K/BvwO+udn8Ocww/SfYT7lbglny7ADie7Er6nfnfzavd1zHGeB5wfX670+MCzgJuzt+vjwPHdX1M+bh+H/gGWVnn9wHrujgu4INkef0DZEr78uXGAfxuHj++Cbxwtft/uJun3RtjTIfpajrFGGMMDuLGGNNpHMSNMabDOIgbY0yHcRA3xpgO4yBupgZJC5Juyavp/a2kjSs8/0mSPpLfPkvSBZXHLupqtUtjlsMWQzM1SNodEZvy2x8AvhQHT4JaSVu/TlaZ7xUJu2jM1GElbqaVfwJOz+tBf1zSrZK+IOnZAJJ+Olftt+QFqY6WdGqu4tcCfwD8cv74L0v6dUl/lp/7g5K25W1uk/SUfP97JP2JpH+WdLekX1y10RvTEAdxM3XkJVFfCHyNbDbhVyLi2cDrgffmh/0P4OURcRbwH4E9xfmRlSf+X8CHI+KsiPjwIU/xZ8B78zY/APxJ5bEtZDNpLwSuTj02Y1LjIG6miQ2SbiGb2n4fWV2ZnySbCk5EfAY4XtITgP8LvEXSK4FjI2J+Bc/zPOBv8tvvy5+j4OMRMYiIrwMnjTUaYybA3Gp3wJgKe3JlvUhelOlQIiKulvS/yWrNfEHS84G9h/m81QtD+6pPf5jtGTMxrMTNtPM54FcBJJ0H7IyI70v6oYj4WkT8IZlyf8Yh5+0Cjl6izX8mq65I3vbnk/famAnhIG6mnTcAZ0u6lSxHXZQVfVV+EfOrZPnwTx5y3meBM4sLm4c89krgZXmbLyVbC9SYTmKLoTHGdBgrcWOM6TAO4sYY02EcxI0xpsM4iBtjTIdxEDfGmA7jIG6MMR3GQdwYYzrM/wdpze+20qWPcAAAAABJRU5ErkJggg==\n",
+ "text/plain": [
+ ""
+ ]
+ },
+ "metadata": {
+ "needs_background": "light"
+ },
+ "output_type": "display_data"
+ }
+ ],
+ "source": [
+ "# Positional encoding correlation\n",
+ "corr = tf.matmul(pos_encoding, pos_encoding, transpose_b=True).numpy()[0]\n",
+ "plt.pcolormesh(corr, cmap='RdBu')\n",
+ "plt.xlabel('Position')\n",
+ "plt.xlim((0, MAX_SEQUENCE_LENGTH))\n",
+ "plt.ylabel('Position')\n",
+ "plt.colorbar()\n",
+ "plt.show()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "#### 1.2.2 - Euclidean distance\n",
+ "\n",
+ "You can also use the euclidean distance instead of the correlation for comparing the positional encoding vectors. In this case, your visualization will display a matrix in which the main diagonal is 0, and its off-diagonal values increase as they move away from the main diagonal.\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 8,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "(100, 100)\n"
+ ]
+ },
+ {
+ "data": {
+ "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWsAAAEKCAYAAADU7nSHAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO2de7AkZ3nef8+cszdpV5dlJXmRcCRiBZAvyI7KAeM4BGEbCJGolIlFWY7AqlJShUGycUCyyyFJlasUx0WMjJ3SxlyELQOyzEXGMaBagzFxLLwyAiSEIiwZsWil1XKRVtJqL+e8+WO6d74903O6z0z3zPTp51c1NT09X3/9fTNn3nnOO8/7tSICY4wx801v1gMwxhhTjoO1Mca0AAdrY4xpAQ7WxhjTAhysjTGmBThYG2NMC2gsWEt6j6T9ku5O9m2XdLuk+7P705PnrpP0NUn3SfrppsZljDFtpEll/T7gFSv2XQvsjojzgd3ZYyRdAFwGfH92zO9JWmhwbMYY0yoaC9YR8Vng2yt2XwrclG3fBLwm2f/BiDgcEQ8CXwN+tKmxGWNM21ic8vnOioh9ABGxT9KZ2f6zgb9J2u3N9g0h6SrgKgAWNv7TxW1nZc/0KzFHVWRKvfz4wc5eb+j5UcTycv9+6ejxfctLxwbPJ9vH+1wYvLy9xQ1D+04g75/B+NMx9bLj1EvGn0x1eWkJgKWjh5MxLxWeSr2FE/pM92UnHj4m2XfCGPJzJa97Ppb0Nclfv/7x2XuRvj7J+fP+C4aRnSvvM5J9yXbJe7Gw2D9/b6H4PV9eWs7ukzklr2X6vhT1lQ57ORvjcjJWkrHmc+31Rry++VzT+aV/4lnTXun7c8KjoQ6SjwIL6VgKjkj7yse9kJx/oeD8Ken7mrZdzPsqOH9K+vym5HXfcOwZAA7vf+T4vkf2P3V8+wBHDkTEGasOroTeKecE2XnKiEPf+mRErMwujM20g/Uoit6TwqgbEbuAXQAbTv/eeNbL3gIMPkzLx44UnmBx4xYAFrJ7gIVNg+3e4sb+QHrF2Zdjh54E4JknHju+75nvPHp8+/DBlf9EwKZt249vb3nWswHYfGrx38rS4UND40/HuuW0fl+btmw4vi8NVk9+92D//pEHj+878uR3Cs+1cevpQ2NZ3LL1+Hb+GqSvxeKGwXY+hjQoHD08CJBPP/4EAM88Pnit8vnB4HVPz79l27bB+LL+eyM+9EvHlofOefjQ4Evq6FOPDx2z+dRnHd/etr1//pNP2VzY/1NP9D+MT3538KE88vTgQ79h80mr9tVbGIz7maf7X+6HDg7e16WlwRfXxk39j+CWrRuP79u0ZfCxzIP80cODL4v0+DxIb0yOyftMSb8sltPjs2C3ZfPgmG3Jdh4Yl5LjjyXbJ23s/12cumUw/qLj0z42Lg4C7KknDf6ezzh5U39fcnyv4Bt7ezLX804bvO5nfuc+AP7ht3/z+L7/fsNAA97I178+1NlaOfYMi8+7pFLTo3e9d8fE50uYdrB+VNLOTFXvBPZn+/cCz0nanQM8POWxGWPM6kgjBV3TTDtY3wZcAVyf3X8s2f9Hkt4BPBs4H/h8eXexqqJeTFV0rqwL1DQUK+pcTcNAUY+jpqFYUadqMx9/kZqGgZotUtMwUNRlajodS5GaTreL1HT/+b7aKVLTMFDURWo6PX+RmoZiRZ2r6fS846hpKFbUuZqGgaIuU9NpX0VqGgaKukhNw0BRF6lpGCjqIjUNA0VdpKbTvorUNAwUdZkaLlLTMFDUZcfDQFEXqWkYKOoiNQ0DRV2kpmGgqFM1/R/f/KLj2zfeMLmwBp0QN6ZJY8Fa0geAlwI7JO0F3k4/SN8i6UrgIeC1ABFxj6RbgK8Ax4A3RkRxwtUYY2bFelTWEfG6EU9dPKL9bwC/scZzDCnqIjUNA2U3jpqGgaIuUtMwUNTjqOl0rEVqGgaKukhNQ7GiLlLTMFDURWoaBoq6SE3DQNkWqel0jkVqGgaKehw1DQNFXaSm++fqK+px1DQMFHWZmoaBoi5S0zBQxEVqGgaKukhNp8cXqemV/eYU5aeL1DQMFPEoNZwr6iI1XeX4ovx0kZpeOcecovx0kZqGgaJO1fS517x10NkNHxrqf60I0MI6C9bGGLPukE5wLk2TdROs7fboU6amwW6Pqm6PMjUNdnsUHT8rt0euqFM1vf/05w31PynrLg1ijDHrjvWYszbGmPWG0PECt2nT6mAt9YbSH1235pWlPtJtW/MmK3QBW/OqFrpA89a8PP2Rpj4e/G61asPKWFkbY0w7cLAeA0lDirrr1rwyNQ225lW15pWpabA1r2qhy8o55tRpzcsVdaqmv31oeJ2YiZBs3TPGmHlHWFmPR683pKi7bs0rU9PpGGzNm6zQBWzNq6vQBSa35uWKunY1naIeC+ut3NwYY9YdsrIeC6m3qqLuotujTE3329jtsbKfcQpdwG6PqoUu0Lzbo0hRL49Y335chN0gxhjTChysx2TlC9d1t0eZmga7Pepa1hTs9qjqnYbpuT1SNf34Mw24QRysjTFm3nGwnhi7Pap5p8Fuj7qWNQW7PebJ7ZEr6lRNP/bU4cK24yKJ3ga7QYwxZr5xGsQYY9qBg/UYxPLyUPqj69a8stQH2JpX1xrUYGteTlnqA5q35uXpjzT18XiS6qqLUem/pumVNzHGGAP9nLV61W4V+nqepLuS2xOSrhnVvt3KeunokKLuujWvTE2nc7Q1b7JCl7SPrlvzytQ0NG/NyxV1qqaPJH+jdbGwUI/GjYj7gAsBJC0A3wQ+Mqp9q4O1McZMFVFJNY/BxcDfR8TXRzVodbBeXjq2qqLuojWvTE2DrXl1LWsKtuZVLXSB5q15uaJO1fRCzYG1v+pe5T53SNqTPN4VEbtGtL0M+MBqnbU6WBtjzHTRyJRQAQci4qLSHqWNwCXAdau1a3WwjqVjQ4q6626PMjWdjsFuj8kKXcBuj6qFLtC82yNX1KPy87XQTBrklcDfRcSjqzVqdbA2xphp00Cwfh0lKRBYR8Habo9q3mmw26OuZU1X9lvUV1fcHmVqGpp3exTl5w/WvJCTBAuL9QVrSScBPwn8+7K26yZYG2PMNFD1nHUpEfE08KzShrQ8WGthcUhRd93tUaamwW6PaS1r2h9rN9weZWoamnd75K9BqqYfPzT4u6gDSTOrYGx1sDbGmGnTkM+6FAdrY4xZAw7WY9Bb3LBq+qOL1ryy1AfYmlfXGtRpX1235pWlPqB5a16e/khTH08fGbzHtaDRP8Y2TauDtTHGTBMheouzWf+u1cFaC4tDirrr1rxJ1TTYmle10AVszaur0AUmt+blijpV04t1pyw0uyVSWx2sjTFm2tRp3VsL6yZY25pXrdAFbM2zNW/6VxyH5q15uaJO1XQzCznV2mVlZnJaSb8k6R5Jd0v6gKTNkrZLul3S/dn96eU9GWPMFMnSIFVudTN1ZS3pbODNwAURcUjSLfSXB7wA2B0R10u6FrgWeNuqnS0vDynqrrs9ytR0el67PSZT02C3R9VCF2je7ZEr6kYXckInvP/TZFaX9VoEtkhaBE4CHgYuBW7Knr8JeM2MxmaMMYWoS8o6Ir4p6beAh4BDwKci4lOSzoqIfVmbfZLOLDpe0lXAVQC9k04fUtR2ewyw26PZZU3Bbo95cntMYyEnmF1RzNSVdZaLvhQ4D3g2cLKky6seHxG7IuKiiLiot2lr+QHGGFMTUv9LocqtbmbhBnk58GBEPAYg6cPAjwGPStqZqeqdwP4ZjM0YY1aliUBchVkE64eAF2XruB6if6HIPcBTwBXA9dn9x8o6knpD6Y+uW/PKUh9ga15da1CDrXk5ZakPaN6aV3T8obrXs6YZ1VyFWeSs75B0K/B3wDHgC8AuYCtwi6Qr6Qf01057bMYYsxrSiV+m02QmRTER8Xbg7St2H6avsivTW1gcUtRdt+ZNqqbB1ryqhS5ga17VQhdo3pqXH5+q6fR9rQOpgRL2iqybCkZjjGka0a2cdW2opyFF3XVrXpma7p/L1jyYvNAFbM2rWugCzVvzckU96n2tBdWbs5Z0GvD7wA8AAfxCRPzforatDtbGGDNN+sq61i+AdwKfiIifkbSRfpFgIe0O1jGsqLvu9ihT02C3R13LmoLdHlULXaB5t0fR+9pEJWFdylrSKcBPAK8HiIgjwMiLRrY7WBtjzBTpSWtxg+yQtCd5vCsidiWPnws8BrxX0guBO4GrI+IpCmh1sF5eWhpS1HZ7DLDbo9llTcFuj6reaWje7ZG/r01fHGCh+nrWByLiolWeXwR+BHhTZml+J/0F7H69qPGsFnIyxpjWUXO5+V5gb0TckT2+lX7wLqTVynrp6OFVFXUX3R6Tqmmw26Oqdzrto+tujzI1DM27PYo+Q0cOF//HMAl15awj4hFJ35D0vIi4j36dyVdGtW91sDbGmGnSQFHMm4CbMyfIA8AbRjV0sDbGmIqINf3AWEpE3AWsltc+TquDdSwvDaU/um7NK0t9gK15da1BnfbRdWteWeoDpmfNS1MfR0b8cDouec56FrQ6WBtjzDRxuXkN2JpXrdAFbM2ra1nTtI+uW/PK1DQ0b83LFXWqptP/GGrBytoYY+afTq1nXSfqLQwpalvzBtia1+yypmBrXtVCl7SPpqx5uaJO57fQwJXIHayNMWbO6XXt4gN10VtYHFLUXXd7lKlpsNujrmVNwW6PqoUuaR9NuT3yOaZqesOmwXtcC85ZG2PM/CO0lrVBaqXVwVq9hSFF3XW3R5maHtWX3R5r906D3R5VvdNpH025PXJFnarpJhZ1GvX7Q9O0OlgbY8w0EbAwm1jd8mAtDSnqrrs9ytR02pfdHpN5p8Fuj6reaWje7ZEr6rSfwzVXMKLml2AdRbuDtTHGTBEBG+q9rFdlHKyNMaYiToNMwMr0R9eteWWpD7A1r641qFfOMaeL1ryy1Ac0b83L+0pTH4eeHHlJw/GQnAYxxph5R9gNMhaShhR11615ZWoabM2b1rKm0B1rXpmahuatebmiTtV0I1eKcRrEGGPmGwk2NLDeSBVaHazV05Ci7ro1r0xNg615dS1rCrbmVS10geatebmiTvuseyEnp0GMMaYl1JkGkfQPwEFgCTgWESMv8dX6YL1SUXfd7VGmpsFuj7qWNQW7PeoqdEnnOK7bI+83PX7LtsHfRR0INaGs/2VEHChr1PpgbYwxU8Or7o1HRAwp6q67PcrUNNjtYbfH9C9iC827PfLjUzW9OfnvqQ76OevKzXdI2pM83hURu1a0CeBTkgK4seD547Q6WBtjzDRZY7n5gdVy0BkviYiHJZ0J3C7pqxHx2aKGrQ7Wy0tLQ4q6626PMjUNdnvUtawp2O1R1TsNzbs9ckWdqunlpSYumFtfdxHxcHa/X9JHgB8FCoP1TAyDkk6TdKukr0q6V9KLJW2XdLuk+7P702cxNmOMGUVu3atyK+1LOlnStnwb+Cng7lHtZ+PuhncCn4iI5wMvBO4FrgV2R8T5wO7ssTHGzBH9K8VUuVXgLOBzkr4IfB74s4j4xKjGU0+DSDoF+Ang9QARcQQ4IulS4KVZs5uAzwBvW62vWDo2lP7oujWvLPUBtubVtQY12Jo3T9a8PP2Rpj7Sz1sd1FkUExEP0BerlZhFzvq5wGPAeyW9ELgTuBo4KyL2AUTEvizhPoSkq4CrAHpbnCkxxkyPfrl5d6x7i8CPAG+KiDskvZM1pDwya8sugMVTz46Virrr1rwyNQ225tW1rCnYmpdTpqaheWterqhTNX3w2wPDQV3MqNp8JjnrvcDeiLgje3wr/eD9qKSdANn9/hmMzRhjVqWHKt3qZurKOiIekfQNSc+LiPuAi4GvZLcrgOuz+4+V9aVeb0hRd92aV6amwda8nEkLXcDWvKqFLtC8NS//vKVq+ugzTxeOe1zE7JT1rHzWbwJulrQReAB4A32Vf4ukK4GHgNfOaGzGGDOSGVWbzyZYR8RdQFFlz8Vr6UcLi0OKuutujzI1nfZht8dkhS5gt0fVQpe0r6bcHrmiTtX0xpNOHhr/RGjOc9aS/k1WrPK4pCckHZT0RPmRxhizflC9Pus1UVVZ/ybwryPi3tpHMAG93sKQou6626NMTYPdHnUtawp2e1T1TkPzbo9cUadqeutpxZ/XSZj3NMij8xaojTFmFswoVlcO1nskfQj4KHBcFkTEhxsZlTHGzCFtuKzXKcDT9BcayQlgpsFaPa2a/uiiNa8s9QG25tW1BjXYmle10AWat+bl6Y809TEqbTkJc23di4g3ND0QY4xpA7Na/a5SsJZ0DvA7wEvoK+rPAVdHxN4Gx1ZhXMOKoOvWvDI1vbLfor5szaun0AW6Y80rU9Npv01Z83JF3YSazpFmd1mvql8S7wVuA54NnA38abbPGGM6hVTtVjdVc9ZnREQanN8n6Zr6hzM+tubVU+jSH6uteVCupsHWvKqFLmkfTVnzyv4TrgMxuzRI1fMekHS5pIXsdjnwrSYHZowx84ikSre6qaqsfwF4F/A/6Oes/zrbN1MihhV1190eZWo67ctuj8kKXcBuj6qFLjA9t0ejS6RqzotiIuIh4JKGx2KMMXONgBlde2D1YC3prRHxm5J+h76iPoGIeHNjI6tALMeQou6626NMTYPdHnZ7TP+K49C826NImT/zeP3Z2jpTHJIWgD3ANyPi1au1LVPWeYn5njoGZowxbaZfwVhrl1fTj7OnlDVcNVhHxJ9mm09HxB+nz0ma+XrTEbGqorbbw26PnCaWNQW7Pap6p/tjbdbtkffVhJpOqStWZ/Ur/wr4DeCXy9pXdYNcV3GfMcasY0RP1W4V+G3grcByWUMoz1m/EngVcLakG5KnTgGKfyo3xpj1ytoKXnZISlPIu7ILfiPp1cD+iLhT0kurdFaWs36Yfr76EuDOZP9B4JcqD7khYunYUPqj69a8stQH2JpX1xrUYGte1UIXaN6aV5T+2HDyqUP7JkERaHmpvGGfAxFRdEUs6C/dcYmkVwGbgVMk/WFEXD6qs7Kc9ReBL0q6OSKspI0xnUdRKWuxKhFxHVkqOVPWv7JaoIbyNMgtEfFvgS9ISq176p8vfmiyIdeHrXnVCl3A1ry6ljUFW/OqFrrA9Kx5qZretGXT0POTEVBDsB6HsjTI1dn9qv4/Y4zpDBHlbdbUXXwG+ExZu7I0yL5s8wBwKCKWJf0T4PnAn084xonpX938REXddWtemZoGW/NyJi10AVvzqha6QPPWvFxRp2p6w4j/RMcmZqesq1r3PgtslnQ2sBt4A/C+pgZljDHzimK50q1uqn7tKCKelnQl8DtZCfoXah/NGllY7K2qqLvo9ihT02C3R13LmoLdHlULXaB5t0euqFM1vbBYVY9WJWB5Nl6LysFa0ouBnwOuXOOxxhizPgjm9gfGnGvo20w+EhH3SHou8OnmhlWN3kJv6Ju5626PMjUMdnvUtawp2O1R1TsNzbs9ckVdv5pOCVie42AdEX8J/KWkbZK2RsQDwExX3DPGmFnQRD66ClUvmPuDwPuB7f2Hegz4dxFxT5ODWwt2e1TzToPdHnUtawp2e1T1TkPzbo8iRZ1+hmpjnoM1cCPwyxHxaThecfO/gB9raFzGGDN/RED1cvNaqRqsT84DNfRN3JJOXu0AY4xZj8x1GgR4QNKvA3+QPb4ceLCZIVVneWl5KP3RdWteWeoDbM2raw1qsDWvrkIXqM+al36Gjhw6OvT8ZMx/UcwvAGcAH85uO+gXxhhjTLeI5Wq3milbyGkz8B+A7wO+DLwlIur+qhqb5aUYUtRdt+aVqem0D1vzJit0SfvoujWvTE2P6qtOa17+GUrV9KGDBwvbjs0My83L0iA3AUeBvwJeCbyAvufaGGM6h5jfnPUFEfGDAJLeDXy++SFVZ3l5aUhRd92aV6am0z5szZus0CXto+vWvDI1nfbVlDUvV9Spmn7m8ccKzzU+AUuzcYOU5ayP/7XXffEBSQuSviDp49nj7ZJul3R/dn96neczxpiJycvN5y1nDbxQ0hPZtoAt2eP84gOll09fhZWXYL8W2B0R10u6Nnv8ttU6kHpDirrrbo8yNZ32YbfHZIUuaR92ewyYldsjV9Spml46PDh/XcwqDbKqso6IhYg4Jbtti4jFZHvsQJ1cgv33k92X0s+Rk92/Ztz+jTGmGWJulXVT5Jdg35bsOyu/2EFE7JN0ZtGBkq4CrgLYcMoZQ4q6626PKgs52e1Rz7KmYLfHyn5W9jVNt0euqFM1vbBp8BtWbcyjsm6C9BLs4xwfEbsi4qKIuGjxpHqvXGyMMauSl5tXudXMLJR14SXYgUcl7cxU9U5gf1lH6RKpdntU806D3R51LWsKdntU9U5D826PXFGnanrzqWcUjmV8gjhWT6lJVsfyWWAT/Vh8a0S8fVT7qSvriLguIs6JiHOBy4C/yC7BfhtwRdbsCuBj0x6bMcasSlCnsj4MvCwiXghcCLxC0otGNZ6nq71cD9ySXTrsIeC1Mx6PMcacQBBETT7riAjgyezhhuw2ck3XmQbr9BLsEfEt4OK1HC+G0x9dt+aVpT7A1ry61qAGW/OqFrpA89a8PP2Rpj5OOnUSd3EBwVquFLND0p7k8a6I2JU2kLQA3El/SY/fjYg7RnU2T8raGGPmnDWtZ30gIi5atbeIJeBCSacBH5H0AxFxd1HbVgfr5eUYUtRdt+aVqWmwNa+uZU3B1rx5sublijpV0xtGfF7HJur7gfHEbuO7kj4DvAIoDNZT/4HRGGPaSxDLS5VuZUg6I1PUSNoCvBz46qj2rVfWKxV11615ZWp65RxzbM2rv9AlneN6t+aVqWlo3pqXK+r0nFH3NRhzN0g97ARuyvLWPeCWiPj4qMatDtbGGDNdYi0/MK7eU8SXgB+u2r7dwTpiSFF33e0xqZoGuz2qFrqA3R51FbrA5G6P/Lypmj5c92W9gtqse2ul3cHaGGOmyprcILXS6mCtnoYUddfdHmVqGuz2qGtZU7Dbo6p3Gpp3e+SKOlXTx47WHFgbcoNUodXB2hhjpouVtTHGzD/1ukHWRKuDda+nofRH1615ZakPsDXP1rwB07riODRvzcvTH2nqo4rfeS0EQdTkBlkrrQ7WxhgzVaysx0M9DSlqW/MG2JrX7BrUYGvePFnzckWdqum6lTURxNEj5e0aoNXB2hhjpkt9RTFrpd3BOoYVddeteWVqGmzNq2tZU7A17/iYS9R0/1zNWvNyFZ2q6WOHnqR2nAYxxpg5J6L+1EpFWh2sI2JIUXfd7VGmpsFuj7qWNU376rrbo0xNQ/Nuj3w7VdPp+evCbhBjjJl3IoglB+s1EzGsqLvu9ihT02C3R86k3mmw26Oqdxqad3vkijo9/5EnvzM0/kmICJaPFn8umqbVwdoYY6ZKYGU9FhpW1F13e5SpabDbo65lTcFuj6reaWje7ZGPoW41vRIHa2OMmXMigmWvZ22MMfOP3SBj0JNWTX900ZpXlvoAW/PqWoM67bfr1ryy1Ac0b80rSn9s3Hp64bzGpkY3iKTnAO8HvgdYBnZFxDtHtW91sDbGmGlSsxvkGPCWiPg7SduAOyXdHhFfKWrc6mCdXikmp+vWvDI1Dbbm1bWsadpH1615ZWo6PW/T1rxUTW/9nvOOb397qOV4LNekrCNiH7Av2z4o6V7gbGD9BWtjjJkqa7Pu7ZC0J3m8KyJ2FTWUdC79K53fMaqzdROsbc2rVugCtubVtawp2JpXtdAFmrfm5Yo6VdNbT9tW2HZs1pazPhARF5U1krQV+BPgmoh4YlS7dROsjTGmaYJ63SCSNtAP1DdHxIdXa9vqYB0xrKi77vaYVE2D3R5VC13Abo+qhS7QvNsjV9SpmtaI/77GJoLlI/X8wChJwLuBeyPiHWXtWx2sjTFmqgQs16esXwL8PPBlSXdl+341Iv53UeOWB+sYUtRdd3uUqWmw26OuZU37Y7XbY+U5Z+X2yBW1TviPaXD+Ogjq81lHxOeAytK/5cHaGGOmSEC43HwcNKSou+72KFPTYLdHXcuagt0eVb3T/TbNuj10/D+mZH7frctdnRMzKzcv/tWhQSQ9R9KnJd0r6R5JV2f7t0u6XdL92X3NdaLGGDMhmc+6yq1uph6sGZRYvgB4EfBGSRcA1wK7I+J8YHf22Bhj5oaIYOnIsUq3upl6GmSVEstLgZdmzW4CPgO8bbW+er3h9IeteQNszWt2DWqwNa9qoUu63ZQ1Lx9DmvpYOjKYVz3MLg0y05z1ihLLs7JATkTsk3TmiGOuAq4C2Lz9rOkM1BhjoJtXillZYqkRinQlWW39LoDt570gVirirlvzytQ02JpX17KmYGte1UIXaN6alyvqVE33Fgfvay0ERPJ3MU1mEqxHlFg+Kmlnpqp3AvtnMTZjjBlFJLUd02bqwXqVEsvbgCuA67P7j5X2xbAi7ro1r0xNg615dS1rCrbmVS10ScfQlDUvV9Spmk7/Y6iFOPG1mCazUNaFJZb0g/Qtkq4EHgJeO4OxGWPMSCJg6UhHimJKSiwvXlNfDCvqrrs9ytQ02O1R17KmYLdH1UIXaN7tkSvq2tV0SkS3ctbGGNNWlh2s107EsKLuutujTE2D3R51LWsKdnvU5Z2Gyd0eRYo6HUstdNG6Z4wxbSM48ctymjhYG2NMVSK68wNjnfR6Gkp/dN2aV5b6AFvzprUGNXTHmleW+oDpWfPSsRz61sOFYxmX6FpRjDHGtBIH6/FYkIYUddeteWVqGGzNq2sN6lF9ddGaV6amoXlrXj6WVE0fPlj/etazqmCcxRKpxhjTTrIKxiq3MiS9R9J+SXdXOXW7lXVPQ4q669a8MjUNtubVtaxp2peteQNmZc3LFXX9anpAUKvP+n3Au4D3V2nc6mBtjDFTJYLlmtwgEfHZbJnoSrQ+WK9UlF13e5SpabDbw26P6V9xHJp3exQp6k3bth/fruM65xFrUtY7JO1JHu/Klngei9YHa2OMmSZruFLMgYi4qK7ztj5Yr1TUXXd7lKnptA+7PZpV09Adt0eZmk7H0LTbI1XTm08fXE3qyaGWYxDhtUGMMWbusc96PKRhRd11t0eZmk77sOpss6gAAAhwSURBVNtjMu802O1R1TsNzbs9ckWdqunNp5xR2HZcgvoWcpL0AfoXCd8haS/w9oh496j2rQ7WxhgzVSJYOlJPsI6I162lvYO1McZUJOLE/46nSauD9UJPQ+mPrlvzylIfaR+25g0Yp9AFbM2rWugCzVvz8vRHmvpY3LJ16JhJWXKwNsaY+SaAGf2+2O5gvdjTkKLuujWvykJOtuad2M/KvmzNG1DXsqbQvDUvV9RNqOkUK2tjjJlzlgOO+Eoxa2ehp1UVdReteWVqGmzNq2tZU7A1r2qhy8o5Fo1lUmtekaJOX4u6cBrEGGPmnCCcBhkHMayou+72KFPTYLfHtJY1he64PSZV01Cf2yOd//KxI0PPT4J/YDTGmJbgYD0hdntU806D3R51LWsKdntU9U6nY2nK7ZG/BqmaTl/3OoiwG8QYY+aewG6QsVjoaUhRd93tMamaTufYdbdHmZoGuz2qeqehebdHrqhTNZ2OtQ6cszbGmJbgNIgxxsw5/Zz1bM7d6mC9aaE3lP7oujWvLPUBtubZmrc+rXn5656O9VjNaRCwsjbGmLkngHpWs147rQ7WG449M6Sou27NK1PTYGteXcua9s9la146DpidNS8faxNq+vgYCLtBjDFm3um7QRys18zh/Y8MKWpb8wbYmtfssqZga17VQhdo3ppXpKjT/whqYYY/MBb/tc4QSa+QdJ+kr0m6dtbjMcaYnFxZV7mVsdZYN1fKWtIC8LvATwJ7gb+VdFtEfKWo/SP7nxpS1F13e5SpabDbo65lTcFuj6qFLjA9t0c6/l5vYej5SalDWa811sGcBWvgR4GvRcQDAJI+CFwKjJyAMcZMi2VqKzdfc6ybt2B9NvCN5PFe4J+lDSRdBVyVPTx8I1+/G+DGG77e33PDhxof5BTYARyY9SBqZj3OCeZwXsUad83UMq+jyfaTI1tNjeeVN1mdAxz55I18fUfF5psl7Uke74qIXdl2aaxbybwF6yLf2QlfY9lkdwFI2hMRF01jYNNkPc5rPc4JPK82sSJwjkVEvKKOsVAh1q1k3n5g3As8J3l8DvDwiLbGGNNW1hzr5i1Y/y1wvqTzJG0ELgNum/GYjDGmbtYc6+YqDRIRxyT9IvBJYAF4T0Tcs8ohu1Z5rs2sx3mtxzmB59Um5mZOY8Q6FDOqxjHGGFOdeUuDGGOMKcDB2hhjWkBrg/V6KEuX9BxJn5Z0r6R7JF2d7d8u6XZJ92f3p5f1NY9IWpD0BUkfzx63el6STpN0q6SvZu/Zi9s+JwBJv5T9/d0t6QOSNrdxXpLeI2m/pLuTfSPnIem6LH7cJ+mnZzPq6rQyWCelmq8ELgBeJ+mC2Y5qLI4Bb4mIFwAvAt6YzeNaYHdEnA/szh63kauBe5PHbZ/XO4FPRMTzgRfSn1ur5yTpbODNwEUR8QP0f+y6jHbO633ASh904Tyyz9llwPdnx/xeFlfml4ho3Q14MfDJ5PF1wHWzHlcN8/oY/bUC7gN2Zvt2AvfNemxjzOUc+h+OlwEfz/a1dl7AKcCDZD/KJ/tbO6dszHkl3Xb67rCPAz/V1nkB5wJ3l70/K2MGfVfGi2c9/tVurVTWFJdqnj2jsdSCpHOBHwbuAM6KiH0A2f2ZsxvZ2Pw28FZOvLBGm+f1XOAx4L1Zauf3JZ1Mu+dERHwT+C3gIWAf8HhEfIqWzyth1DxaF0PaGqzXXKo5z0jaCvwJcE1EPDHr8UyKpFcD+yPizlmPpUYWgR8B/mdE/DDwFO1IDaxKlsO9FDgPeDZwsqTLZzuqqdC6GNLWYL1uytIlbaAfqG+OiA9nux+VtDN7fiewf1bjG5OXAJdI+gfgg8DLJP0h7Z7XXmBvRNyRPb6VfvBu85wAXg48GBGPRcRR4MPAj9H+eeWMmkfrYkhbg/W6KEuXJODdwL0R8Y7kqduAK7LtK+jnsltDRFwXEedExLn035u/iIjLafG8IuIR4BuS8pXbLqa/nGVr55TxEPAiSSdlf48X0//htO3zyhk1j9uAyyRtknQecD7w+RmMrzqzTppP8EPCq4D/B/w98GuzHs+Yc/hx+v96fQm4K7u9CngW/R/n7s/ut896rBPM8aUMfmBs9byAC4E92fv1UeD0ts8pm9d/Ab4K3A38AbCpjfMCPkA/736UvnK+crV5AL+WxY/7gFfOevxlN5ebG2NMC2hrGsQYYzqFg7UxxrQAB2tjjGkBDtbGGNMCHKyNMaYFOFibuUHSkqS7stXf/ljSSWs8/tmSbs22L5T0quS5S9q6OqMx4CvFmDlC0pMRsTXbvhm4M04sFlpLX6+nv5LcL9Y4RGNmhpW1mVf+Cvi+bD3ij0r6kqS/kfRDAJL+RabC78oWVtom6dxMlW8E/ivws9nzPyvp9ZLelR37jyTtzvrcLel7s/3vk3SDpL+W9ICkn5nZ7I1ZgYO1mTskLdJfq/zL9KvrvhARPwT8KvD+rNmvAG+MiAuBfw4cyo+PiCPAfwI+FBEXRsSHVpziXcD7sz5vBm5InttJv7L01cD1dc/NmHFxsDbzxBZJd9Ev6X6I/ropP06/BJqI+AvgWZJOBf4P8A5JbwZOi4hjazjPi4E/yrb/IDtHzkcjYjkivgKcNdFsjKmRxVkPwJiEQ5lSPk62uNBKIiKul/Rn9NdS+RtJLweeGfO86Q83h9PTj9mfMbVjZW3mnc8CPwcg6aXAgYh4QtI/jogvR8R/o6/En7/iuIPAthF9/jX91QDJ+v5c7aM2pmYcrM2885+BiyR9iX4OOV/u8prsx8Qv0s9X//mK4z4NXJD/wLjiuTcDb8j6/Hn614o0Zq6xdc8YY1qAlbUxxrQAB2tjjGkBDtbGGNMCHKyNMaYFOFgbY0wLcLA2xpgW4GBtjDEt4P8DR0NjzeqvkVIAAAAASUVORK5CYII=\n",
+ "text/plain": [
+ ""
+ ]
+ },
+ "metadata": {
+ "needs_background": "light"
+ },
+ "output_type": "display_data"
+ }
+ ],
+ "source": [
+ "# Positional encoding euclidean distance\n",
+ "eu = np.zeros((MAX_SEQUENCE_LENGTH, MAX_SEQUENCE_LENGTH))\n",
+ "print(eu.shape)\n",
+ "for a in range(MAX_SEQUENCE_LENGTH):\n",
+ " for b in range(a + 1, MAX_SEQUENCE_LENGTH):\n",
+ " eu[a, b] = tf.norm(tf.math.subtract(pos_encoding[0, a], pos_encoding[0, b]))\n",
+ " eu[b, a] = eu[a, b]\n",
+ " \n",
+ "plt.pcolormesh(eu, cmap='RdBu')\n",
+ "plt.xlabel('Position')\n",
+ "plt.xlim((0, MAX_SEQUENCE_LENGTH))\n",
+ "plt.ylabel('Position')\n",
+ "plt.colorbar()\n",
+ "plt.show()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Nice work! You can use these visualizations as checks for any positional encodings you create. "
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "## 2 - Semantic embedding\n",
+ "\n",
+ "You have gained insight into the relationship positional encoding vectors have with other vectors at different positions by creating correlation and distance matrices. Similarly, you can gain a stronger intuition as to how positional encodings affect word embeddings by visualizing the sum of these vectors. \n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "### 2.1 - Load pretrained embedding\n",
+ "\n",
+ "To combine a pretrained word embedding with the positional encodings you created, start by loading one of the pretrained embeddings from the [glove](https://nlp.stanford.edu/projects/glove/) project. You will use the embedding with 100 features. "
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 9,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Found 400000 word vectors.\n",
+ "d_model: (100,)\n"
+ ]
+ }
+ ],
+ "source": [
+ "embeddings_index = {}\n",
+ "GLOVE_DIR = \"glove\"\n",
+ "f = open(os.path.join(GLOVE_DIR, 'glove.6B.100d.txt'))\n",
+ "for line in f:\n",
+ " values = line.split()\n",
+ " word = values[0]\n",
+ " coefs = np.asarray(values[1:], dtype='float32')\n",
+ " embeddings_index[word] = coefs\n",
+ "f.close()\n",
+ "\n",
+ "print('Found %s word vectors.' % len(embeddings_index))\n",
+ "print('d_model:', embeddings_index['hi'].shape)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "**Note:** This embedding is composed of 400,000 words and each word embedding has 100 features."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Consider the following text that only contains two sentences. Wait a minute - these sentences have no meaning! Instead, the sentences are engineered such that:\n",
+ "* Each sentence is composed of sets of words, which have some semantic similarities among each groups.\n",
+ "* In the first sentence similar terms are consecutive, while in the second sentence, the order is random. "
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 10,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "texts = ['king queen man woman dog wolf football basketball red green yellow',\n",
+ " 'man queen yellow basketball green dog woman football king red wolf']"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "First, run the following code cell to apply the tokenization to the raw text. Don't worry too much about what this step does - it will be explained in detail in later ungraded labs. A quick summary (not crucial to understanding the lab):\n",
+ "\n",
+ "* If you feed an array of plain text of different sentence lengths, and it will produce a matrix with one row for each sentence, each of them represented by an array of size `MAX_SEQUENCE_LENGTH`.\n",
+ "* Each value in this array represents each word of the sentence using its corresponding index in a dictionary(`word_index`). \n",
+ "* The sequences shorter than the `MAX_SEQUENCE_LENGTH` are padded with zeros to create uniform length. \n",
+ "\n",
+ "Again, this is explained in detail in later ungraded labs, so don't worry about this too much right now!"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 11,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Found 11 unique tokens.\n",
+ "(2, 100)\n",
+ "[[ 1 2 3 4 5 6 7 8 9 10 11 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
+ " 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
+ " 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
+ " 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
+ " 0 0 0 0]\n",
+ " [ 3 2 11 8 10 5 4 7 1 9 6 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
+ " 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
+ " 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
+ " 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
+ " 0 0 0 0]]\n"
+ ]
+ }
+ ],
+ "source": [
+ "tokenizer = Tokenizer(num_words=MAX_NB_WORDS)\n",
+ "tokenizer.fit_on_texts(texts)\n",
+ "sequences = tokenizer.texts_to_sequences(texts)\n",
+ "\n",
+ "word_index = tokenizer.word_index\n",
+ "print('Found %s unique tokens.' % len(word_index))\n",
+ "\n",
+ "data = pad_sequences(sequences, padding='post', maxlen=MAX_SEQUENCE_LENGTH)\n",
+ "\n",
+ "print(data.shape)\n",
+ "\n",
+ "print(data)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "To simplify your model, you will only need to obtain the embeddings for the different words that appear in the text you are examining. In this case, you will filter out only the 11 words appearing in our sentences. The first vector will be an array of zeros and will codify all the unknown words."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 12,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "(12, 100)\n"
+ ]
+ }
+ ],
+ "source": [
+ "embedding_matrix = np.zeros((len(word_index) + 1, EMBEDDING_DIM))\n",
+ "for word, i in word_index.items():\n",
+ " embedding_vector = embeddings_index.get(word)\n",
+ " if embedding_vector is not None:\n",
+ " # words not found in embedding index will be all-zeros.\n",
+ " embedding_matrix[i] = embedding_vector\n",
+ "print(embedding_matrix.shape)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Create an embedding layer using the weights extracted from the pretrained glove embeddings."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 13,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "embedding_layer = Embedding(len(word_index) + 1,\n",
+ " EMBEDDING_DIM,\n",
+ " embeddings_initializer=tf.keras.initializers.Constant(embedding_matrix),\n",
+ " trainable=False)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Transform the input tokenized data to the embedding using the previous layer. Check the shape of the embedding to make sure the last dimension of this matrix contains the embeddings of the words in the sentence. "
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 14,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "(2, 100, 100)\n"
+ ]
+ }
+ ],
+ "source": [
+ "embedding = embedding_layer(data)\n",
+ "print(embedding.shape)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "### 2.2 - Visualization on a Cartesian plane\n",
+ "\n",
+ "Now, you will create a function that allows you to visualize the encoding of our words in a Cartesian plane. You will use PCA to reduce the 100 features of the glove embedding to only 2 components. "
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 15,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "from sklearn.decomposition import PCA\n",
+ "\n",
+ "def plot_words(embedding, sequences, sentence):\n",
+ " pca = PCA(n_components=2)\n",
+ " X_pca_train = pca.fit_transform(embedding[sentence,0:len(sequences[sentence]),:])\n",
+ "\n",
+ "\n",
+ " fig, ax = plt.subplots(figsize=(12, 6)) \n",
+ " plt.rcParams['font.size'] = '12'\n",
+ " ax.scatter(X_pca_train[:, 0], X_pca_train[:, 1])\n",
+ " words = list(word_index.keys())\n",
+ " for i, index in enumerate(sequences[sentence]):\n",
+ " ax.annotate(words[index-1], (X_pca_train[i, 0], X_pca_train[i, 1]))\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Nice! Now you can plot the embedding of each of the sentences. Each plot should disply the embeddings of the different words. \n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 16,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "image/png": "iVBORw0KGgoAAAANSUhEUgAAAuEAAAFlCAYAAABFiTbBAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO3de1RXVf7/8dcWiFARFDABCUSLsu9UNJiWF6oxUTHHNK+lqeXUmjGzzEbFb2nTZGU287NmvpV+Ryt1tItpakpXyfJSKHQXR03ID94JFAO57d8f6OcrpXnhw/lweT7WYq3P2edz9n6fs4r18rDPPsZaKwAAAADOaeTtAgAAAICGhhAOAAAAOIwQDgAAADiMEA4AAAA4jBAOAAAAOIwQDgAAADjM1xuDhoaG2piYGG8MDQAAgAZk8+bNB621Yd6u4+e8EsJjYmKUnp7ujaEBAADQgBhjsr1dw6kwHQUAAABwGCEcAAAAcBghHAAAAHAYIRwAAABwGCEcAAAAcBghHAAAAHAYIRxAnZaRkaFrrrlGgYGBGjx4sIYMGaKpU6dq/vz56tKlS5XvGmO0fft2SdKxY8f00EMP6eKLL9ZFF12ke++9V0VFRe7vrly5UldffbWCg4N1/fXX68svv3Tvi4mJ0TPPPKMrr7xSQUFBGjx4sIqLi505YQBAvUAIB1BnlZSUqF+/fho+fLjy8vI0cOBAvfnmm2d17J///Gdt27ZNmZmZ2r59u1wulx577DFJ0pYtWzR69Gi9+OKLOnTokO655x717dtXx44dcx//2muvac2aNfr+++/15Zdfav78+TVxigCAeooQDqDO2rhxo0pLSzV+/Hj5+fnptttuU4cOHc54nLVWc+bM0d/+9je1aNFCgYGBmjJlihYvXixJmjNnju655x517NhRPj4+uvPOO+Xv76+NGze6+xg3bpwiIiLUokUL3XLLLcrMzKyx8wQA1D9eeWMmAFTHsgyXZqZmadv6NSpq1EzLM3PVLz5SkhQdHX3G4w8cOKCffvpJv/3tb91t1lqVl5dLkrKzs/Xyyy/rueeec+8vKSlRbm6ue7tVq1buz40bN66yDwCAMyGEA6hTlmW4NHnpVyoqLZdP0xYqLjigSW9WztfuFx+pnJwctW3bVk2aNNFPP/3kPm7v3r3uz6GhoQoICNA333yjyMjIX4wRFRWllJQUpaSk1PwJAQAaJKajAKhTZqZmqai08o61f+RlkvHR/o1v6el3vtXSpUv12WefSZKuuuoqffPNN8rMzFRxcbGmTZvm7qNRo0YaM2aMHnjgAe3fv1+S5HK5lJqaKkkaM2aMXnjhBW3atEnWWh09elSrVq3SkSNHnD1ZAEC9RQgHUKfk5v/fCibGx09ht05R4VcfaONjv9eSJUvUv39/SdKll16qRx55RN27d9cll1zyi5VSnnrqKbVr106dOnVSs2bN1L17d2VlZUmSEhISNGfOHI0dO1bNmzdXu3btePASAOBRxlrr+KAJCQk2PT3d8XEB1D0xMTGaO3euunfvLknq/OSHcuUXKXfuH9Wix7268OIrJUmRwQH6dNJNGjlypFq3bq3HH3/cm2UDAGoJY8xma22Ct+v4Oe6EA6hTJibFKcDPRxF3/9MdwAP8fDQxKc7LlQEAcPZ4MBNAnXJiFZSZqVnKzS9SRHCAJibFudsBAKgLuBMOoM7YunWr2rRpo+KsdXK9MFpzuvvr00k3KXP5HA0aNEgjRozQm2++qbfeeksnT3nbsmWL4uPjFRgYqIEDB2rw4MGaOnWqF88EANDQEcIB1AlbtmxRjx499Nxzz2nIkCG/2P/2229ryJAhys/PV9++fTV27FhJlet733rrrRo5cqTy8vI0dOhQvfXWW06XDwBAFYRwALXeunXr1LdvX7388svq06fPKb/TpUsX9e7dWz4+Pho+fLi++OILSZVv1SwrK9O4cePk5+en/v3769prr3WyfAAAfqHaIdwYc6Ex5jNjzBfGmG+MMdM9URgAnPDCCy/o+uuv14033nja7/z8DZbFxcUqKytTbm6uIiMjZYxx74+KiqrRegEAOBNP3Ak/Jukma+1Vkq6W1NMY08kD/QJooJZluNT5yQ/VZtIq7S0o1qiH/6qcnBw98MAD59xXeHi4XC6XTl6O9YcffvBkuQAAnLNqh3BbqfD4pt/xH+cXHwdQL5x4Lb0rv0hWUlmF1aItB3Tf0//Sxx9/rEmTJp1Tf9ddd518fHz0/PPPq6ysTMuXL3e/VRMAAG/xyBKFxhgfSZsltZP0D2vtplN85w+S/iBJF198sSeGBVAPnfxa+hOOlVfon+v36r333tONN94oPz+/s+7vggsu0NKlS3X33Xdr8uTJ6tWrl/r06SN/f39Plw4AwFnz6BszjTHBkt6SdJ+19uvTfY83ZgI4nTaTVp3yT2lG0vdPJntkjI4dO+ree+/VqFGjPNIfAKD2ahBvzLTW5ktaK6mnJ/sF0HBEBAecU/vZSEtL0969e1VWVqaXX35ZX375pXr25NcUAMB7PLE6StjxO+AyxgRI6i5pa3X7BdAwnXgt/cmq+1r6rKwsXXXVVQoKCtKsWbP0xhtvKDw8vLqlAgBw3qo9HcUYc6WklyX5qDLUv2atfezXjmE6CoBfsyzDxWvpAQAeUVuno1T7wUxr7ZeS4j1QCwBIkvrFRxK6AQD1Gm/MBAAAABxGCAcAAAAcRggHAAAAHEYIBwAAABxGCAcAAAAcRggHAAAAHEYIBwAAABxGCAcAAAAcRggHAAAAHEYIBwAAABxGCAcAAAAcRggHAAAAHEYIBwAAABxGCAcAAAAcRggHAAAAHEYIBwAAABxGCAcAAAAcRggHAAAAHEYIBwAAABxGCAcAAAAcRggHAAAAHEYIBwAAABxGCAcAAAAcRggHAAAAHEYIBwAAABxGCAcAAAAcRggHAAAAHEYIBwAAABxGCAcAAAAcRggHAAAAHEYIBwAAABxGCAcAAAAcRggHAAAAHEYIBwAAABxGCAcAAAAcRggHAAAAHEYIBwAAABxGCAcAAAAcRggHAAAAHEYIBwAAABxGCAcAAAAcRggHvGTevHm65ZZb3Nvt2rXToEGD3NtRUVHKzMzU+vXr1aFDBwUFBalDhw5av369+zs33HCDpk6dquuvv15NmzbVLbfcokOHDun2229Xs2bN1KFDB+3atcv9/fvvv19RUVFq1qyZfvvb32rdunXufdOmTdOgQYM0YsQIBQYG6oorrlB6enrNXgQAABqoaodwY0yUMeYjY8x3xphvjDH3e6IwoL5LTEzUunXrVFFRoT179qi0tFSffvqpJGnnzp0qLCzUxRdfrOTkZI0bN06HDh3Sgw8+qOTkZB06dMjdz+LFi/Xqq6/K5XJpx44duu666zRq1Cjl5eXp8ssv1/Tp093f7dChgzIzM5WXl6dhw4Zp4MCBKi4udu9/++23NWTIEOXn56tv374aO3ascxcEAIAGxBN3wsskTbDWXi6pk6Q/GWPae6BfoF6LjY1VYGCgMjMzlZaWpqSkJEVGRmrr1q1KS0tT165dtWrVKl1yySUaPny4fH19NXToUF122WVasWKFu59Ro0apbdu2CgoKUq9evdS2bVt1795dvr6+GjhwoDIyMtzfveOOOxQSEiJfX19NmDBBx44dU1ZWlnt/ly5d1Lt3b/n4+Gj48OH64osvHL0mAAA0FL7V7cBau0fSnuOfjxhjvpMUKenb6vYN1DfLMlyamZql3PwiRQQHKPY3HbR27Vpt375diYmJCg4OVlpamjZs2KDExETl5uYqOjq6Sh/R0dFyuVzu7Ysuusj9OSAg4BfbhYWF7u1Zs2Zp7ty5ys3NlTFGhw8f1sGDB937W7Vq5f7cuHFjFRcXq6ysTL6+1f5VAQAATuLROeHGmBhJ8ZI2nWLfH4wx6caY9AMHDnhyWKBOWJbh0uSlX8mVXyQryZVfpG0mSouXr9G6deuUmJioxMREpaWlKS0tTYmJiYqIiFB2dnaVfnJychQZGXnO469bt05PPfWUXnvtNf3444/Kz89XUFCQrLUeOkMAAHC2PBbCjTFNJb0paby19vDP91trX7LWJlhrE8LCwjw1LFBnzEzNUlFpeZW2RpFXaPPGT1RUVKTWrVura9euWrNmjQ4dOqT4+Hj17t1b27Zt06JFi1RWVqYlS5bo22+/VZ8+fc55/CNHjsjX11dhYWEqKyvTY489psOHf/G/KgAAcIBHQrgxxk+VAXyhtXapJ/oE6pvc/KJftPm1iJTxu1Bdu3aVJDVr1kyxsbHq3LmzfHx8FBISopUrV2rWrFkKCQnR008/rZUrVyo0NPScx09KSlKvXr106aWXKjo6WhdeeKGioqKqfV4AAODcmer+KdoYYyS9LCnPWjv+bI5JSEiwLH2Ghqbzkx/KdYogHhkcoE8n3eSFigAAqP+MMZuttQneruPnPHEnvLOk4ZJuMsZkHv/p7YF+gXplYlKcAvx8qrQF+PloYlKclyoCAADe4onVUT6RZDxQC1Cv9YuvfJjy5NVRJibFudsBAEDDwbpjgIP6xUcSugEAAK+tBwAAAJxGCAcAAAAcRggHAAAAHEYIBwAAABxGCAcAAAAcRggHAAAAHEYIBwAAABxGCAcAAAAcRggHAAAAHEYIBwAAABxGCAcAAAAcRggHAAAAHEYIBwAAABxGCAcAAAAcRggHAAAAHEYIBwAAABxGCAcAAAAcRggHAAAAHEYIBwAAABxGCAcAAAAcRggHAAAAHEYIB+qwmJgYzZw5U1deeaWaNGmiu+66S/v27VOvXr0UGBio7t2768cff5QkDRw4UK1atVJQUJC6deumb775xt3PyJEj9ac//UnJyckKDAxUx44dtWPHDm+dFgAA9R4hHKjj3nzzTb333nvatm2bVqxYoV69eumJJ57QwYMHVVFRodmzZ0uSevXqpf/85z/av3+/rrnmGt1+++1V+vn3v/+tRx99VD/++KPatWunlJQUb5wOAAANgq+3CwBQPffdd58uuugiSVLXrl3VsmVLxcfHS5JuvfVWffDBB5Kk0aNHu4+ZNm2amjdvroKCAgUFBUmS+vfvr2uvvVaSdPvtt+vBBx908jQAAGhQCOFAHbMsw6WZqVnKzS/S3oJibT/i494XEBDgDuQntgsLC1VeXq6UlBS9/vrrOnDggBo1qvwj2MGDB90hvFWrVu7jGjdurMLCQofOCACAhofpKEAdsizDpclLv5Irv0hWUlmF1f+u+17LMly/etyiRYu0fPlyvf/++yooKNCuXbskSdbami8aAAD8AiEcqENmpmapqLS8Stux8grNTM361eOOHDkif39/hYSE6KefftKUKVNqskwAAHAGhHCgDsnNLzqn9hNGjBih6OhoRUZGqn379urUqVNNlAcAAM6S8cafoxMSEmx6errj4wJ1XecnP5TrFIE7MjhAn066yQsVAQBQuxljNltrE7xdx89xJxyoQyYmxSnAz6dKW4CfjyYmxXmpIgAAcD5YHQWoQ/rFR0qSe3WUiOAATUyKc7cDAIC6gRAO1DH94iMJ3QAA1HFMRwEAAAAcRggHAAAAHEYIBwAAABxGCAcAAAAcRggHAAAAHEYIBwAAABxGCAcAAAAcRggHAAAAHEYIBwAAABzmkRBujPmXMWa/MeZrT/QHAAAA1GeeuhM+X1JPD/UFAAAA1GseCeHW2o8l5XmiLwAAAKC+c2xOuDHmD8aYdGNM+oEDB5waFgAAAKh1HAvh1tqXrLUJ1tqEsLAwp4YFAAAAah1WRwEAAAAcRggHAAAAHOapJQr/LWmDpDhjzG5jzF2e6BcAAACoj3w90Ym1dqgn+gEAAAAaAqajAAAAAA4jhAMAAAAOI4QDAAAADiOEAwAAAA4jhAMAAAAOI4QDAAAADiOEAwAAAA4jhAMAAAAOI4QDAAAADiOEw+tGjhypqVOnersMAAAAxxDCAQAAAIcRwgEAAACHEcLhuIyMDF1zzTUKDAzU4MGDVVxc7N43Z84ctWvXTi1atFDfvn2Vm5vr3vfuu+8qLi5OQUFB+uMf/6jExETNnTvXG6cAAABQLYRwOKqkpET9+vXT8OHDlZeXp4EDB+rNN9+UJH344YeaPHmyXnvtNe3Zs0fR0dEaMmSIJOngwYO67bbbNGPGDB06dEhxcXFav369N08FAADgvBHC4aiNGzeqtLRU48ePl5+fn2677TZ16NBBkrRw4UKNHj1a11xzjfz9/TVjxgxt2LBBu3bt0jvvvKMrrrhC/fv3l6+vr8aNG6dWrVp5+WwAAADODyEcjsrNzVVkZKSMMe626Oho974TnyWpadOmCgkJkcvlUm5urqKiotz7jDFq3bq1c4UDAAB4kK+3C0D9tyzDpZmpWcrNL1KTvD3K3fWDrLXuIJ6Tk6O2bdsqIiJC2dnZ7uOOHj2qQ4cOKTIyUuHh4dq9e7d7n7W2yjYAAEBdwp1w1KhlGS5NXvqVXPlFspKOBLXRkZIK3f3wYyorK9PSpUv12WefSZKGDRumefPmKTMzU8eOHdOUKVPUsWNHxcTEKDk5WV999ZWWLVumsrIy/eMf/9DevXu9e3IAAADniRCOGjUzNUtFpeXubePjp9B+U7Rk4QI1b95cS5YsUf/+/SVJv/vd7/SXv/xFAwYMUHh4uHbs2KHFixdLkkJDQ/X666/r4YcfVkhIiL799lslJCTI39/fK+cFAABQHcZa6/igCQkJNj093fFx4bw2k1bpVP+FGUnfP5l83v1WVFSodevWWrhwoW688cbz7gcAANRvxpjN1toEb9fxc9wJR42KCA44p/Zfk5qaqvz8fB07dkxPPPGErLXq1KlTdUsEAABwHCEcNWpiUpwC/HyqtAX4+WhiUtw597Vhwwa1bdtWoaGhWrFihZYtW6aAgHMP8wAAAN7GdBTUuJNXR4kIDtDEpDj1i4/0dlkAAKABqK3TUViiEDWuX3wkoRsAAOAkTEcBAAAAHEYIBwAAABxGCAcAAAAcRggHAAAAHEYIBwAAABxGCAcAAAAcRggHAAAAHEYIBwAAABxGCAcAAAAcRgiH19xwww2aO3eue3vq1KkKDQ1Vq1atvFgVAABAzeO19agVfvjhB82aNUvZ2dlq2bKlt8sBAACoUdwJR62QnZ2tkJAQAjgAAGgQCOE4L/PmzdMtt9zi3m7Xrp0GDRrk3o6KilJmZqbWr1+vDh06KCgoSB06dND69et/0df777+vm2++Wbm5uWratKlGjhzpxCkAAAB4DSEc5yUxMVHr1q1TRUWF9uzZo9LSUn366aeSpJ07d6qwsFAXX3yxkpOTNW7cOB06dEgPPvigkpOTdejQoSp9de/eXatXr1ZERIQKCws1f/58L5wRAACAcwjhOC+xsbEKDAxUZmam0tLSlJSUpMjISG3dulVpaWnq2rWrVq1apUsuuUTDhw+Xr6+vhg4dqssuu0wrVqzwdvkAAABexYOZOG+JiYlau3attm/frsTERAUHBystLU0bNmxQYmKicnNzFR0dXeWY6OhouVwuL1UMAABQOxDCcdaWZbg0MzVLuflFiggO0FVtrtTatWv1/fffa8qUKQoODtbChQu1YcMGjR07Vt99952ys7Or9JGTk6OePXt66QwAAABqB6aj4Kwsy3Bp8tKv5MovkpXkyi9S6qEWeu+DD1VUVKTWrVura9euWrNmjQ4dOqT4+Hj17t1b27Zt06JFi1RWVqYlS5bo22+/VZ8+fbx9OgAAAF7lkRBujOlpjMkyxmw3xkzyRJ+oXWamZqmotLxKW3mzcJX7+Ktr166SpGbNmik2NladO3eWj4+PQkJCtHLlSs2aNUshISF6+umntXLlSoWGhnrjFAAAAGoNY62tXgfG+EjaJulmSbslfS5pqLX229Mdk5CQYNPT06s1LpzVZtIqneq/FCPp+yeTnS4HAADgrBhjNltrE7xdx8954k74tZK2W2t3WmtLJC2W9HsP9ItaJCI44JzaAQAAcHqeCOGRkn44aXv38bYqjDF/MMakG2PSDxw44IFh4aSJSXEK8POp0hbg56OJSXFeqggAAKDu8kQIN6do+8XMBWvtS9baBGttQlhYmAeGhZP6xUdqRv/fKDI4QEZSZHCAZvT/jfrF/+LfWwAAADgDTyxRuFtS1EnbrSXleqBf1DL94iMJ3QAAAB7giTvhn0u6xBjTxhhzgaQhkt72QL8AAABAvVTtO+HW2jJjzFhJqZJ8JP3LWvtNtSsDAAAA6imPvDHTWvuOpHc80RcAAABQ3/HGTAAAAMBhhHAAAADAYYRwAAAAwGGEcAAAAMBhhHAAAADAYYRwAAAAwGGEcAAAAMBhhHAAAADAYYRwAAAAwGGEcAAAAMBhhHAAAADAYYRwAAAAwGGEcAAAAMBhhHAAAADAYYRwAAAAwGGEcAAAAMBhhHAAAADAYYRwAAAAwGGEcAAAAMBhhHAAAADAYYRwAAAAwGGEcAAAAMBhhHAAAADAYYRwAAAAwGGEcAAAAMBhhHAAAADAYYRwAAAAwGGEcAAAAMBhhHAAAADAYYRwAAAAwGGEcAAAAMBhhHAAAADAYYRwAAAAwGGEcAAAAMBhhHAAAADAYYRwAAAAwGGEcAAAAMBhhHAAAADAYYRwAAAAwGGEcAAAAMBhhHAAAADAYYRwAAAAwGHVCuHGmIHGmG+MMRXGmARPFQUAAADUZ9W9E/61pP6SPvZALQAAAECD4Fudg62130mSMcYz1QAAAAANgGNzwo0xfzDGpBtj0g8cOODUsAAAAECtc8Y74caY9yW1OsWuFGvt8rMdyFr7kqSXJCkhIcGedYUAAABAPXPGEG6t7e5EIQAAAEBDwRKFAAAAgMOqu0ThrcaY3ZKuk7TKGJPqmbIAAACA+qu6q6O8JektD9UCAAAANAhMRwEAAAAcRggHAAAAHEYIBwAAABxGCAcAAAAcRggHAAAAHEYIBwAAABxGCAcAAECNi4mJ0fvvv++x/ubPn68uXbp4pC9jzHxjzOPneew0Y8yC459jjDHWGHPGZcAJ4QAAAGgwjDEjjTGfeLsOQjgAAADgMEI4AAAAHPH555+rffv2at68uUaNGqXi4mL9+OOP6tOnj8LCwtS8eXP16dNHu3fvdh8zf/58xcbGKjAwUG3atNHChQtP2ffEiRPVpUsXFRQUqKCgQHfddZfCw8Ml6UpjzOPGGB9jzOWSXpB0nTGm0BiTf1IXocaY94wxR4wxacaY6BM7jDH/zxjzgzHmsDFmszGma3WvBSEcAAAAjli4cKFSU1O1Y8cObdu2TY8//rgqKio0atQoZWdnKycnRwEBARo7dqwk6ejRoxo3bpxWr16tI0eOaP369br66qur9FlRUaExY8boyy+/1LvvvqugoCDdeeed8vX11fbt2yXpW0k9JN1trf1O0r2SNlhrm1prg0/q6nZJf5EUKilT0slp/3NJV0tqIWmRpNeNMRdW51oQwgEAAOCIsWPHKioqSi1atFBKSor+/e9/KyQkRAMGDFDjxo0VGBiolJQUpaWluY9p1KiRvv76axUVFSk8PFxXXHGFe19paamGDh2qvLw8rVixQo0bN9a+ffu0evVq/f3vf1eTJk0kqUzS3yQNOUN5q6y1H1trj0lKUeXd8ihJstYusNYestaWWWtnSfKXFFeda0EIBwAAgCOioqLcn6Ojo5Wbm6uffvpJ99xzj6Kjo9WsWTN169ZN+fn5Ki8vV5MmTbRkyRK98MILCg8PV3JysrZu3eruY/v27Vq+fLkeffRRXXDBBZKk7OxslZaWKjw8XMHBwVLlHewXJbU8Q3k/nPhgrS2UlCcpQpKMMROMMd8ZYwqOT2EJUuUd8/NGCAcAAECNWJbhUucnP1SbSau0t6BYK9Z/5d6Xk5OjiIgIzZo1S1lZWdq0aZMOHz6sjz/+WJJkrZUkJSUl6b333tOePXt02WWXacyYMe4+Lr/8cs2bN0+9evVSVlaWpMqg7+/vr4MHDyo/P1+SMq21zay1J26h29OU6/4XgjGmqSqnnuQen//9Z0mDJDU/PoWlQJKpzrUhhAMAAMDjlmW4NHnpV3LlF8lKKquwmj/nRc1d87ny8vL0xBNPaPDgwTpy5IgCAgIUHBysvLw8TZ8+3d3Hvn379Pbbb+vo0aPy9/dX06ZN5ePjU2WcoUOH6oknnlD37t21Y8cOhYeHq0ePHpowYYIOHz4sSTLGtDXGJJ7oVlJrY8wFPyu5tzGmy/H2v0jaZK39QVKgKqe0HJDka4x5RFKz6l4fQjgAAAA8bmZqlopKy6u0BVzeTePuvE2xsbGKjY3V1KlTNX78eBUVFSk0NFSdOnVSz5493d+vqKjQrFmzFBERoRYtWigtLU3//Oc/fzHWnXfeqUceeUQ33XSTdu3apVdeeUUlJSVq3769VDkd5Q1J4ce//qGkbyTtNcYcPKmbRZIeVeU0lN+q8kFNSUqVtFrSNknZkop10tSV82VO3Op3UkJCgk1PT3d8XAAAADijzaRVp5z3YSR9/2SyY3UYYzZbaxMcG/AscSccAAAAHhcRHHBO7Q0NIRwAAAAeNzEpTgF+VedvB/j5aGJStVb2qzd8vV0AAAAA6p9+8ZGSKueG5+YXKSI4QBOT4tztDR0hHAAAADWiX3wkofs0mI4CAAAAOIwQDgAAADiMEA4AAAA4jBAOAACAasnKylJ8fLwCAwM1e/Zsj/U7f/58denS5byOXbt2rVq3bu3eNsbsMsZ091Rt1cWDmQAAAKiWp59+WjfccIMyMjLOu49du3apTZs2Ki0tla9v/Y+o3AkHAABAtWRnZ+uKK67wdhl1CiEcAAAA5+2mm27SRx99pLFjx6pp06b64osvNGLECIWFhSk6OlqPP/64KioqJEkVFRV6/PHHFR0drZYtW2rEiBEqKCiQJHXr1k2SFBwcrKZNm2rDhg2SJGut7rvvPgUFBemyyy7TBx984B573rx5uvzyyxUYGKjY2Fi9+OKLDp/9+SOEAwAA4Lx9+OGH6tq1q55//nkVFhZq1qxZKigo0M6dO5WWlqZXXnlF8+bNk1Q5x3v+/Pn66KOPtHPnThUWFmrs2LGSpI8//liSlJ+fr8LCQl133XWSpE2bNik2Nn9pJaQAABCJSURBVFYHDx7U9OnT1b9/f+Xl5UmSWrZsqZUrV+rw4cOaN2+eHnjgAW3ZssULV+HcEcIBAADgEeXl5VqyZIlmzJihwMBAxcTEaMKECXr11VclSQsXLtSDDz6o2NhYNW3aVDNmzNDixYtVVlZ22j5btmyp8ePHy8/PT4MHD1ZcXJxWrVolSUpOTlbbtm1ljFFiYqJ69OihdevWOXKu1VX/Z70DAADA45ZluNyvpM/LydcVOT/qloMHVVJSoujoaPf3oqOj5XK5JEm5ubm/2FdWVqZ9+/addpzIyEgZY6ock5ubK0lavXq1pk+frm3btqmiokI//fSTfvOb33j6VGsEd8IBAABwTpZluDR56Vdy5RfJSjpWVq6lW1z65Idj8vPzU3Z2tvu7OTk5ioysfHV9RETEL/b5+vrqoosuqhK0T+ZyuWStrXJMRESEjh07pgEDBuihhx7Svn37lJ+fr969e1f5bm1GCAcAAMA5mZmapaLS8iptJeUVevb97Ro0aJBSUlJ05MgRZWdn69lnn9Udd9whSRo6dKj+9re/6fvvv1dhYaGmTJmiwYMHy9fXV2FhYWrUqJF27txZpd/9+/dr9uzZKi0t1euvv67vvvtOvXv3VklJiY4dO6awsDD5+vpq9erVevfddx27BtXFdBQAAACck9z8otO2r3zuOd13332KjY3VhRdeqDFjxmj06NGSpNGjRys3N1fdunVTcXGxkpKS9Nxzz0mSGjdurJSUFHXu3FmlpaVas2aNJKljx476z3/+o9DQUF100UV64403FBISIkmaPXu2Bg0apGPHjumWW25R3759HTh7zzDeuGWfkJBg09PTHR8XAAAA1df5yQ/lOkUQjwwO0KeTbvJCRadnjNlsrU3wdh0/x3QUAAAAnJOJSXEK8POp0hbg56OJSXFeqqjuYToKAAAAzkm/+MoHLU+sjhIRHKCJSXHudpwZIRwAAADnrF98JKG7GpiOAgAAADiMEA4AAAA4jBBeh/3aK14BAABQexHCa6EtW7YoPj5egYGBGjhwoAYPHqypU6dq7dq1at26tZ566im1atVKo0aNUkVFhZ588km1bdtWISEhGjRokPLy8tx9bdy4Uddff72Cg4N11VVXae3ate59N9xwg/77v/9bnTt3VmBgoHr06KGDBw964YwBAAAalmqFcGPMTGPMVmPMl8aYt4wxwZ4qrKEqKSnRrbfeqpEjRyovL09Dhw7VW2+95d6/d+9e5eXlKTs7Wy+99JJmz56tZcuWKS0tTbm5uWrevLn+9Kc/Sap8zWtycrKmTp2qvLw8PfPMMxowYIAOHDjg7m/RokWaN2+e9u/fr5KSEj3zzDOOnzMAAEBDU9074e9J+i9r7ZWStkmaXP2SGraNGzeqrKxM48aNk5+fn/r3769rr73Wvb9Ro0aaPn26/P39FRAQoBdffFF//etf1bp1a/n7+2vatGl64403VFZWpgULFqh3797q3bu3GjVqpJtvvlkJCQl655133P2NGjVKl156qQICAjRo0CBlZmZ647QBAAAalGotUWitffekzY2SbqteOQ3TsgyXe53NC7I3qHHzMBlj3PujoqLcn8PCwnThhRe6t7Ozs3XrrbeqUaP/+/eUj4+P9u3bp+zsbL3++utasWKFe19paaluvPFG93arVq3cnxs3bqzCwkKPnx8AAACq8uSc8NGSVp9upzHmD8aYdGNM+snTIRq6ZRkuTV76lVz5RbKSCkxT7crZrbe27HZ/54cffnB/PjmcS5UBffXq1crPz3f/FBcXKzIyUlFRURo+fHiVfUePHtWkSZOcOj0AAACcwhlDuDHmfWPM16f4+f1J30mRVCZp4en6sda+ZK1NsNYmhIWFeab6emBmapaKSsvd2/6Rl8mqkSZMe0plZWVavny5Pvvss9Mef++99yolJUXZ2dmSpAMHDmj58uWaNm2aPvvsM61YsUKpqakqLy9XcXGx1q5dq927d5+2PwAAANS8M05HsdZ2/7X9xpg7JfWR9DtrrfVUYQ1Fbn5RlW3j46ewW6do95rZCg6ep169eqlPnz7y9/c/5fH333+/rLXq0aOHcnNz1bJlSw0ePFgXXHCBmjRpouXLl+vhhx/W0KFD5ePjo2uvvVb/8z//48SpAQAA4DRMdXKzMaanpGclJVprz3qOSUJCgk1PTz/vceuTzk9+KNfPgrgkRQYH6NNJN0mSOnbsqDFjxujuu+8+636nTZum7du3a8GCBR6rFQAAoK4xxmy21iZ4u46fq+6c8OclBUp6zxiTaYx5wQM1NSgTk+IU4OdTpc3mfqO7f9tc0dHRGjRokD7//HP98Y9/1CeffHLaNb+///57JSYmKjAwUDfffDPrfQMAANRi1V0dpZ2nCmmo+sVHSpJ7dZSI4ABd5Ws16Y6eOnDggPbv369//etfuvnmm3XllVfq1VdfVc+ePfXBBx9owIAB2rp1q8LCwjRs2DBdd911evfdd7Vp0yYlJyfr97///RlGBwAAgDdUazrK+WI6ytmJiYnRI488otGjR+upp57S119/rVdffdW9PykpScOGDdONN96o2NhYFRQUqEmTJpKkYcOGqVGjRkxHAQAADVptnY5SrTvh8LyT1wzfW1CsnGMBkvSra36feFPmiQAuSdHR0VWWNgQAAEDtQQivRU6sGX5iycKyCqt/fbJLV3dyudf8njNnzi+Oy87O1o8//qijR4+6g3hOTs4v1hQHAABA7eDJl/Wgmn6+ZrgkHSuv0MzULN1xxx2nXfM7OjpaCQkJevTRR1VSUqJPPvmkyh1zAAAA1C6E8Frk52uGn9weFRWl5cuX64knnlBYWJiioqI0c+ZMVVRUSJIWLVqkTZs2qUWLFpo+fbpGjBjhZOkAAAA4BzyYWYuczZrhAAAAOHu19cFM7oTXIqdaMzzAz0cTk+K8VBEAAABqAg9m1iKnWjN8YlKcux0AAAD1AyG8lukXH0noBgAAqOeYjgIAAAA4jBAOAAAAOIwQDgAAADiMEA4AAAA4jBAOAAAAOIwQDgAAADiMEA4AAAA4jBAOAAAAOIwQfpbmz5+vLl26uLeNMdq+fbsXKwIAAEBdRQgHAAAAHEYIBwAAABzW4EL4zJkzNWDAgCpt9913n8aPH6+CggLdddddCg8PV2RkpKZOnary8vIz9llQUKARI0YoLCxM0dHRevzxx1VRUSFJio6O1ubNmyVJCxYskDFG3377rSRp7ty56tevn4fPEAAAALVdgwvhd9xxh9asWaP8/HxJUllZmZYsWaLhw4frzjvvlK+vr7Zv366MjAy9++67mjt37hn7vO+++1RQUKCdO3cqLS1Nr7zyiubNmydJSkxM1Nq1ayVJH3/8sWJjY5WWlubeTkxMrJkTBQAAQK3V4EJ4eHi4unXrptdff12StGbNGoWGhqp169ZavXq1/v73v6tJkyZq2bKlHnjgAS1evPhX+ysvL9eSJUs0Y8YMBQYGKiYmRhMmTNCrr74qqTKEnwjd69at0+TJk93baWlphHAAAIAGyNfbBThhWYZLM1OzlJtfpIjgAHVKvEULFizQmDFjtGDBAg0fPlzZ2dkqLS1VeHi4+7iKigpFRUX9at8HDx5USUmJoqOj3W3R0dFyuVySKkP4Qw89pL1796q8vFyDBw/W9OnTtWvXLhUUFOjqq6+umZMGAABArVXvQ/iyDJcmL/1KRaWVc7td+UVaaVrLlfGFvv76a61cuVJPP/20/Pz85O/vr4MHD8rX9+wvS2hoqPz8/JSdna327dtLknJychQZGSlJateunRo3bqzZs2erW7duCgwMVKtWrfTSSy+pS5cuatSowf0xAgAAoMGr9wlwZmqWO4CfcMz6qMllnTVs2DBde+21uvjiixUeHq4ePXpowoQJOnz4sCoqKrRjxw731JHT8fHx0aBBg5SSkqIjR44oOztbzz77rO644w73dxITE/X888+7p57ccMMNVbYBAADQsNT7EJ6bX3TK9kaXJuqrr77S8OHD3W2vvPKKSkpK1L59ezVv3ly33Xab9uzZc8YxnnvuOTVp0kSxsbHq0qWLhg0bptGjR7v3JyYm6siRI+rWrdsptwEAANCwGGut44MmJCTY9PR0R8bq/OSHcp0iiIfqsL6dfZf27t2rZs2aOVILAAAAnGWM2WytTfB2HT9X7++ET0yKU4CfT5W2C32NwnM+0JAhQwjgAAAAcFy9fzCzX3zlA5InVke5qLH0xZMD5dMmRv9cs8a7xQEAAKBBqvfTUQAAANBwMR0FAAAAgCRCOAAAAOA4QjgAAADgMEI4AAAA4DBCOAAAAOAwQjgAAADgMEI4AAAA4DBCOAAAAOAwQjgAAADgMEI4AAAA4DCvvLbeGHNAUrbjA3tPqKSD3i6iAeA61zyucc3jGtc8rnHN4xrXPK7x2Yu21oZ5u4if80oIb2iMMenW2gRv11HfcZ1rHte45nGNax7XuOZxjWse17juYzoKAAAA4DBCOAAAAOAwQrgzXvJ2AQ0E17nmcY1rHte45nGNax7XuOZxjes45oQDAAAADuNOOAAAAOAwQrhDjDEzjTFbjTFfGmPeMsYEe7um+sYYM9AY840xpsIYwxPjHmSM6WmMyTLGbDfGTPJ2PfWRMeZfxpj9xpivvV1LfWWMiTLGfGSM+e7474r7vV1TfWOMudAY85kx5ovj13i6t2uqr4wxPsaYDGPMSm/XgvNDCHfOe5L+y1p7paRtkiZ7uZ766GtJ/SV97O1C6hNjjI+kf0jqJam9pKHGmPberapemi+pp7eLqOfKJE2w1l4uqZOkP/Hfsscdk3STtfYqSVdL6mmM6eTlmuqr+yV95+0icP4I4Q6x1r5rrS07vrlRUmtv1lMfWWu/s9ZmebuOeuhaSduttTuttSWSFkv6vZdrqnestR9LyvN2HfWZtXaPtXbL8c9HVBlgIr1bVf1iKxUe3/Q7/sPDZx5mjGktKVnSXG/XgvNHCPeO0ZJWe7sI4CxFSvrhpO3dIrigjjPGxEiKl7TJu5XUP8enSWRK2i/pPWst19jz/i7pYUkV3i4E58/X2wXUJ8aY9yW1OsWuFGvt8uPfSVHln0QXOllbfXE21xgeZ07Rxp0t1FnGmKaS3pQ03lp72Nv11DfW2nJJVx9/9uktY8x/WWt51sFDjDF9JO231m42xtzg7Xpw/gjhHmSt7f5r+40xd0rqI+l3lrUhz8uZrjFqxG5JUSdtt5aU66VagGoxxvipMoAvtNYu9XY99Zm1Nt8Ys1aVzzoQwj2ns6S+xpjeki6U1MwYs8Bae4eX68I5YjqKQ4wxPSX9WVJfa+1P3q4HOAefS7rEGNPGGHOBpCGS3vZyTcA5M8YYSf8r6Ttr7bPerqc+MsaEnVj9yxgTIKm7pK3erap+sdZOtta2ttbGqPL38YcE8LqJEO6c5yUFSnrPGJNpjHnB2wXVN8aYW40xuyVdJ2mVMSbV2zXVB8cfKB4rKVWVD7K9Zq39xrtV1T/GmH9L2iApzhiz2xhzl7drqoc6Sxou6abjv4czj99NhOeES/rIGPOlKv8B/561liX0gFPgjZkAAACAw7gTDgAAADiMEA4AAAA4jBAOAAAAOIwQDgAAADiMEA4AAAA4jBAOAAAAOIwQDgAAADiMEA4AAAA47P8DOexqc8Ezk2sAAAAASUVORK5CYII=\n",
+ "text/plain": [
+ ""
+ ]
+ },
+ "metadata": {
+ "needs_background": "light"
+ },
+ "output_type": "display_data"
+ }
+ ],
+ "source": [
+ "plot_words(embedding, sequences, 0)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Plot the word of embeddings of the second sentence. Recall that the second sentence contains the same words are the first sentence, just in a different order. You can see that the order of the words does not affect the vector representations. "
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 17,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "image/png": "iVBORw0KGgoAAAANSUhEUgAAAuQAAAFoCAYAAAAfPu5bAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO3de1hVZf7//9ctMISKoIIKSCBadvhNRWFWHrA+Jnkc0zymppZTn8mcynE+mk5p42RlzszPmvlUOqPloexgmpraaSTLQ6FYViOOJ7CNmoqgKGfu7x/o/khppmy4EZ6P69rXtde91rrXe61ru6+Xi3vd21hrBQAAAMCNOq4LAAAAAGozAjkAAADgEIEcAAAAcIhADgAAADhEIAcAAAAcIpADAAAADvm7LiAsLMzGxsa6LgMAAAA12KZNmw5Za8Nd13EmzgN5bGysUlJSXJcBAACAGswYk+66hrNhyAoAAADgEIEcAAAAcIhADgAAADhEIAcAAAAcIpADAAAADhHIAQAAAIcI5ABqjNTUVF1//fUKDg7WgAEDNHDgQE2aNElz585V+/bty21rjNGOHTskSQUFBfrd736nSy+9VE2bNtUDDzygvLw877bLly/Xddddp9DQUN1yyy366quvvOtiY2P13HPP6ZprrlFISIgGDBig/Pz8qjlhAECNQCAHUCMUFhaqd+/eGjp0qLKystSvXz+9/fbbP2vf//mf/9H27du1ZcsW7dixQx6PR08++aQkafPmzRo5cqReeuklHT58WPfff7969eqlgoIC7/5vvPGGVq1apd27d+urr77S3LlzK+MUAQA1FIEcQI2wYcMGFRUV6eGHH1ZAQIDuuusutWnT5pz7WWs1a9Ys/eUvf1GjRo0UHBysxx57TK+//rokadasWbr//vvVtm1b+fn56Z577lFgYKA2bNjg7WPMmDGKjIxUo0aN1LNnT23ZsqXSzhMAUPM4/6VOAKiIJakeTV+dpu3rVimvTgMt3ZKp3vFRkqSYmJhz7n/w4EGdOHFCN9xwg7fNWquSkhJJUnp6ul555RU9//zz3vWFhYXKzMz0Ljdr1sz7vm7duuXWAQBwLgRyABetJakeTVi8VXlFJfKr30j5OQc1/u2y8d2946OUkZGhli1bql69ejpx4oR3v/3793vfh4WFKSgoSN98842ioqJ+dIzo6GhNnDhREydOrPwTAgDUSgxZAXDRmr46TXlFZXeyA6OukIyfvt/wjp5971stXrxYn3/+uSTp2muv1TfffKMtW7YoPz9fkydP9vZRp04djRo1So888oi+//57SZLH49Hq1aslSaNGjdKLL76ojRs3ylqr48ePa8WKFTp27FjVniwAoMYikAO4aGVm/99MKMYvQOF3PqbcrR9pw5O/0qJFi9SnTx9J0uWXX67HH39cnTt31mWXXfajGVeeeeYZtWrVSjfddJMaNGigzp07Ky0tTZKUkJCgWbNmafTo0WrYsKFatWrFQ5sAAJ8y1lqnBSQkJNiUlBSnNQC4OMTGxmr27Nnq3LmzJKnd0x/Lk52nzNm/UaMuD+iSS6+RJEWFBumz8bdp+PDhat68uaZOneqybABANWCM2WStTXBdx5lwhxzARWtcUmsFBfgp8r6/e8N4UICfxiW1dlwZAAA/Hw91ArhonZpNZfrqNGVm5ykyNEjjklp72wEAuBhwhxzARWnbtm1q0aKF8tPWyvPiSM3qHKjPxt+mLUtnqX///ho2bJjefvttvfPOOzp9WNzmzZsVHx+v4OBg9evXTwMGDNCkSZMcngkAoLYjkAO46GzevFldunTR888/r4EDB/5o/bvvvquBAwcqOztbvXr10ujRoyWVzR9+5513avjw4crKytKgQYP0zjvvVHX5AACUQyAHcFFZu3atevXqpVdeeUU9evQ44zbt27dXt27d5Ofnp6FDh+rLL7+UVPZrnsXFxRozZowCAgLUp08f3XjjjVVZPgAAP+LTQG6MmW+M2WeMOWqM2W6Muc+X/QPAiy++qFtuuUW33nrrWbf54S9n5ufnq7i4WJmZmYqKipIxxrs+Ojq6UusFAOBcfH2HfJqkWGttA0m9JE01xtxwjn0A4KyWpHrU7umP1WL8Cu3PydeI3/9JGRkZeuSRR867r4iICHk8Hp0+3evevXt9WS4AAOfNp4HcWvuNtbbg1OLJV0tfHgNA7bEk1aMJi7fKk50nK6m41Grh5oN66Nl/6pNPPtH48ePPq7+bb75Zfn5+euGFF1RcXKylS5d6f80TAABXfD7toTHm75KGSwqSlCrpvTNs82tJv5akSy+91NclAKghpq9OU15RSbm2gpJS/X3dfn3wwQe69dZbFRAQ8LP7+8UvfqHFixfrvvvu04QJE9S1a1f16NFDgYGBvi4dAICfrVJ+qdMY4yfpZkmdJD1jrS0627b8UieAs2kxfoXO9A1lJO1+urtPjtG2bVs98MADGjFihE/6AwBUT7XulzqttSXW2k8lNZf035VxDAA1X2Ro0Hm1/xzJycnav3+/iouL9corr+irr77SHXfcccH9AQBQUZU97aG/GEMO4AKNS2qtoAC/cm1BAX4al9T6gvtMS0vTtddeq5CQEM2YMUNvvfWWIiIiKloqAAAXzGdDVowxTSTdJmm5pDxJnSUtljTYWrv0bPsxZAXAT1mS6tH01WnKzM5TZGiQxiW1Vu/4KNdlAQAuMtV5yIovH+q0Khue8qLK7rynS3r4p8I4AJxL7/goAjgAoEbzWSC31h6UlOir/gAAAIDaoLLHkAMAAAD4CQRyAAAAwCECOQAAAOAQgRwAAABwiEAOAAAAOEQgBwAAABwikAMAAAAOEcgBAAAAhwjkAAAAgEMEcgAAAMAhAjkAAADgEIEcAAAAcIhADgAAADhEIAcAAAAcIpADAAAADhHIAQAAAIcI5AAAAIBDBHIAAADAIQI5AAAA4BCBHAAAAHCIQA4AAAA4RCAHAAAAHCKQAwAAAA4RyAEAAACHCOQAAACAQwRyAAAAwCECOQAAAOAQgRwAAABwiEAOAAAAOEQgBwAAABwikAMAAAAOEcgBAAAAhwjkAAAAgEMEcgAAAMAhAjkAAADgEIEcAAAAcIhADgAAADhEIAcAAAAcIpADAAAADhHIAQAAAIcI5EA1MGfOHPXs2dO73KpVK/Xv39+7HB0drS1btmjdunVq06aNQkJC1KZNG61bt867TadOnTRp0iTdcsstql+/vnr27KnDhw/r7rvvVoMGDdSmTRvt2bPHu/1vf/tbRUdHq0GDBrrhhhu0du1a77rJkyerf//+GjZsmIKDg3X11VcrJSWlci8CAAC1lM8CuTEm0BjzD2NMujHmmDEm1RjT1Vf9AzVZYmKi1q5dq9LSUu3bt09FRUX67LPPJEm7du1Sbm6uLr30UnXv3l1jxozR4cOH9eijj6p79+46fPiwt5/XX39d8+bNk8fj0c6dO3XzzTdrxIgRysrK0pVXXqkpU6Z4t23Tpo22bNmirKwsDR48WP369VN+fr53/bvvvquBAwcqOztbvXr10ujRo6vuggAAUIv48g65v6S9khIlhUj6g6Q3jDGxPjwGUCPFxcUpODhYW7ZsUXJyspKSkhQVFaVt27YpOTlZHTp00IoVK3TZZZdp6NCh8vf316BBg3TFFVdo2bJl3n5GjBihli1bKiQkRF27dlXLli3VuXNn+fv7q1+/fkpNTfVuO2TIEDVu3Fj+/v4aO3asCgoKlJaW5l3fvn17devWTX5+fho6dKi+/PLLKr0mAADUFv6+6shae1zS5NOalhtjdku6QdIeXx0HqCmWpHo0fXWaMrPzFBkapLhfttGaNWu0Y8cOJSYmKjQ0VMnJyVq/fr0SExOVmZmpmJiYcn3ExMTI4/F4l5s2bep9HxQU9KPl3Nxc7/KMGTM0e/ZsZWZmyhijo0eP6tChQ971zZo1876vW7eu8vPzVVxcLH9/n31tAAAAVeIYcmNMU0mXS/rmDOt+bYxJMcakHDx4sLJKAKqtJakeTVi8VZ7sPFlJnuw8bTfRen3pKq1du1aJiYlKTExUcnKykpOTlZiYqMjISKWnp5frJyMjQ1FRUed9/LVr1+qZZ57RG2+8oSNHjig7O1shISGy1vroDAEAwM9VKYHcGBMgaYGkV6y123643lr7srU2wVqbEB4eXhklANXa9NVpyisqKddWJ+pqbdrwqfLy8tS8eXN16NBBq1at0uHDhxUfH69u3bpp+/btWrhwoYqLi7Vo0SJ9++236tGjx3kf/9ixY/L391d4eLiKi4v15JNP6ujRo746PQAAcB58HsiNMXUkzZNUKImnwIAzyMzO+1FbQKMomYBL1KFDB0lSgwYNFBcXp3bt2snPz0+NGzfW8uXLNWPGDDVu3FjPPvusli9frrCwsPM+flJSkrp27arLL79cMTExuuSSSxQdHV3h8wIAAOfP+PJP1MYYI+mfkmIldbPW/jh1/EBCQoJlOjXUNu2e/lieM4TyqNAgfTb+NgcVAQBQsxljNllrE1zXcSa+vkP+v5KulNTz54RxoLYal9RaQQF+5dqCAvw0Lqm1o4oAAIArPpsuwRgTI+l+SQWS9pfdLJck3W+tXeCr4wA1Qe/4sgcxT59lZVxSa287AACoPXw57WG6JHPODQFIKgvlBHAAAFBp0x4CAAAAODcCOQAAAOAQgRwAAABwiEAOAAAAOEQgBwAAABwikAMAAAAOEcgBAAAAhwjkAAAAgEMEcgAAAMAhAjkAAADgEIEcAAAAcIhADgAAADhEIAcAAAAcIpADAAAADhHIAQAAAIcI5AAAAIBDBHIAAADAIQI5AAAA4BCBHAAAAHCIQA4AAAA4RCAHAAAAHCKQAzVEbGyspk+frmuuuUb16tXTvffeqwMHDqhr164KDg5W586ddeTIEUlSv3791KxZM4WEhKhjx4765ptvvP0MHz5cDz74oLp3767g4GC1bdtWO3fudHVaAADUeARyoAZ5++239cEHH2j79u1atmyZunbtqqeeekqHDh1SaWmpZs6cKUnq2rWr/vOf/+j777/X9ddfr7vvvrtcP6+99pqeeOIJHTlyRK1atdLEiRNdnA4AALWCv+sCAPjOQw89pKZNm0qSOnTooCZNmig+Pl6SdOedd+qjjz6SJI0cOdK7z+TJk9WwYUPl5OQoJCREktSnTx/deOONkqS7775bjz76aFWeBgAAtQqBHLhILUn1aPrqNGVm5ykyNEgnCku8YVySgoKCfrScm5urkpISTZw4UW+++aYOHjyoOnXK/lB26NAhbyBv1qyZd7+6desqNze3is4KAIDahyErwEVoSapHExZvlSc7T1aSJztPR04U6rMdh86578KFC7V06VJ9+OGHysnJ0Z49eyRJ1trKLRoAAJwRgRy4CE1fnaa8opJybdZKb6TsPee+x44dU2BgoBo3bqwTJ07oscceq6wyAQDAz0AgBy5Cmdl5Z2w/nFt4zn2HDRummJgYRUVF6aqrrtJNN93k6/IAAMB5MK7/TJ2QkGBTUlKc1gBcbNo9/bE8ZwjlUaFB+mz8bQ4qAgCgejPGbLLWJriu40y4Qw5chMYltVZQgF+5tqAAP41Lau2oIgAAcKGYZQW4CPWOj5KkcrOsjEtq7W0HAAAXDwI5cJHqHR9FAAcAoAZgyAoAAADgEIEcAAAAcIhADgAAADhEIAcAAAAcIpADAAAADhHIAQAAAIcI5AAAAIBDBHIAAADAIQI5AAAA4JBPA7kxZrQxJsUYU2CMmevLvgEAAICayN/H/WVKmiopSVKQj/sGAAAAahyfBnJr7WJJMsYkSGruy74BAACAmsjJGHJjzK9PDm1JOXjwoIsSAAAAgGrBSSC31r5srU2w1iaEh4e7KAEAAACoFphlBQAAAHCIQA4AAAA45NOHOo0x/if79JPkZ4y5RFKxtbbYl8cBAAAAagpf3yGfJClP0nhJQ06+n+TjYwAAAAA1hq+nPZwsabIv+wQAAABqMsaQAwAAAA4RyAEAAACHCOQAAACAQwRyAAAAwCECOQAAAOAQgRwAAABwiEAOAAAAOEQgBwAAABwikKNaGT58uCZN4sddAQBA7UEgBwAAABwikAMAAAAOEcjhVGpqqq6//noFBwdrwIABys/P966bNWuWWrVqpUaNGqlXr17KzMz0rnv//ffVunVrhYSE6De/+Y0SExM1e/ZsF6cAAABQIQRyOFNYWKjevXtr6NChysrKUr9+/fT2229Lkj7++GNNmDBBb7zxhvbt26eYmBgNHDhQknTo0CHdddddmjZtmg4fPqzWrVtr3bp1Lk8FAADgghHI4cyGDRtUVFSkhx9+WAEBAbrrrrvUpk0bSdKCBQs0cuRIXX/99QoMDNS0adO0fv167dmzR++9956uvvpq9enTR/7+/hozZoyaNWvm+GwAAAAuDIEczmRmZioqKkrGGG9bTEyMd92p95JUv359NW7cWB6PR5mZmYqOjvauM8aoefPmVVc4AACAD/m7LgC1y5JUj6avTlNmdp7qZe1T5p69stZ6Q3lGRoZatmypyMhIpaene/c7fvy4Dh8+rKioKEVEROi7777zrrPWllsGAAC4mHCHHFVmSapHExZvlSc7T1bSsZAWOlZYqvt+/6SKi4u1ePFiff7555KkwYMHa86cOdqyZYsKCgr02GOPqW3btoqNjVX37t21detWLVmyRMXFxfrb3/6m/fv3uz05AACAC0QgR5WZvjpNeUUl3mXjF6Cw3o9p0YL5atiwoRYtWqQ+ffpIkv7rv/5Lf/zjH9W3b19FRERo586dev311yVJYWFhevPNN/X73/9ejRs31rfffquEhAQFBgY6OS8AAICKMNZapwUkJCTYlJQUpzWgarQYv0Jn+rQZSbuf7n7B/ZaWlqp58+ZasGCBbr311gvuBwAA1FzGmE3W2gTXdZwJd8hRZSJDg86r/aesXr1a2dnZKigo0FNPPSVrrW666aaKlggAAFDlCOSoMuOSWisowK9cW1CAn8YltT7vvtavX6+WLVsqLCxMy5Yt05IlSxQUdP7BHgAAwDWGrKBKnT7LSmRokMYltVbv+CjXZQEAgBquOg9ZYdpDVKne8VEEcAAAgNMwZAUAAABwiEAOAAAAOEQgBwAAABwikAMAAAAOEcgBAAAAhwjkAAAAgEMEcgAAAMAhAjkAAADgEIEcAAAAcIhAjmqhU6dOmj17tnd50qRJCgsLU7NmzRxWBQAAUPn8XRcA/NDevXs1Y8YMpaenq0mTJq7LAQAAqFTcIUe1k56ersaNGxPGAQBArUAgR4XNmTNHPXv29C63atVK/fv39y5HR0dry5YtWrdundq0aaOQkBC1adNG69at+1FfH374oW6//XZlZmaqfv36Gj58eFWcAgAAgDMEclRYYmKi1q5dq9LSUu3bt09FRUX67LPPJEm7du1Sbm6uLr30UnXv3l1jxozR4cOH9eijj6p79+46fPhwub46d+6slStXKjIyUrm5uZo7d66DMwIAAKg6BHJUWFxcnIKDg7VlyxYlJycrKSlJUVFR2rZtm5KTk9WhQwetWLFCl112mYYOHSp/f38NGjRIV1xxhZYtW+a6fAAAAKd4qBM+kZiYqDVr1mjHjh1KTExUaGiokpOTtX79eiUmJiozM1MxMTHl9omJiZHH43FUMQAAQPVAIMcFWZLq0fTVacrMzlNkaJCubXGN1qxZo927d+uxxx5TaGioFixYoPXr12v06NH697//rfT09HJ9ZGRk6I477nB0BgAAANUDQ1Zw3pakejRh8VZ5svNkJXmy87T6cCN98NHHysvLU/PmzdWhQwetWrVKhw8fVnx8vLp166bt27dr4cKFKi4u1qJFi/Ttt9+qR48erk8HAADAKZ8GcmNMI2PMO8aY48aYdGPMYF/2j+ph+uo05RWVlGsraRChEr9AdejQQZLUoEEDxcXFqV27dvLz81Pjxo21fPlyzZgxQ40bN9azzz6r5cuXKywszMUpAAAAVBvGWuu7zox5TWUh/15J10laIekWa+03Z9snISHBpqSk+KwGVL4W41foTJ8aI2n3092ruhwAAIBzMsZsstYmuK7jTHx2h9wYU09SX0l/sNbmWms/lfSupKG+Ogaqh8jQoPNqBwAAwNn5csjK5ZJKrLXbT2v7UtLVP9zQGPNrY0yKMSbl4MGDPiwBVWFcUmsFBfiVawsK8NO4pNaOKgIAALh4+TKQ15eU84O2HEnBP9zQWvuytTbBWpsQHh7uwxJQFXrHR2lan18qKjRIRlJUaJCm9fmlesdHuS4NAADgouPLaQ9zJTX4QVsDScd8eAxUE73jowjgAAAAPuDLO+TbJfkbYy47re1aSWd9oBMAAACo7XwWyK21xyUtlvSkMaaeMaadpF9JmuerYwAAAAA1ja9/GOg3koIkfS/pNUn//VNTHgIAAAC1nS/HkMtamyWpty/7BAAAAGoyX98hBwAAAHAeCOQAAACAQwRyAAAAwCECOQAAAOAQgRwAAABwiEAOAAAAOEQgBwAAABwikAMAAAAOEcgBAAAAhwjkAAAAgEMEcgAAAMAhAjkAAADgEIEcAAAAcIhADgAAADhEIAcAAAAcIpADAAAADhHIAQAAAIcI5AAAAIBDBHIAAADAIQI5AAAA4BCBHAAAAHCIQA4AAAA4RCAHAAAAHCKQAwAAAA4RyAEAAACHCOQAAACAQwRyAAAAwCECOQAAAOAQgRwAAABwiEAOAAAAOEQgBwAAABwikAMAAAAOEcgBAAAAhwjkAAAAgEMEcgAAAMAhAjkAAADgEIEcAAAAcIhADgAAADhEIAcAAAAcIpADAAAADhHIAQAAAId8EsiNMaONMSnGmAJjzFxf9AkAAADUBv4+6idT0lRJSZKCfNQnAAAAUOP5JJBbaxdLkjEmQVJzX/QJAAAA1AZOxpAbY359cohLysGDB12UAAAAAFQLTgK5tfZla22CtTYhPDzcRQkAAABAtXDOQG6MWWOMsWd5fVoVRQIAAAA11TnHkFtrO1VBHQAAAECt5JOHOo0x/if78pPkZ4y5RFKxtbbYF/0DAAAANZWvxpBPkpQnabykISffT/JR3wAAAECN5atpDydLmuyLvgAAAIDaxMksKwAAAADKEMgBAAAAhwjkAAAAgEMEcgAAAMAhAjkAAADgEIEcAAAAVSo2NlYffvihz/qbO3eu2rdv75O+jDFzjTFTL3DfycaY+Sffx578ZftzzmpIIAcAAECtZIwZboz51HUdBHIAAADAIQI5AAAAqtwXX3yhq666Sg0bNtSIESOUn5+vI0eOqEePHgoPD1fDhg3Vo0cPfffdd9595s6dq7i4OAUHB6tFixZasGDBGfseN26c2rdvr5ycHOXk5Ojee++VpGuMMR5jzFRjjJ8x5kpJL0q62RiTa4zJPq2LMGPMB8aYY8aYZGNMzKkVxpj/3xiz1xhz1BizyRjToaLXgkAOAACAKrdgwQKtXr1aO3fu1Pbt2zV16lSVlpZqxIgRSk9PV0ZGhoKCgjR69GhJ0vHjxzVmzBitXLlSx44d07p163TdddeV67O0tFSjRo3SV199pffff18hISG655575O/vL0lfS4qX1EXSfdbaf0t6QNJ6a219a23oaV3dLemPksIkbZF0evL/QtJ1khpJWijpTWPMJRW5FgRyAAAAVLnRo0crOjpajRo10sSJE/Xaa6+pcePG6tu3r+rWravg4GBNnDhRycnJ3n3q1Kmjr7/+Wnl5eYqIiNDVV1/tXVdUVKRBgwYpKytLy5YtU926dXXgwAGtXLlSf/3rXyWp1Fr7vaS/SBp4jvJWWGs/sdYWSJqosrvo0ZJkrZ1vrT1srS221s6QFCipdUWuBYEcAAAAVS46Otr7PiYmRpmZmTpx4oTuv/9+xcTEqEGDBurYsaOys7NVUlKievXqadGiRXrxxRcVERGh7t27a9u2bd4+duzYoaVLl+qJJ57QL37xC0lSenq6ioqKFBERIUnXnRyW8pKkJucob++pN9baXElZkiIlyRgz1hjzb2NMzsn+QlR2J/2CEcgBAABQ6ZaketTu6Y/VYvwK7c/J17J1W73rMjIyFBkZqRkzZigtLU0bN27U0aNH9cknn0iSrLWSpKSkJH3wwQfat2+frrjiCo0aNcrbx5VXXqk5c+aoa9euSktLk1QW+gMDA3Xo0CFJ2mKtDbXWNrDWnrq1bs9Srvd/C8aY+iobnpJ5crz4/0jqL6nhyWEuOZJMRa4NgRwAAACVakmqRxMWb5UnO09WUnGp1dxZL2n2qi+UlZWlp556SgMGDNCxY8cUFBSk0NBQZWVlacqUKd4+Dhw4oHfffVfHjx9XYGCg6tevLz8/v3LHGTRokJ566il17txZO3fuVEREhLp06aKxY8dKUh1jTB1jTEtjTOKpbiU1N8b84gcldzPGtD/Z/kdJG621eyUFSyqWdFCSvzHmcUkNKnp9COQAAACoVNNXpymvqKRcW9CVHTXmnrsUFxenuLg4TZo0SQ8//LDy8vIUFhamm266SXfccYd3+9LSUs2YMUORkZFq1KiRkpOT9fe///1Hx7rnnnv0+OOP67bbbtOePXv06quvqrCwUJL+P0lHJL0lKeLk5h9L+kbSfmPModO6WSjpCZUNVblBZQ95StJqSSslbZeULilfpw1vuVDm1J8AXElISLApKSlOawAAAEDlaTF+xRnHhhhJu5/uXiU1GGM2WWsTquRg54k75AAAAKhUkaFB59Ve2xDIAQAAUKnGJbVWUED58d5BAX4al1Sh2QJrDH/XBQAAAKBm6x0fJalsLHlmdp4iQ4M0Lqm1t722I5ADAACg0vWOjyKAnwVDVgAAAACHCOQAAACAQwRyAAAAwCECOQAAAHwmLS1N8fHxCg4O1syZM33W79y5c9W+ffsL2nfNmjWSdM2pZWPMHmNMZ99UVnE81AkAAACfefbZZ9WpUyelpqZecB979uxRixYtVFRUJH//mh9XuUMOAAAAn0lPT9fVV1/tuoyLCoEcAAAAPnHbbbfpX//6l0aPHq369evryy+/1LBhwxQeHq6YmBhNnTpVpaWlkqTS0lJNnTpVMTExatKkiYYNG6acnBxJUtfjSxMAABFYSURBVMeOHSVJoaGhql+/vtavXy9JstbqoYceUkhIiK644gp99NFH3mPPmTNHV155pYKDgxUXF6eXXnqpis/+whHIAQAA4BMff/yxOnTooBdeeEG5ubmaMWOGcnJytGvXLiUnJ+vVV1/VnDlzJJWNCZ87d67+9a9/adeuXcrNzdXo0aMlSZ988okkKTs7W7m5ubr55pslSRs3blRcXJwOHTqkKVOmqE+fPsrKypIkNWnSRMuXL9fRo0c1Z84cPfLII9q8ebODq3D+COQAAADwuZKSEi1atEjTpk1TcHCwYmNjNXbsWM2bN0+StGDBAj366KOKi4tT/fr1NW3aNL3++usqLi4+a59NmjTRww8/rICAAA0YMECtW7fWihUrJEndu3dXy5YtZYxRYmKiunTporVr11bJuVZUzR8lDwAAgEq1JNWj6avTlJmdp6yMbF2dcUQ9Dx1SYWGhYmJivNvFxMTI4/FIkjIzM3+0rri4WAcOHDjrcaKiomSMKbdPZmamJGnlypWaMmWKtm/frtLSUp04cUK//OUvfX2qlYI75AAAALhgS1I9mrB4qzzZebKSCopLtHizR5/uLVBAQIDS09O922ZkZCgqKkqSFBkZ+aN1/v7+atq0abnQfTqPxyNrbbl9IiMjVVBQoL59++p3v/udDhw4oOzsbHXr1q3cttUZgRwAAAAXbPrqNOUVlZRrKywp1Z8/3KH+/ftr4sSJOnbsmNLT0/XnP/9ZQ4YMkSQNGjRIf/nLX7R7927l5ubqscce04ABA+Tv76/w8HDVqVNHu3btKtfv999/r5kzZ6qoqEhvvvmm/v3vf6tbt24qLCxUQUGBwsPD5e/vr5UrV+r999+vsmtQUQxZAQAAwAXLzM47a/vy55/XQw89pLi4OF1yySUaNWqURo4cKUkaOXKkMjMz1bFjR+Xn5yspKUnPP/+8JKlu3bqaOHGi2rVrp6KiIq1atUqS1LZtW/3nP/9RWFiYmjZtqrfeekuNGzeWJM2cOVP9+/dXQUGBevbsqV69elXB2fuGcX0rPyEhwaakpDitAQAAABem3dMfy3OGUB4VGqTPxt/moKIzM8ZsstYmuK7jTBiyAgAAgAs2Lqm1ggL8yrUFBfhpXFJrRxVdfBiyAgAAgAvWO77sIc1Ts6xEhgZpXFJrbzvOjUAOAACACukdH0UArwCGrAAAAAAOEcgBAAAAhwjkNcRP/cwsAAAAqi8CeTW3efNmxcfHKzg4WP369dOAAQM0adIkrVmzRs2bN9czzzyjZs2aacSIESotLdXTTz+tli1bqnHjxurfv7+ysrK8fW3YsEG33HKLQkNDde2112rNmjXedZ06ddIf/vAHtWvXTsHBwerSpYsOHTrk4IwBAABqlwoHcmNMoDHmH8aYdGPMMWNMqjGmqy+Kq+0KCwt15513avjw4crKytKgQYP0zjvveNfv379fWVlZSk9P18svv6yZM2dqyZIlSk5OVmZmpho2bKgHH3xQUtlPzXbv3l2TJk1SVlaWnnvuOfXt21cHDx709rdw4ULNmTNH33//vQoLC/Xcc89V+TkDAADUNr64Q+4vaa+kREkhkv4g6Q1jTKwP+q7VNmzYoOLiYo0ZM0YBAQHq06ePbrzxRu/6OnXqaMqUKQoMDFRQUJBeeukl/elPf1Lz5s0VGBioyZMn66233lJxcbHmz5+vbt26qVu3bqpTp45uv/12JSQk6L333vP2N2LECF1++eUKCgpS//79tWXLFhenDQAAUKtUeNpDa+1xSZNPa1pujNkt6QZJeyraf22zJNXjncfzF+nrVbdhuIwx3vXR0dHe9+Hh4brkkku8y+np6brzzjtVp87//T/Lz89PBw4cUHp6ut58800tW7bMu66oqEi33nqrd7lZs2be93Xr1lVubq7Pzw8AAADl+XwMuTGmqaTLJX3zE9v82hiTYoxJOX3IRG23JNWjCYu3ypOdJyspx9TXnozv9M7m77zb7N271/v+9KAulYX1lStXKjs72/vKz89XVFSUoqOjNXTo0HLrjh8/rvHjx1fV6QEAAOAMfBrIjTEBkhZIesVau+1s21lrX7bWJlhrE8LDw31ZwkVt+uo05RWVeJcDo66QVR2NnfyMiouLtXTpUn3++edn3f+BBx7QxIkTlZ6eLkk6ePCgli5dqsmTJ+uLL77QsmXLtHr1apWUlCg/P19r1qzRd999d9b+AAAAUPnOGciNMWuMMfYsr09P266OpHmSCiWNrsSaa6zM7Lxyy8YvQOF3PqbvNq5QaGio5s+frx49eigwMPCM+//2t79Vr1691KVLFwUHB+umm27Sxo0bJZUNQVm6dKmeeuophYeHKzo6WtOnT1dpaWmlnxcAAADOzlhrK95J2diJf0qKldTNWpv303v8n4SEBJuSklLhGmqCdk9/LE/2jy9dVGiQPht/mySpbdu2GjVqlO67776f3e/kyZO1Y8cOzZ8/32e1AgAAXEyMMZustQmu6zgTXw1Z+V9JV0rqeT5hHOWNS2qtoAC/cm028xvdd0NDxcbGqn///vriiy/04IMP6tNPPz3rnOK7d+9WYmKigoODdfvttzOfOAAAQDVW4VlWjDExku6XVCBp/2kPGt5vrV1Q0f5rk97xUZLknWUlMjRI1/pbjR9yhw4ePKgDBw7on//8p26//XZdc801mjdvnu644w599NFH6tu3r7Zt26bw8HANHjxYN998s95//31t3LhR3bt3169+9SvHZwcAAIAz8cmQlYpgyMrPExsbq8cff1wjR47UM888o6+//lrz5s3zrk9KStLgwYN16623Ki4uTjk5OapXr54kafDgwapTpw5DVgAAQK1VnYesVPgOOSrP6XOS78/JV0ZBkCT95Jzip36h81QYl6SYmJhy0yUCAACg+iCQV1On5iQ/NQ1icanVPz/do+tu8njnFJ81a9aP9ktPT9eRI0d0/PhxbyjPyMj40ZzlAAAAqB58/sNA8I0fzkkuSQUlpZq+Ok1Dhgw565ziMTExSkhI0BNPPKHCwkJ9+umn5e6kAwAAoHohkFdTP5yT/PT26Ojon5xTfOHChdq4caMaNWqkKVOmaNiwYVVZOgAAAM4DD3VWUz9nTnIAAAD8PNX5oU7ukFdTZ5qTPCjAT+OSWjuqCAAAAJWBhzqrqTPNST4uqbW3HQAAADUDgbwa6x0fRQAHAACo4RiyAgAAADhEIAcAAAAcIpADAAAADhHIAQAAAIcI5AAAAIBDBHIAAADAIQI5AAAA4BCBHAAAAHCIQH4B5s6dq/bt23uXjTHasWOHw4oAAABwsSKQAwAAAA4RyAEAAACHanUgnz59uvr27Vuu7aGHHtLDDz+snJwc3XvvvYqIiFBUVJQmTZqkkpKSc/aZk5OjYcOGKTw8XDExMZo6dapKS0slSTExMdq0aZMkaf78+TLG6Ntvv5UkzZ49W7179/bxGQIAAKC6q9WBfMiQIVq1apWys7MlScXFxVq0aJGGDh2qe+65R/7+/tqxY4dSU1P1/vvva/bs2efs86GHHlJOTo527dql5ORkvfrqq5ozZ44kKTExUWvWrJEkffLJJ4qLi1NycrJ3OTExsXJOFAAAANVWrQ7kERER6tixo958801J0qpVqxQWFqbmzZtr5cqV+utf/6p69eqpSZMmeuSRR/T666//ZH8lJSVatGiRpk2bpuDgYMXGxmrs2LGaN2+epLJAfiqAr127VhMmTPAuJycnE8gBAABqIX/XBVS1JakeTV+dpszsPEWGBummxJ6aP3++Ro0apfnz52vo0KFKT09XUVGRIiIivPuVlpYqOjr6J/s+dOiQCgsLFRMT422LiYmRx+ORVBbIf/e732n//v0qKSnRgAEDNGXKFO3Zs0c5OTm67rrrKuekAQAAUG3VqkC+JNWjCYu3Kq+obCy4JztPy01zeVK/1Ndff63ly5fr2WefVUBAgAIDA3Xo0CH5+//8SxQWFqaAgAClp6frqquukiRlZGQoKipKktSqVSvVrVtXM2fOVMeOHRUcHKxmzZrp5ZdfVvv27VWnTq3+gwUAAECtVKsS4PTVad4wfkqB9VO9K9pp8ODBuvHGG3XppZcqIiJCXbp00dixY3X06FGVlpZq586d3uElZ+Pn56f+/ftr4sSJOnbsmNLT0/XnP/9ZQ4YM8W6TmJioF154wTs8pVOnTuWWAQAAULvUqkCemZ13xvY6lydq69atGjp0qLft1VdfVWFhoa666io1bNhQd911l/bt23fOYzz//POqV6+e4uLi1L59ew0ePFgjR470rk9MTNSxY8fUsWPHMy4DAACgdjHWWqcFJCQk2JSUlCo5VrunP5bnDKE8TEf17cx7tX//fjVo0KBKagEAAEDVMcZsstYmuK7jTGrVHfJxSa0VFOBXru0Sf6OIjI80cOBAwjgAAACqXK16qLN3fNnDladmWWlaV/ry6X7yaxGrv69a5bY4AAAA1Eq1asgKAAAAaieGrAAAAAA4IwI5AAAA4BCBHAAAAHCIQA4AAAA4RCAHAAAAHCKQAwAAAA4RyAEAAACHCOQAAACAQwRyAAAAwCECOQAAAOCQsda6LcCYg5LSnRZRtcIkHXJdRA3HNa4aXOfKxzWufFzjysc1rnxc458nxlob7rqIM3EeyGsbY0yKtTbBdR01Gde4anCdKx/XuPJxjSsf17jycY0vfgxZAQAAABwikAMAAAAOEcir3suuC6gFuMZVg+tc+bjGlY9rXPm4xpWPa3yRYww5AAAA4BB3yAEAAACHCOQAAACAQwRyB4wxgcaYfxhj0o0xx4wxqcaYrq7rqmmMMaONMSnGmAJjzFzX9dQUxphGxph3jDHHT36GB7uuqSbhc1v5+A6uGsaY+caYfcaYo8aY7caY+1zXVFMZYy4zxuQbY+a7rgUXxt91AbWUv6S9khIlZUjqJukNY8wvrbV7XBZWw2RKmiopSVKQ41pqkr9JKpTUVNJ1klYYY7601n7jtqwag89t5eM7uGpMk3SvtbbAGHOFpDXGmFRr7SbXhdVAf5P0hesicOG4Q+6Atfa4tXaytXaPtbbUWrtc0m5JN7iurSax1i621i6RdNh1LTWFMaaepL6S/mCtzbXWfirpXUlD3VZWc/C5rXx8B1cNa+031tqCU4snXy0dllQjGWMGSsqW9JHrWnDhCOTVgDGmqaTLJXGHEdXd5ZJKrLXbT2v7UtLVjuoBKozv4MpjjPm7MeaEpG2S9kl6z3FJNYoxpoGkJyWNdV0LKoZA7pgxJkDSAkmvWGu3ua4HOIf6knJ+0JYjKdhBLUCF8R1cuay1v1HZ90MHSYslFfz0HjhPf5T0D2vtXteFoGII5JXAGLPGGGPP8vr0tO3qSJqnsvG4o50VfBH6udcYPpcrqcEP2hpIOuagFqBC+A6uGtbakpPD25pL+m/X9dQUxpjrJHWW9BfXtaDieKizElhrO51rG2OMkfQPlT0Y181aW1TZddUkP+cao1Jsl+RvjLnMWvufk23Xij/14yLDd7AT/mIMuS91khQrKaPs46z6kvyMMVdZa693WBcuAHfI3flfSVdK6mmtzXNdTE1kjPE3xlwiyU9lX1KXGGP4T2gFWGuPq+zPzk8aY+oZY9pJ+pXK7jLCB/jcVhm+gyuRMaaJMWagMaa+McbPGJMkaZCkj13XVoO8rLL/4Fx38vWipBUqm6EJFxkCuQPGmBhJ96vsH9B+Y0zuydfdjkuraSZJypM0XtKQk+8nOa2oZviNyqbj+17Sa5L+mykPfYrPbSXjO7hKWJUNT/lO0hFJz0l62Fq71GlVNYi19oS1dv+pl8qGFOZbaw+6rg3nz1hrXdcAAAAA1FrcIQcAAAAcIpADAAAADhHIAQAAAIcI5AAAAIBDBHIAAADAIQI5AAAA4BCBHAAAAHCIQA4AAAA4RCAHAAAAHPp/tivh0dFlDeAAAAAASUVORK5CYII=\n",
+ "text/plain": [
+ ""
+ ]
+ },
+ "metadata": {
+ "needs_background": "light"
+ },
+ "output_type": "display_data"
+ }
+ ],
+ "source": [
+ "plot_words(embedding, sequences, 1)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "## 3 - Semantic and positional embedding\n",
+ "\n",
+ "Next, you will combine the original glove embedding with the positional encoding you calculated earlier. For this exercise, you will use a 1 to 1 weight ratio between the semantic and the positional embedding. "
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 18,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "image/png": "iVBORw0KGgoAAAANSUhEUgAAAssAAAFoCAYAAAChaLYGAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO3deXgUVb7G8feQxBAg0EAChBATdpBHJRpEAQk6SGQREWQVEPQ64AyiDnIvCM6AOoAyzMx1mauiAy444IIgIIKKICiigUS9KuGydSBhC6FDAtlz7h/RHqIUgnSnSfh+nqefJ3Wq6tSvjrH7pXK6ylhrBQAAAODnagS6AAAAAOBCRVgGAAAAHBCWAQAAAAeEZQAAAMABYRkAAABwQFgGAAAAHAQHuoCIiAgbFxcX6DIAAABQjW3dujXLWht5rvsFPCzHxcUpOTk50GUAAACgGjPGuH/NfkzDAAAAABwQlgEAAAAHhGUAAADAAWEZkqSFCxeqW7du3mVjjHbu3BnAigAAAAKPsAwAAAA4ICwDAAAADgjL1czcuXM1aNCgCm333XefHnjgAeXk5Ojuu+9WVFSUoqOjNX36dJWWlv5inzk5ORo9erQiIyMVGxurxx9/XGVlZZKk2NhYbd26VZL02muvyRij7777TpL04osvasCAAT4+QwAAgMpDWK5mRo4cqffff18ej0eSVFJSoiVLlmjUqFG68847FRwcrJ07dyolJUVr167Viy+++It93nfffcrJydHu3bu1YcMGvfLKK1qwYIEkKTExUevXr5ckffLJJ2rRooU2bNjgXU5MTPTPiQIAAFQCwnI1ExUVpe7du+vNN9+UJL3//vuKiIhQs2bNtHr1av39739X7dq11ahRIz344INavHjxGfsrLS3VkiVLNHv2bIWHhysuLk6TJk3Sq6++Kqk8LP8Yjjdu3KipU6d6lzds2EBYBgAAVRphuZpYlpKhrnPWqfmUVdrtulp/f+4lSeVTI0aNGiW3263i4mJFRUXJ5XLJ5XJp3LhxOnz48Bn7zcrKUlFRkWJjY71tsbGxysjIkFQeljdu3KiDBw+qtLRUQ4cO1aeffqq9e/cqJydHHTt29N9JAwAA+FnAH3eN87csJUNTl36j/OLy+ccF0Vdr/7L/1n+/+ZFWrlypJ598UiEhIQoNDVVWVpaCg8/+P3tERIRCQkLkdrt12WWXSZLS09MVHR0tSWrVqpVq1aqlp556St27d1d4eLiaNGmiF154Qd26dVONGvx7DAAAVF0kmWpg7po0b1CWJBN8iWq16aKHJ96ja665RpdeeqmioqLUq1cvTZo0ScePH1dZWZl27drlnTLhJCgoSEOGDNG0adOUm5srt9utv/71rxo5cqR3m8TERD3zzDPeKRc9evSosAwAAFBVEZargUxP/s/aal/+G508uEejRo3ytr3yyisqKirSZZddpvr16+v222/XgQMHfrH/p59+WrVr11aLFi3UrVs3jRgxQnfddZd3fWJionJzc9W9e/fTLgMAAFRVxlob0AISEhJscnJyQGuo6rrOWaeMnwTmkuOHdeDFe3Us67Dq1q0boMoAAAAuDMaYrdbahHPdjyvL1cDkpLYKCwnyLltbppNbl6tH7wEEZQAAgPPAF/yqgQHx5V+2m7smTfsPH9P+Z0cqutmlWvjWhwGuDAAAoGpjGgYAAACqPaZhAAAAAD5GWAYAAAAcEJYBAAAAB4RlAAAAwAFhGQAAAHBAWAYAAAAcEJYBAAAAB4RlAAAAwAFhGQAAAHBAWAYAAAAcEJYBAAAAB4RlAAAAwAFhGQAAAHBAWAYAAAAcEJYBAAAAB34Jy8aY1saYAmPMa/7oHwAAAKgM/rqy/KykL/3UNwAAAFApfB6WjTHDJHkkfeTrvgEAAIDK5NOwbIypK+lRSZN+YbvfGmOSjTHJR44c8WUJAAAAgM/4+sryY5JestbuO9NG1toXrLUJ1tqEyMhIH5cAAAAA+EawrzoyxnSU1FNSvK/6BAAAAALJZ2FZUg9JcZLSjTGSVEdSkDHmMmvtVT48DgAAAFApfBmWX5C0+JTlh1Qenu/14TEAAACASuOzsGytPSnp5I/Lxpg8SQXWWr7BBwAAgCrJl1eWK7DWzvBX3wAAAEBl4HHXAAAAgAPCMgAAAOCAsAwAAAA4ICwDAAAADgjLAAAAgAPCMgAAAOCAsAwAAAA4ICwDAAAADgjLAAAAgAPCMgAAAOCAsAwAAAA4ICwDAAAADgjLAAAAgAPCMgAAAOCAsAwAAAA4ICwDAAAADgjLuKCUlJQEugQAAAAvwjIqxbZt2xQfH6/w8HANHjxYQ4cO1fTp07V+/Xo1a9ZMTzzxhJo0aaKxY8eqrKxMc+bMUcuWLdWwYUMNGTJE2dnZ3r4+//xzdenSRS6XS1deeaXWr1/vXdejRw898sgj6tq1q8LDw9WrVy9lZWUF4IwBAEB1QFiG3xUVFem2227TmDFjlJ2dreHDh+udd97xrj948KCys7Pldrv1wgsv6KmnntKyZcu0YcMGZWZmqn79+vr9738vScrIyFDfvn01ffp0ZWdn6y9/+YsGDRqkI0eOePt7/fXXtWDBAh0+fFhFRUX6y1/+UunnDAAAqgfCMvzu888/V0lJiSZOnKiQkBANHDhQ11xzjXd9jRo1NHPmTIWGhiosLEzPP/+8/vznP6tZs2YKDQ3VjBkz9NZbb6mkpESvvfaa+vTpoz59+qhGjRq66aablJCQoPfee8/b39ixY9WmTRuFhYVpyJAhSk1NDcRpAwCAaiA40AWg+svMzFR0dLSMMd62mJgY78+RkZGqWbOmd9ntduu2225TjRr//rdcUFCQDh06JLfbrTfffFMrVqzwrisuLtYNN9zgXW7SpIn351q1aikvL8/n5wQAAC4OhGX4zbKUDM1dk6bdX+9VdtpuvbNtv267qpkkad++fWrZsqUkVQjRUnmQ/uc//6muXbv+rM+YmBiNGjVK8+fP9/8JAACAix7TMOAXy1IyNHXpN8rw5OuS6HYqtTU0/uE/6+0v3Vq+fLm++OILx33Hjx+vadOmye12S5KOHDmi5cuXS5JGjhypFStWaM2aNSotLVVBQYHWr1+v/fv3V8p5AQCAiwthGX4xd02a8otLJUkmKESRtz2sYylrNKRbO7322mvq16+fQkNDT7vv/fffr/79+6tXr14KDw/Xtddeqy1btkgqv7K8fPlyzZo1S5GRkYqJidHcuXNVVlZWaecGAAAuHsZaG9ACEhISbHJyckBrgO81n7JKp/vNMpL2zOmrzp07a/z48Ro7dmxllwYAAC5Cxpit1tqEc92PK8vwi6ausArLBenfqDTvmKLCL9HLL7+sr7/+WjfffHOAqgMAADg7fMEPfjE5qa2mLv3GOxWjODtDWe8+oSOlhZrXupXeeustRUVFBbhKAACAMyMswy8GxEdLKp+7nOnJV7set2ny7CnedgAAgKqAsAy/GRAfTTgGAABVGnOWAQAAAAeEZQAAAMABYRkAAABwQFgGAAAAHBCWAQAAAAeEZQAAAMABYRkAAABwQFgGAAAAHBCWAQAAAAeEZQAAAMABYRkAAABwQFgGAAAAHBCWAQAAAAeEZQAAAMABYRkAAABwQFgGAAAAHBCWq5gZM2Zo5MiRgS4DAADgokBYBgAAABwQlgOspKQk0CUAAADAAWE5AOLi4vTEE0/oiiuuUO3atbVp0yZ16dJFLpdLV155pdavX+/dds+ePUpMTFR4eLhuuukmZWVlBa5wAACAi0xwoAu4WP3rX//SqlWrVKNGDV1xxRV69dVXdfPNN+ujjz7SoEGDtH37dkVGRmrEiBG67rrrtHbtWm3ZskV9+/bVrbfeGujyAQAALgpcWQ6QiRMnKiYmRq+99pr69OmjPn36qEaNGrrpppuUkJCg9957T+np6fryyy/12GOPKTQ0VN27d9ctt9wS6NIBAAAuGlxZriTLUjI0d02aMj35OphToPTCMEmS2+3Wm2++qRUrVni3LS4u1g033KDMzEzVr19ftWvX9q6LjY3Vvn37Kr1+AACAixFhuRIsS8nQ1KXfKL+4VJJUUmb1z0171fHaDMXExGjUqFGaP3/+z/Zzu906duyYTpw44Q3M6enpMsZUav0AAAAXK6ZhVIK5a9K8QflHhaVlmrsmTSNHjtSKFSu0Zs0alZaWqqCgQOvXr9f+/fsVGxurhIQE/elPf1JRUZE2bdpU4Qo0AAAA/IuwXAkyPfmO7TExMVq+fLlmzZqlyMhIxcTEaO7cuSorK5Mkvf7669qyZYsaNGigmTNnavTo0ZVZOgAAwEXNWGsDWkBCQoJNTk4OaA3+1nXOOmWcJjBHu8L06ZQbA1ARAADAxcUYs9Vam3Cu+3FluRJMTmqrsJCgCm1hIUGanNQ2QBUBAADgbPgsLBtjQo0xLxlj3MaYXGNMijGmt6/6r8oGxEdr9sDLFe0Kk1H5FeXZAy/XgPjoQJcGAACAM/Dl3TCCJe2TlCgpXVIfSW8YYy631u714XGqpAHx0YRjAACAKsZnYdlae0LSjFOaVhpj9ki6WtJeXx0HAAAAqCx+m7NsjGksqY2kb0+z7rfGmGRjTPKRI0f8VQIAAABwXvwSlo0xIZIWSXrZWrv9p+uttS9YaxOstQmRkZH+KAEAAAA4bz4Py8aYGpJelVQkaYKv+wcAAAAqi08fd23Kn8P8kqTGkvpYa4t92T8AAABQmXwaliX9j6T2knpaa0//2DoAAACgivDlfZZjJY2T1FHSQWNM3g+vO3x1DAAAAKAy+fLWcW5Jxlf9AQAAAIHG464BAAAAB4RlAAAAwAFhGQAAAHBAWAYAAAAcEJYBAAAAB4RlAAAAwAFhGQAAAHBAWAYAAAAcEJYBAAAAB4RlAAAAwAFhGQAAAHBAWAYAAAAcEJYBAAAAB4RlAAAAwAFhGQAAAHBAWAYAAAAcEJYBAAAAB4RlAAAAwAFhGQAAAHBAWAYAAAAcEJYBAAAAB4RlAAAAwAFhGQAAAHBAWAYAAAAcEJYBAAAAB4RlAAAAwAFhGQAAAHBAWAYAAAAcEJYBAAAAB4RlAAAAwAFhGQAAAHBAWAYAAAAcEJYBAAAAB4RlAAAAVLqUlBRdddVVCg8P19ChQzVs2DBNnz5dCxcuVLdu3Spsa4zRzp07JUmFhYV66KGHdOmll6px48YaP3688vPzvduuXLlSHTt2lMvlUpcuXfT111+f2s9eY8xDxpivjTE5xpglxpiaZ6qTsAwAAIBKVVRUpAEDBmjUqFHKzs7W4MGD9fbbb5/Vvv/1X/+lHTt2KDU1VTt37lRGRoYeffRRSdK2bdt011136fnnn9fRo0c1btw49e/fX4WFhad2MUTSzZKaS7pC0pgzHY+wDAAAgEr1+eefq7i4WA888IBCQkJ0++23q1OnTr+4n7VW8+fP19/+9jc1aNBA4eHhevjhh7V48WJJ0vz58zVu3Dh17txZQUFBuvPOOxUaGqrPP//81G6estZmWmuzJa2Q1PFMxwz+9acJAAAAnJ1lKRmauyZNmZ58XeLerNr1G8kY410fGxv7i30cOXJEJ0+e1NVXX+1ts9aqtLRUkuR2u/Xyyy/r6aef9q4vKipSZmbmqd0cPOXnk5KanumYXFkGAACAXy1LydDUpd8ow5MvKynH1NHe9H16Z9t+7zbp6emSpNq1a+vkyZPe9oMH/51tIyIiFBYWpm+//VYej0cej0c5OTnKy8uTJMXExGjatGnedR6PRydPntTw4cN/de2EZQAAAPjV3DVpyi8u9S6HRreTNUH6w59mq6SkREuXLtUXX3whSbryyiv17bffKjU1VQUFBZoxY4Z3vxo1auiee+7Rgw8+qMOHD0uSMjIytGbNGknSPffco+eee05btmyRtVYnTpzQqlWrlJub+6trJywDAICLQlxcnD788MOftXfo0EHr16+v/IIuIpme/ArLJihEkbc9rMwv3lf9+vW1ZMkSDRw4UJLUpk0b/fGPf1TPnj3VunXrn90Z44knnlCrVq107bXXqm7duurZs6fS0tIkSQkJCZo/f74mTJig+vXrq1WrVlq4cOF51W6stefVwflKSEiwycnJAa0BAABUf3FxcXrxxRfVs2fPQJdy0ek6Z50yfhKYJSnaFaZPp9woSRozZoyaNWumxx9/3C81GGO2WmsTznU/riwDAADAryYntVVYSFCFtrCQIE1Oahugis4eYRkAAFx0tm/frubNm2vx4sUVpmfMmDFDQ4YM0ejRoxUeHq4OHTro1L+Ab9u2TfHx8QoPD9fgwYM1dOhQTZ8+PVCnUWUMiI/W7IGXK9oVJqPyK8qzB16uAfHRgS7tFxGWAQDARWXbtm3q1auXnn76aQ0bNuxn6999910NGzZMHo9H/fv314QJEySV34Lstttu05gxY5Sdna3hw4frnXfeqezyq6wB8dH6dMqN2jOnrz6dcuPPgvLChQv9NgXjfBCWAQDARWPjxo3q37+/Xn75ZfXr1++023Tr1k19+vRRUFCQRo0apa+++kpS+YM0SkpKNHHiRIWEhGjgwIG65pprKrN8BAAPJQEAANXWqQ/COJhToP9+5h/q9ZsbdMMNNzju06RJE+/PtWrVUkFBgUpKSpSZmano6OgKD9KIiYnxa/0IPK4sAwCAaumnD8IoKbOqc+N4ffX9Tj344IPn3F9UVJQyMjJ06p3E9u3b58OKcSEiLAMAgGrppw/CkKTioJpyDZyhTz75RFOmTDmn/q677joFBQXpmWeeUUlJiZYvX+59kAaqL6ZhAACAaumnD8L40eHCIG394APdcMMNCgkJOev+LrnkEi1dulT/8R//oalTp6p3797q16+fQkNDfVUyLkA8lAQAAFRLZ/MgjPPVuXNnjR8/XmPHjvVJf/AfHkoCAABwCn88CGPDhg06ePCgSkpK9PLLL+vrr7/WzTfffL6l4gLGNAwAAFAt/Xgf3x/vhtHUFabJSW3P60EYaWlpGjJkiPLy8tSyZUu99dZbioqK8lXJuAAxDQMAAADVHtMwAAAAAB8jLAMAAAAOCMsAAACAA8IyAAAA4MCnYdkY08AY844x5oQxxm2MGeHL/gEAAIDK5Otbxz0rqUhSY0kdJa0yxnxlrf3Wx8cBAAAA/M5nV5aNMbUlDZL0iLU2z1q7SdK7kkb56hgAAABAZfLlNIw2kkqttTtOaftKUgcfHgMAAACoNL4My3Uk5fykLUdS+E83NMb81hiTbIxJPnLkiA9LAAAAAHzHl2E5T1Ldn7TVlZT70w2ttS9YaxOstQmRkZE+LAEAAADwHV+G5R2Sgo0xrU9pu1ISX+4DAABAleSzsGytPSFpqaRHjTG1jTFdJd0q6VVfHQMAAADnJy4uTnPnztUVV1yh2rVr6+6779ahQ4fUu3dvhYeHq2fPnjp27JgkafDgwWrSpInq1aun7t2769tv/30NdMyYMfr973+vvn37Kjw8XJ07d9auXbsCdVp+4+uHkvxOUpikw5L+JelebhsHAABwYXn77bf1wQcfaMeOHVqxYoV69+6tWbNmad68eUpNTdVTTz0lSfrkk0/UpUsXHT58WFdddZXi4+OVmpqqzz77TCtXrtQ//vEP7dmzR6tWrVKrVq00bdo09ejRQ9OnT1eXLl1Up04d3XLLLTp69KjuuOMO1a1bV506ddLevXu9tdx///2KiYlR3bp1dfXVV2vjxo3edTNmzNCQIUM0evRohYeHq0OHDkpOTq7UsfJpWLbWZltrB1hra1trL7XWvu7L/gEAAHD+7rvvPjVu3FjR0dG6/vrr1blzZ8XHx6tnz546efKktm3bpgMHDqhmzZrasmWLQkNDNWbMGBUXF8vlcqlv375q3769xo4dq0ceeUS33nqrbrnlFqWmpkqSFi9erFdffVUZGRnatWuXrrvuOo0dO1bZ2dlq3769Zs6c6a2lU6dOSk1NVXZ2tkaMGKHBgweroKDAu/7dd9/VsGHD5PF41L9/f02YMKFSx4rHXQMAAFRzy1Iy1HXOOjWfskoHcwq0MzfIuy4sLEyNGzeWJLVo0UI1a9bUgQMH9PHHH6t+/fo6evSoateurS5dukiS3nrrLbVu3VotW7ZU06ZNNXz4cLVr106pqanKy8uTJI0dO1YtW7ZUvXr11Lt3b7Vs2VI9e/ZUcHCwBg8erJSUFO/xR44cqYYNGyo4OFiTJk1SYWGh0tLSvOu7deumPn36KCgoSKNGjdJXX31VGUPmRVgGAACoxpalZGjq0m+U4cmXlVRSZvXSxj1alpIhSdqXfVIvbdqj5lNWqeucdYqIjpPH49FLL72kw4cP684779S8efM0YMAASdKhQ4cUGxtb4RixsbHKysryLv8YvqWKYfzH5R9DtSTNmzdP7du3V7169eRyuZSTk1OhryZNmnh/rlWrlgoKClRSUuKbwTkLhGUAAIBqbO6aNOUXl1ZoKywt09w1aVqWkqHUfR7lFhTLSsrw5OuQaaCDR45q+/btcrlc6tmzp9atW6eVK1dKKg/Cbre7Qn/p6emKiIg459o2btyoJ554Qm+88YaOHTsmj8ejevXqyVr7q8/X1wjLAAAA1VimJ9+xfe6aNJWWVQymxhWl3ByPatasqdatW2vs2LF6++23VVxcLElKTEzUjh07tHv3bpWVlWnJkiX67rvvdN11151zbbm5uQoODlZkZKRKSkr06KOP6vjx4+d+kn5EWAYAAKjGmrrCKiw3u/efCovrqKauMGV68hVxy0NydbvDu97VZZiCapXfKm758uXKy8tTfHy8evToIWutOnXqpJUrV+rEiRN69tln9eSTT2rlypW69dZbtX///nOqLSkpSb1791abNm0UGxurmjVrKiYmxifn7Ssm0Je5ExISbGXfAgQAAOBi8eOc5VOnYoSFBGn2wMs1d02aMk5z5TnaFaZPp9xYmWX6nTFmq7U24Vz3C/ZHMQAAALgwDIiPllQ+dznTk6+mrjBNTmrrbT9dkJ6c1DYgtV6ICMsAAADV3ID4aG84/mm75BykQVgGAAC4qDkFaZTjC34AAACAA8IyAAAA4ICwDAAAADggLAMAAAAOCMsAAACAA8IyAAAA4ICwDAAAADggLAMAAAAOCMsAAACAA8IyAAAA4ICwDACAD4wZM0bTp08PdBkAfIywDAAAADggLAMAAAAOCMsAAPwKKSkpuuqqqxQeHq6hQ4eqoKDAu27+/Plq1aqVGjRooP79+yszM9O7bu3atWrbtq3q1aun3/3ud0pMTNSLL74YiFMAcBYIywAAnKOioiINGDBAo0aNUnZ2tgYPHqy3335bkrRu3TpNnTpVb7zxhg4cOKDY2FgNGzZMkpSVlaXbb79ds2fP1tGjR9W2bVt99tlngTwVAL+AsAwAwDn6/PPPVVxcrAceeEAhISG6/fbb1alTJ0nSokWLdNddd+mqq65SaGioZs+erc2bN2vv3r1677331KFDBw0cOFDBwcGaOHGimjRpEuCzAXAmhGUAAM5RZmamoqOjZYzxtsXGxnrX/fizJNWpU0cNGzZURkaGMjMzFRMT411njFGzZs0qr3AA54ywDADAWVqWkqGuc9bpwXf36qu0PXpn237vuvT0dElS06ZN5Xa7ve0nTpzQ0aNHFR0draioKO3f/+99rLUVlgFceAjLAACchWUpGZq69BtlePJ1SXQ7lamGxk99XG9/6dbSpUv1xRdfSJJGjBihBQsWKDU1VYWFhXr44YfVuXNnxcXFqW/fvvrmm2+0bNkylZSU6Nlnn9XBgwcDfGYAzoSwDADAWZi7Jk35xaWSJBMUosjbHtax1A80pFt7LVmyRAMHDpQk/eY3v9Fjjz2mQYMGKSoqSrt27dLixYslSREREXrzzTf1n//5n2rYsKG+++47JSQkKDQ0NGDnBeDMjLU2oAUkJCTY5OTkgNYAAMAvaT5llU73iWkk7ZnT91f1WVZWpmbNmmnRokW64YYbzqs+AGdmjNlqrU041/24sgwAwFlo6go7p3Yna9askcfjUWFhoWbNmiVrra699lpflAjADwjLAACchclJbRUWElShLSwkSJOT2p5TP5s3b1bLli0VERGhFStWaNmyZQoLO7fADaDyEJYB4ALSo0ePCk9zmz59uiIiIrgX7wVgQHy0Zg+8XNGuMBlJ0a4wzR54uQbER59TPzNmzNDRo0eVm5urLVu2qHPnzv4pGIBPBAe6AADA6e3bt0/z5s2T2+1Wo0aNAl0OVB6YzzUcA6jauLIMABcot9uthg0bEpQBIIAIywDgIwsWLNAtt9ziXW7VqpWGDBniXY6JiVFqaqo+++wzderUSfXq1VOnTp302Wef/ayvDz/8UDfddJMyMzNVp04djRkzpjJOAQDwE4RlAPCRxMREbdy4UWVlZTpw4ICKi4v16aefSpJ2796tvLw8XXrpperbt68mTpyoo0eP6g9/+IP69u2ro0ePVuirZ8+eWr16tZo2baq8vDwtXLgwAGcEACAsA4CPtGjRQuHh4UpNTdWGDRuUlJSk6Ohobd++XRs2bND111+vVatWqXXr1ho1apSCg4M1fPhwtWvXTitWrAh0+QCA0+ALfgDgQ4mJiVq/fr127typxMREuVwubdiwQZs3b1ZiYqIyMzMVGxtbYZ/Y2FhlZGQEqGIAwJkQlgHgPC1LydDcNWnK9OQr6GRj7Vj+vvKzD+jhhx+Wy+XSokWLtHnzZk2YMEHff/+93G53hf3T09N18803B6h6AMCZMA0DAM7DspQMTV36jTI8+bKS8iPaauvnm5TlyVWzZs10/fXX6/3339fRo0cVHx+vPn36aMeOHXr99ddVUlKiJUuW6LvvvlO/fv0CfSoAgNPgyjIAnIe5a9KUX1zqXQ5pEC0TUlOljdtJkurWrasWLVooMjJSQUFBatiwoVauXKn7779f9957r1q1aqWVK1cqIiIiUKcAADgDY60NaAEJCQk2OTk5oDUAwK/VfMoqne5d1EjaM6dvZZcDAHBgjNlqrU041/2YhgEA56GpK+yc2gEAVQthGQDOw+SktgoLCarQFhYSpMlJbQNUEQDAl5izDADnYUB8tCR574bR1BWmyUltve0AgKqNsAwA52lAfNKe5PcAABEHSURBVDThGACqKaZhAAAAAA4IywAAAIADwjIAAADggLAMAAAAOCAsAwAAAA4IywAAAIADwjIAAADggLAMAAAAOCAsAwAAAA4IywAAAIADwjIAAADg4LzDsjEm1BjzkjHGbYzJNcakGGN6+6I4AAAAIJB8cWU5WNI+SYmS6kl6RNIbxpg4H/QNAAAABEzw+XZgrT0hacYpTSuNMXskXS1p7/n2DwAAAASKz+csG2MaS2oj6dszbPNbY0yyMSb5yJEjvi4BAAAA8AmfhmVjTIikRZJettZud9rOWvuCtTbBWpsQGRnpyxIAAAAAn/nFsGyMWW+MsQ6vTadsV0PSq5KKJE3wY80AAABApfjFOcvW2h6/tI0xxkh6SVJjSX2stcXnXxoAAAAQWOf9Bb8f/I+k9pJ6WmvzfdQnAAAAEFC+uM9yrKRxkjpKOmiMyfvhdcd5VwcAAAAEkC9uHeeWZHxQCwAAAHBB4XHXAAAAgAPCMgAAAOCAsAwAAAA4ICwDAAAADgjLAAAAgAPCMgAAAOCAsAwAAAA4ICwDAAAADgjLAAAAgAPCMgAAAOCAsAwAAAA4ICwDAAAADgjLAAAAgAPCMgAAAOCAsAwAAAA4ICwDAAAADgjLAAAAgAPCMgAAAOCAsAwAAAA4ICwDAAAADgjLAAAAgAPCMgAAAOCAsAwAAAA4ICwDAAAADgjLAAAAgAPCMgAAAOCAsAwAAAA4ICwDAAAADgjLAAAAgAPCMgAAAOCAsAwAAAA4ICwDAAAADgjLAAAAgAPCMgAAAOCAsAwAAAA4ICwDAAAADgjLAAAAgAPCMgAAAOCAsAwAAAA4ICwDAAAADgjLAAAAgAPCMgAAAOCAsAwAAAA4ICw7iIuL04cffuiz/hYuXKhu3br5pK8xY8Zo+vTpv2rfGTNmaOTIkZKkvXv3yhijkpISn9QFAABQ3RCWL3C+DNkAAAA4N4RlAAAAwAFh+Qy+/PJLXXbZZapfv77Gjh2rgoICHTt2TP369VNkZKTq16+vfv36af/+/d59Fi5cqBYtWig8PFzNmzfXokWLTtv35MmT1a1bN+Xk5CgnJ0d33323oqKiFB0drenTp6u0tFTff/+9xo8fr82bN6tOnTpyuVze/bOysnTTTTcpPDxciYmJcrvd3nX333+/YmJiVLduXV199dXauHGj/wYJAACgGiMsn8GiRYu0Zs0a7dq1Szt27NDjjz+usrIyjR07Vm63W+np6QoLC9OECRMkSSdOnNDEiRO1evVq5ebm6rPPPlPHjh0r9FlWVqZ77rlHX3/9tdauXat69erpzjvvVHBwsHbu3KmUlBStXbtWL774otq3b6/nnntO1113nfLy8uTxeCrU9sgjjygrK0sdO3bUHXfc4V3XqVMnpaamKjs7WyNGjNDgwYNVUFBQOYMGAABQjRCWz2DChAmKiYlRgwYNNG3aNP3rX/9Sw4YNNWjQINWqVUvh4eGaNm2aNmzY4N2nRo0a+t///V/l5+crKipKHTp08K4rLi7W8OHDlZ2drRUrVqhWrVo6dOiQVq9erb///e+qXbu2GjVqpAcffFCLFy8+Y219+/ZV9+7dFRoaqj//+c/avHmz9u3bJ0kaOXKkGjZsqODgYE2aNEmFhYVKS0vzzyABAABUY4TlM4iJifH+HBsbq8zMTJ08eVLjxo1TbGys6tatq+7du8vj8ai0tFS1a9fWkiVL9NxzzykqKkp9+/bV9u3bvX3s3LlTy5cv15/+9CddcsklkiS3263i4mJFRUXJ5XLJ5XJp3LhxOnz48FnXVqdOHTVo0ECZmZmSpHnz5ql9+/aqV6+eXC6XcnJylJWV5cuhAQAAuCgQln+wLCVDXeesU/Mpq9R1zjqdLCr1XqmVpPT0dDVt2lTz5s1TWlqatmzZouPHj+uTTz6RJFlrJUlJSUn64IMPdODAAbVr10733HOPt4/27dtrwYIF6t27t/dKb0xMjEJDQ5WVlSWPxyOPx6Pjx4/r22+/lSQZY05b76m15eXlKTs7W02bNtXGjRv1xBNP6I033tCxY8fk8XhUr149b30AAAA4e4RllQflqUu/UYYnX1ZShidfx04Wafa8v2v//v3Kzs7WrFmzNHToUOXm5iosLEwul0vZ2dmaOXOmt59Dhw7p3Xff1YkTJxQaGqo6deooKCiowrGGDx+uWbNmqWfPntq1a5eioqLUq1cvTZo0ScePH1dZWZl27drlndrRuHFj7d+/X0VFRRX6ee+997Rp0yYVFRXpkUceUefOnRUTE6Pc3FwFBwcrMjJSJSUlevTRR3X8+HG/jyEAAEB1RFiWNHdNmvKLSyu0WSupZTf16tVLLVq0UIsWLTR9+nQ98MADys/PV0REhK699lrdfPPN3n3Kyso0b948NW3aVA0aNNCGDRv0j3/842fHu/POO/XHP/5RN954o/bu3atXXnlFRUVF3jtv3H777Tpw4IAk6cYbb1SHDh3UpEkTRUREePsYMWKEZs6cqQYNGmjr1q3eu24kJSWpd+/eatOmjWJjY1WzZs0KUzYAAABw9kyg/zyfkJBgk5OTA1pD8ymrdLpRMJL2zOlb2eUAAADAx4wxW621Cee6H1eWJTV1hZ1TOwAAAC4OhGVJk5PaKiyk4tzisJAgTU5qG6CKAAAAcCG4qMNyWlqa4uPjNap7O3Ut/ELRrjAZSdGuMM0eeLkGxEf/qn4XLlyobt26/ap9169fr2bNmnmX4+Li9OGHH/6qvgAAAHB+gn3ZmTGmtaRvJL1lrR3py7794cknn1SPHj2UkpLyq/vYu3evmjdvruLiYgUH+3Q4AQAAEGC+vrL8rKQvfdyn37jd7gpP2AMAAABO5bOwbIwZJskj6SNf9elPN954oz7++GNNmDBBderU0VdffaXRo0crMjJSsbGxevzxx1VWViap/JZwjz/+uGJjY9WoUSONHj1aOTk5kqTu3btLklwul+rUqaPNmzdLKn9IyX333ad69eqpXbt2+uijfw/LggUL1L59e4WHh6tFixZ6/vnnK/nsAQAAcDZ8EpaNMXUlPSppki/6qwzr1q3T9ddfr2eeeUZ5eXmaN2+ecnJytHv3bm3YsEGvvPKKFixYIKl8DvLChQv18ccfa/fu3crLy9OECRMkyfsEP4/Ho7y8PF133XWSpC1btqhFixbKysrSzJkzNXDgQGVnZ0uSGjVqpJUrV+r48eNasGCBHnzwQW3bti0AowAAAIAz8dWV5cckvWSt3feLW0oyxvzWGJNsjEk+cuSIj0r49UpLS7VkyRLNnj1b4eHhiouL06RJk/Tqq69KkhYtWqQ//OEPatGiherUqaPZs2dr8eLFKikpceyzUaNGeuCBBxQSEqKhQ4eqbdu2WrVqlSSpb9++atmypYwxSkxMVK9evbRx48ZKOVcAAACcvV8My8aY9cYY6/DaZIzpKKmnpL+d7UGttS9YaxOstQmRkZHnU/85W5aSoa5z1qn5lFVKSfcoJf2YsrKyVFRUpNjYWO92sbGxysjIkCRlZmb+bF1JSYkOHTrkeJzo6GgZYyrsk5mZKUlavXq1rr32WjVo0EAul0vvvfeesrKyfH2qAAAAOE+/GJattT2stcbh1U1SD0lxktKNMQclPSRpkDHmgptXsCwlQ1OXfqMMT76spMKSUi3dlqFN+woVEhIit9vt3TY9PV3R0eW3jmvatOnP1gUHB6tx48YVAvGpMjIydOrTEdPT09W0aVMVFhZq0KBBeuihh3To0CF5PB716dNHgX6SIgAAAH7OF9MwXpDUUlLHH17PSVolKckHffvU3DVpyi8urdBWVFqmv364U0OGDNG0adOUm5srt9utv/71rxo5svzud8OHD9ff/vY37dmzR3l5eXr44Yc1dOhQBQcHKzIyUjVq1NDu3bsr9Hv48GE99dRTKi4u1ptvvqnvv/9effr0UVFRkQoLCxUZGang4GCtXr1aa9eurbQxAAAAwNk77xsDW2tPSjr547IxJk9SgbU28JORfyLTk+/YvvLpp3XfffepRYsWqlmzpu655x7dddddkqS77rpLmZmZ6t69uwoKCpSUlKSnn35aklSrVi1NmzZNXbt2VXFxsd5//31JUufOnfV///d/ioiIUOPGjfXWW2+pYcOGkqSnnnpKQ4YMUWFhoW655Rb179+/Es4eAAAA58oE+s//CQkJNjk5uVKO1XXOOmWcJjBHu8L06ZQbK6UGAAAAVD5jzFZrbcK57ndRPe56clJbhYUEVWgLCwnS5KS2AaoIAAAAF7KL6vnMA+LLv7A3d02aMj35auoK0+Sktt52AAAA4FQXVViWygMz4RgAAABn46KahgEAAACcC8IyAAAA4ICwDAAAADggLAMAAAAOCMsAAACAA8IyAAAA4ICwDAAAADggLAMAAAAOCMsAAACAA8IyAAAA4MBYawNbgDFHJLkDWkTliJCUFegiqiHG1T8YV99jTP2DcfUPxtX3GFP/OJdxjbXWRp7rAQIeli8Wxphka21CoOuobhhX/2BcfY8x9Q/G1T8YV99jTP2jMsaVaRgAAACAA8IyAAAA4ICwXHleCHQB1RTj6h+Mq+8xpv7BuPoH4+p7jKl/+H1cmbMMAAAAOODKMgAAAOCAsAwAAAA4ICxXMmPMa8aYA8aY48aYHcaY/wh0TVWZMSbUGPOSMcZtjMk1xqQYY3oHuq7qwBgzwRiTbIwpNMYsDHQ9VZUxpoEx5h1jzIkffk9HBLqmqo7fTf/g/dQ/+Nz3L2NMa2NMgTHmNX8dI9hfHcPRbEl3W2sLjTHtJK03xqRYa7cGurAqKljSPkmJktIl9ZH0hjHmcmvt3kAWVg1kSnpcUpKksADXUpU9K6lIUmNJHSWtMsZ8Za39NrBlVWn8bvoH76f+wee+fz0r6Ut/HoAry5XMWvuttbbwx8UfXi0DWFKVZq09Ya2dYa3da60ts9aulLRH0tWBrq2qs9YutdYuk3Q00LVUVcaY2pIGSXrEWptnrd0k6V1JowJbWdXG76Z/8H7qH3zu+48xZpgkj6SP/HkcwnIAGGP+YYw5KWm7pAOS3gtwSdWGMaaxpDaSuGqHC0EbSaXW2h2ntH0lqUOA6gHOGu+nvsPnvu8ZY+pKelTSJH8fi7AcANba30kKl3S9pKWSCs+8B86GMSZE0iJJL1trtwe6HkBSHUk5P2nLUfn//8AFi/dT3+Jz3y8ek/SStXafvw9EWPYhY8x6Y4x1eG06dVtrbekPf5JtJunewFR84TvbMTXG1JD0qsrnhk4IWMFVxLn8ruK85Emq+5O2upJyA1ALcFZ4P/UPPvd9xxjTUVJPSX+rjOPxBT8fstb2+BW7BYu5S47OZkyNMUbSSyr/AlUfa22xv+uq6n7l7yrO3Q5JwcaY1tba//uh7UrxZ21coHg/rRR87p+/HpLiJKWX/8qqjqQgY8xl1tqrfH0wrixXImNMI2PMMGNMHWNMkDEmSdJwSesCXVsV9z+S2ku6xVqbH+hiqgtjTLAxpqakIJW/CdU0xvAP7HNgrT2h8j+5PmqMqW2M6SrpVpVftcOvxO+mX/F+6kN87vvNCyr/B0fHH17PSVql8jvk+BxhuXJZlf/pZb+kY5L+IukBa+3ygFZVhRljYiWNU/n/LAeNMXk/vO4IcGnVwXRJ+ZKmSBr5w8/TA1pR1fQ7ld/e7LCkf0m6l9vGnTd+N/2A91O/4HPfD6y1J621B398qXzKW4G19og/jmestf7oFwAAAKjyuLIMAAAAOCAsAwAAAA4IywAAAIADwjIAAADggLAMAAAAOCAsAwAAAA4IywAAAIADwjIAAADggLAMAAAAOPh/tOwhkh8Al3kAAAAASUVORK5CYII=\n",
+ "text/plain": [
+ ""
+ ]
+ },
+ "metadata": {
+ "needs_background": "light"
+ },
+ "output_type": "display_data"
+ },
+ {
+ "data": {
+ "image/png": "iVBORw0KGgoAAAANSUhEUgAAAsQAAAFoCAYAAABQY+2LAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO3deXRV5fn28eshiTHAIYEQIAkhISLjWyUaRJmCFI0MUkCZh4LWGXEqv6JghZaCirRd2MECFlRQRIsgIIJKCSA4BEGsVlCGBBKGDCQkGMj0vH8ApwSZDDs5Cfv7WeuslT3d+95nnXW83Dzn2cZaKwAAAMCtavi6AQAAAMCXCMQAAABwNQIxAAAAXI1ADAAAAFcjEAMAAMDVCMQAAABwNX9fN1C/fn0bExPj6zYAAABwGdu8eXOmtTbsbNt8HohjYmKUnJzs6zYAAABwGTPGpJxrG0MmAAAA4GqOBmJjzFpjzDFjTP7J13Yn6wMAAABOq4g7xGOstbVPvlpUQH0AAADAMQyZAHBZi4mJ0YcffuhYvXnz5qlTp06O1Bo1apQmTpxYrmMnTZqk4cOHS5L27NkjY4yKi4sd6QsA3KYiAvE0Y0ymMeZjY0zXCqgPANWOk0EaAOAspwPxbyTFSoqUNEvSMmPMVWfuZIy51xiTbIxJzsjIcLgFAAAA4OI5GoittZ9aa/Ostcetta9I+lhSz7PsN8taG2+tjQ8LO+t0cADgmM8//1ytW7dW3bp1NXr0aB07dkyHDx9W7969FRYWprp166p3797at2+f95h58+YpNjZWHo9HTZs21YIFC85ae9y4cerUqZNyc3OVm5uru+++W+Hh4YqMjNTEiRNVUlKi//73v7r//vu1adMm1a5dWyEhId7jMzMzdcstt8jj8SghIUEpKf+bFeiRRx5RVFSU6tSpo+uvv17r16+vuDcJAFysoscQW0mmgs8BAOe1YMECrVq1Sjt37tSOHTs0ZcoUlZaWavTo0UpJSVFqaqqCgoI0ZswYSdLRo0c1duxYrVy5Unl5edq4caPatm1bpmZpaanuuecebdu2TatXr1ZwcLB++ctfyt/fX99//722bNmi1atXa86cOWrVqpVeeukl3XTTTcrPz1dOTk6Z3p5++mllZmaqbdu2GjZsmHdbu3bttHXrVmVnZ2vo0KEaMGCAjh07VjlvGgC4iGOB2BgTYoxJNMZcaYzxN8YMk9RF0iqnzgEA5TFmzBhFRUWpXr16mjBhgt544w2FhobqjjvuUM2aNeXxeDRhwgQlJSV5j6lRo4b+85//qKCgQOHh4WrTpo13W1FRkYYMGaLs7GwtW7ZMNWvW1MGDB7Vy5Ur9+c9/Vq1atdSgQQM99thjWrhw4Xl769Wrl7p06aLAwED94Q9/0KZNm7R3715J0vDhwxUaGip/f3898cQTOn78uLZvZzZLAHCak0+qC5A0RVJLSSWSvpXU11rLtzeASrVkS5qmr9qu9JwCHcg9pn2FNb3boqOjlZ6erh9++EGPPfaY3n//fR0+fFiSlJeXp5KSEtWqVUtvvvmmXnjhBd19993q2LGjZsyYoZYtW0qSvv/+e3355Zf67LPPdMUVV0iSUlJSVFRUpPDwcO+5SktLFRUVdd5eT99eu3Zt1atXT+np6YqKitKMGTM0Z84cpaenyxijI0eOKDMz07H3CQBwgmN3iK21GdbadtZaj7U2xFp7o7X2A6fqA8DFWLIlTU8u/kppOQWykopLrea8/7mWbEmTJKWmpioiIkIzZszQ9u3b9emnn+rIkSNat26dJMlaK0lKTEzUBx98oP3796tly5a65557vOdo1aqV5s6dqx49enjv2EZFRSkwMFCZmZnKyclRTk6Ojhw5oq+//lqSZMzZR4+duhssSfn5+crOzlZERITWr1+v5557TosWLdLhw4eVk5Oj4OBgb38AAOcwDzGAy8r0VdtVUFRSZl3258s0ZdEGZWdna+rUqRo0aJDy8vIUFBSkkJAQZWdna/Lkyd79Dx48qHfffVdHjx5VYGCgateuLT8/vzI1hwwZoqlTp6p79+7auXOnwsPDdeutt+qJJ57QkSNHVFpaqp07d3qHYTRs2FD79u1TYWFhmTrvvfeeNmzYoMLCQj399NNq3769oqKilJeXJ39/f4WFham4uFi/+93vdOTIkQp61wDA3QjEAC4r6TkFP1pXq3WCts0ep9jYWMXGxmrixIl69NFHVVBQoPr16+vGG2/Ubbfd5t2/tLRUM2bMUEREhOrVq6ekpCT97W9/+1HdX/7yl/rtb3+rbt26ac+ePXr11VdVWFjondHizjvv1P79+yVJ3bp1U5s2bdSoUSPVr1/fW2Po0KGaPHmy6tWrp82bN3tns0hMTFSPHj3UvHlzRUdH68orr7zg8AsAQPkYX//zW3x8vE1OTvZpDwAuHx2fXaO0s4TiyJAgfTy+mw86AgBUBcaYzdba+LNt4w4xgMvKuMQWCgooO7whKMBP4xJb+KgjAEBV5+QsEwDgc33jIiXJO8tEREiQxiW28K4HAOBMBGIAl52+cZEEYADARWPIBAAAAFyNQAwAAABXIxADAADA1QjEAAAAcDUCMQAAAFyNQAwAAABXIxADAADA1QjEAAAAcDUCMQAAAFyNQAwAAABXIxADAADA1QjEAAAAcDUCMQAAAFyNQAwAAABXIxADAADA1QjEAAAAcDUCMQAAAFyNQAwAAABXIxADAADA1QjEAAAAcDUCMQAAAFyNQAwAAABXIxADAADA1SokEBtjrjbGHDPGzK+I+gAAAIBTKuoO8V8lfV5BtQEAAADHOB6IjTGDJeVI+sjp2gAAAIDTHA3Expg6kn4n6YkL7HevMSbZGJOckZHhZAsAAADAT+L0HeLfS3rZWrv3fDtZa2dZa+OttfFhYWEOtwAAAABcPH+nChlj2krqLinOqZoAAABARXMsEEvqKilGUqoxRpJqS/IzxrS21l7n4HkAAAAAxzgZiGdJWnja8q91IiA/4OA5AAAAAEc5NobYWvuDtfbAqZekfEnHrLX8ag4AAFyS7du3Ky4uTh6PRzNnznSs7rx589SpU6dyHbt27Vo1btzYuxwTE6MPP/zQqdZQiZy8Q1yGtXZSRdUGAADu8vzzz6tr167asmVLuWvs2bNHTZs2VVFRkfz9KywCoRri0c0AAKDKS0lJUZs2bXzdBi5TBGIAAFCldevWTf/+9781ZswY1a5dW19++aVGjhypsLAwRUdHa8qUKSotLZUklZaWasqUKYqOjlaDBg00cuRI5ebmSpK6dOkiSQoJCVHt2rW1adMmSZK1Vg8//LCCg4PVsmVLffTR/54tNnfuXLVq1Uoej0exsbH6xz/+UclXj8pAIAYAAFXamjVr1LlzZ/3lL39Rfn6+ZsyYodzcXO3atUtJSUl69dVXNXfuXEknxgTPmzdP//73v7Vr1y7l5+drzJgxkqR169ZJknJycpSfn6+bbrpJkvTpp58qNjZWmZmZmjx5svr376/s7GxJUoMGDbR8+XIdOXJEc+fO1WOPPaYvvvjCB+8CKhKBGAAAVBslJSV68803NW3aNHk8HsXExOiJJ57Qa6+9JklasGCBHn/8ccXGxqp27dqaNm2aFi5cqOLi4nPWbNCggR599FEFBARo0KBBatGihVasWCFJ6tWrl6666ioZY5SQkKBbb71V69evr5RrReUhEAMAgGojMzNThYWFio6O9q6Ljo5WWlqaJCk9Pf1H24qLi3Xw4MFz1oyMjNTJZyh4j0lPT5ckrVy5UjfeeKPq1aunkJAQvffee8rMzHT6suBjBGIAAFAlLdmSpo7PrlHT8Su0JTVHW1IPq379+goICFBKSop3v9TUVEVGRkqSIiIifrTN399fDRs2LBN6T5eWliZrbZljIiIidPz4cd1xxx369a9/rYMHDyonJ0c9e/Yssy8uDwRiAABQ5SzZkqYnF3+ltJwCWUnHi0u0+Is0Ldt2QAMHDtSECROUl5enlJQU/fGPf9Tw4cMlSUOGDNGf/vQn7d69W/n5+Xrqqac0aNAg+fv7KywsTDVq1NCuXbvKnOvQoUOaOXOmioqK9NZbb+m///2vevbsqcLCQh0/flxhYWHy9/fXypUrtXr1ah+8G6hoTMIHAACqnOmrtqugqKTMusKSUk1ftV3LX3xRDz/8sGJjY3XllVfqnnvu0V133SVJuuuuu5Senq4uXbro2LFjSkxM1IsvvihJqlmzpiZMmKCOHTuqqKhI77//viSpffv2+u6771S/fn01bNhQb7/9tkJDQyVJM2fO1MCBA3X8+HHdfvvt6tOnTyW+C6gsxte3/ePj421ycrJPewAAAFVL0/ErdLaEYiTtfrZXZbeDy4AxZrO1Nv5s2xgyAQAAqpyIkKCftB64FARiAABQ5YxLbKGgAL8y64IC/DQusYWPOsLljDHEAACgyukbd2LWiOmrtis9p0ARIUEal9jCux5wEoEYAABUSX3jIgnAqBQMmQAAAICrEYgBAADgagRiAAAAuBqBGAAAAK5GIAYAAICrEYgBAADgagRiAAAAuBqBGAAAAK5GIAYAAICrEYgBAADgagRiAAAAuBqBGAAAAK5GIAYAAICrEYgBAADgagRiAAAAuJqjgdgYM98Ys98Yc8QYs8MY8ysn6wMAAABOc/oO8TRJMdbaOpL6SJpijLne4XMAAAAAjnE0EFtrv7bWHj+1ePJ1lZPnAAAAAJzk+BhiY8zfjDE/SPpW0n5J7zl9DgAAAMApjgdia+2DkjySOktaLOn4mfsYY+41xiQbY5IzMjKcbgEAAAC4aBUyy4S1tsRau0FSY0kPnGX7LGttvLU2PiwsrCJaAAAAAC5KRU+75i/GEAMAAKAKcywQG2MaGGMGG2NqG2P8jDGJkoZIWuPUOQAAAACn+TtYy+rE8IiXdCJop0h61Fq71MFzAAAAAI5yLBBbazMkJThVDwAAAKgMPLoZAAAArkYgBgAAgKsRiAEAAOBqBGIAAAC4GoEYAAAArkYgBgAAgKsRiAEAAOBqBGIAAAC4GoEYAAAArkYgBgAAgKsRiAEAAOBqBGIAAAC4GoEYAAAArkYgBgAAgKsRiAEAAOBqBGIAAAC4GoEYAAAArkYgBgAAgKsRiAEAAOBqBGIAAAC4GoEYAAAArkYgBgAAgKsRiAEAAOBqBGIAAAC4GoEYAAAArkYgBgAAgKsRiAEAAOBqBGIAAAC4GoEYAAAAruZYIDbGBBpjXjbGpBhj8owxW4wxPZyqDwAAAFQEJ+8Q+0vaKylBUrCkpyUtMsbEOHgOAAAAwFH+ThWy1h6VNOm0VcuNMbslXS9pj1PnAQAAAJxUYWOIjTENJTWX9PVZtt1rjEk2xiRnZGRUVAsAAADABVVIIDbGBEhaIOkVa+23Z2631s6y1sZba+PDwsIqogUAAADgojgeiI0xNSS9JqlQ0hin6wMAAABOcmwMsSQZY4yklyU1lNTTWlvkZH0AAADAaY4GYkl/l9RKUndrbYHDtQEAAADHOTkPcbSk+yS1lXTAGJN/8jXMqXMAAAAATnNy2rUUScapegAAAEBl4NHNAAAAcDUCMQAAAFyNQAwAAABXIxADAADA1QjEAAAAcDUCMQAAAFyNQAwAAABXIxADAADA1QjEAAAAcDUCMQAAAFyNQAwAAABXIxADAADA1QjEAAAAcDUCMQAAAFyNQAwAAABXIxADAADA1QjEAAAAcDUCMQAAAFyNQAwAAABXIxADAADA1QjEAAAAcDUCMQAAAFyNQAwAAABXIxADAADA1QjEAAAAcDUCMQAAAFyNQAwAAABXIxADAADA1RwNxMaYMcaYZGPMcWPMPCdrAwAAABXB3+F66ZKmSEqUFORwbQAAAMBxjgZia+1iSTLGxEtq7GRtAAAAoCIwhhgAAACu5pNAbIy59+RY4+SMjAxftAAAAABI8lEgttbOstbGW2vjw8LCfNECAAAAIIkhEwAAAHA5p6dd8zfGXCnJT5KfMeZKY4zTM1kAqIJiYmI0ffp0XXPNNapVq5buvvtuHTx4UD169JDH41H37t11+PBhSdKAAQPUqFEjBQcHq0uXLvr666+9dUaNGqWHHnpIvXr1ksfjUfv27bVz505fXRYAwAWcvkM8UVKBpPGShp/8e6LD5wBQRf3rX//SBx98oB07dmjZsmXq0aOHpk6dqszMTJWWlmrmzJmSpB49eui7777ToUOHdN1112nYsGFl6rzxxht65plndPjwYTVr1kwTJkzwxeUAAFzC6WnXJkma5GRNANXHww8/rIYNG0qSOnfurAYNGiguLk6S1K9fP3300UeSpLvuust7zKRJk1S3bl3l5uYqODhYktS/f3/dcMMNkqRhw4bp8ccfr8zLAAC4DMMZAJTbki1pmr5qu9JzCnQg95i+z/PzbgsKCvKG41PL+fn5Kikp0YQJE/TWW28pIyNDNWqc+IeqzMxMbyBu1KiR97iaNWsqPz+/kq4IAOBG/KgOQLks2ZKmJxd/pbScAllJxaVWL6/frSVb0s573Ouvv66lS5fqww8/VG5urvbs2SNJstZWfNMAAJwFgRhAuUxftV0FRSVl1h0vKdX0VdvPe1xeXp4CAwMVGhqqH374QU899VRFtgkAwAURiAGUS3pOwU9af8rIkSMVHR2tyMhItW7dWjfeeGNFtAcAwEUzvv5nyvj4eJucnOzTHgD8dB2fXaO0s4TfyJAgfTy+mw86AgDg3Iwxm6218Wfbxh1iAOUyLrGFggL8yqwLCvDTuMQWPuoIAIDyYZYJAOXSNy5SkryzTESEBGlcYgvvegAAqgsCMYBy6xsXSQAGAFR7DJkAAACAqxGIAQAA4GoEYgAAALgagRgAAACuRiAGAACAqxGIAQAA4GoEYgAAALgagRgAAACuRiAGAACAqxGIAQAA4GoEYgAAALgagRgAAACuRiAGAACAqxGIAQAA4GoEYgDwkblz5+r222/3Ljdr1kwDBw70LkdFRWnr1q3auHGj2rVrp+DgYLVr104bN2707tO1a1dNnDhRHTp0UO3atXX77bcrKytLw4YNU506ddSuXTvt2bPHu/8jjzyiqKgo1alTR9dff73Wr1/v3TZp0iQNHDhQI0eOlMfjUZs2bZScnFyxbwIAVAEEYgDwkYSEBK1fv16lpaXav3+/ioqK9PHHH0uSdu3apfz8fDVp0kS9evXS2LFjlZWVpccff1y9evVSVlaWt87ChQv12muvKS0tTTt37tRNN92k0aNHKzs7W61atdLkyZO9+7Zr105bt25Vdna2hg4dqgEDBujYsWPe7e+++64GDx6snJwc9enTR2PGjKm8NwQAfIRADAA+EhsbK4/Ho61btyopKUmJiYmKjIzUt99+q6SkJHXu3FkrVqzQ1VdfrREjRsjf319DhgxRy5YttWzZMm+d0aNH66qrrlJwcLCuv/56HThwQP369dOwYcO0e/duvffee5o3b546deqk4cOHKzQ0VP7+/vr1r3+tgoICbd++XcePH9fq1au99R566CENGDBAX375pSRp+fLlatu2rUJCQtShQwdt27bNe/6YmBi98MILuuaaaxQcHKxBgwaVCdkAUNURiAGgki3ZkqaOz65R0/ErdKx+C724YKnWrVunhIQEde3aVUlJSUpKSlJCQoLS09MVHR1d5vjo6GilpaV5lxs2bChJKiws1LvvvquWLVsqOztbAwYM0MaNG1VUVOTdd8aMGWrVqpWCg4MlSXl5ecrMzNRvfvMbZWVl6fbbb9f333+vtLQ0zZ49W8eOHdNnn32mu+66S//4xz+UlZWl++67T3369NHx48e9dRctWqT3339fu3fv1rZt2zRv3rwKfAcBwFkEYgCoREu2pOnJxV8pLadAVlJpw1Z6e9kqrfhgjRISEpSQkFAmEEdERCglJaVMjdTUVEVGRv6o9ieffKLS0lK1bt1aAQEBuvPOO9WiRQvv9tzcXD333HNatGiRDh8+LEnyeDwqLS3V7NmzlZiYqMDAQHk8Hj311FPeu9Avv/yy7rvvPrVv315+fn765S9/qcDAQH3yySfe2mPHjlVERITq1aun22+/XVu3bq2Adw8AKgaBGAAq0fRV21VQVOJdvrLJz3Q0ZZsOZeepcePG6ty5s95//31lZWUpLi5OPXv21I4dO/T666+ruLhYb775pr755hv17t37R7XT09Pl8XjKrDt191iSSkpK5O/vr7CwMBUXF0uS8vPzlZOTox9++EGzZs3SokWLFBISottuu03Z2dmSpJSUFM2YMUMhISHe1969e5Wenu6t3ahRI+/fNWvWVH5+vjNvGABUAn9fNwAAbpKeU1BmOaBepGoEXCn/iNaSpDp16ig2NlZhYWHy8/NTaGioli9frkceeUQPPPCAmjVrpuXLl6t+/freGltSD6vjs2u0a9seHcrKUWrWUe+2Q4cOSZJq1aqlwMBA9ejRQ82bN1dQUJAkKTw8XMHBwQoKCtL999+vQ4cOaf78+ZKkPXv2qGnTpoqKitKECRM0YcKECn1vAMBXHL1DbIypZ4x5xxhz1BiTYowZ6mR9AKjuIkKCfrSu8ZjXdM2wJ73LycnJWrlypXe5U6dO2rx5s3Jzc7V582Z16tTJu+3RPy3QGvszpeUU6IrIljKBtfV1Yaj+9XmKFi9erB07dujBBx/Utddeq2+++UYPP/ywDh06pH79+kmS1q5dq1tvvVX33HOP9u3bpz/+8Y+SpLS0NG3fvl3WWt1333166aWX9Omnn8paq6NHj2rFihXKy8urqLcJACqV00Mm/iqpUFJDScMk/d0Y08bhcwBV2qhRozRx4kRft4EqalxiCwUF+JVZFxTgp3GJLc5xxPmdPgTD+AUorN9TOrz1Aw3s1Epvvvmm+vfvL0lq3ry5fvvb36p79+66+uqry4RqSXruuefUrFkz3XjjjapTp466d++u7du3S5Li4+M1e/ZsjRkzRnXr1lWzZs340RyAy4qx1jpTyJhakg5L+n/W2h0n170mKc1aO/5cx8XHx1smfsflZNSoUWrcuLGmTJni61ZQRS3Zkqbpq7YrPadAESFBGpfYQn3jfvwjuYvRdPwKne1b3Eja/WwvPo8AcJIxZrO1Nv5s25wcQ9xcUsmpMHzSl5ISztLQvZLulaQmTZo42AIAVH194yLLHYDPFBESpLQzxiWfWg8AuDhODpmoLSn3jHW5kjxn7mitnWWtjbfWxoeFhTnYAlD5tmzZouuuu04ej+dHDySYPXu2mjVrpnr16qlPnz5lfpW/evVqtWjRQsHBwXrwwQeVkJCgOXPm+OISUI05PQQDANzIyUCcL6nOGevqSOJXF7hsFRYWqm/fvhoxYoT3QQj/+te/JElr1qzRk08+qUWLFmn//v2Kjo7W4MGDJUmZmZm68847NW3aNGVlZalFixbauHGjLy8F1VTfuEhN6/8zRYYEyUiKDAnStP4/896BnjdvHsMlAOACKmIMcRtr7Xcn170qKZ0xxLhcrVu3ToMHD1ZaWpqMMZKkDh06qFu3btq/f79CQ0P1/PPPSzox32vdunX13Xffad26dfr73/+uTZs2SZKstWrSpImeeeYZ/epXv/LZ9QAAcLmqlDHE1tqjxpjFkn5njPmVpLaSfiGpg1PnAKqC038QdUXKJtWq28AbhiV5H7Obnp6u6667zru+du3aCg0NVVpamtLT0xUVFeXdZoxR48aNK+8iAACAl9PTrj0oKUjSIUlvSHrAWvu1w+cAfObMx+7mmtrak7pX73yxz7tPamqqJP3okbtHjx5VVlaWIiMjFR4ern37/neMtbbMMgAAqDyOBmJrbba1tq+1tpa1tom19nUn6wO+duZjdwMjW8oaPz3+zDQVFxdr8eLF+uyzzyRJQ4cO1dy5c7V161YdP35cTz31lNq3b6+YmBj16tVLX331lZYsWaLi4mL99a9/1YEDB3x1WQAAuJrTd4iBy9qZj9099SCE9M/eV926dcs8COHnP/+5fv/73+uOO+5QeHi4du7cqYULF0qS6tevr7feekv/93//p9DQUH3zzTeKj49XYGBgpV8TAABu59iP6sqLH9WhOun47JqzzvkaGRKkj8d3K3fd0tJSNW7cWAsWLNDNN998KS0CAICzON+P6rhDDPwETs75umrVKuXk5Oj48eOaOnWqrLW68cYbnWoVAABcJCefVAdc9k7N7erEY3c3bdqkoUOHqrCwUK1bt9aSJUsUFMTTxQAAqGwMmfgJiouL5e/P/0MAAABUNwyZuIAvvvhCcXFx8ng8GjBggAYNGqSJEydq7dq1aty4sZ577jk1atRIo0ePVmlpqZ599lldddVVCg0N1cCBA5Wdne2t9cknn6hDhw4KCQnRtddeq7Vr13q3de3aVU8//bQ6duwoj8ejW2+9VZmZmT64YgAAAJzi+kBcWFiofv36adSoUcrOztaQIUP0zjvveLcfOHBA2dnZSklJ0axZszRz5kwtWbJESUlJSk9PV926dfXQQw9JktLS0tSrVy9NnDhR2dnZeuGFF3THHXcoIyPDW+/111/X3LlzdejQIRUWFuqFF16o9GsGAADA/7g+EH/yyScqLi7W2LFjFRAQoP79++uGG27wbq9Ro4YmT56swMBABQUF6R//+If+8Ic/qHHjxgoMDNSkSZP09ttvq7i4WPPnz1fPnj3Vs2dP1ahRQ7fccovi4+P13nvveeuNHj1azZs3V1BQkAYOHKitW7f64rIBAABwkusHxKanpysyMrLMo3dPf6RuWFiYrrzySu9ySkqK+vXrpxo1/vf/En5+fjp48KBSUlL01ltvadmyZd5tRUVFZabRatSokffvmjVrKj8/3/FrAgAAwMVzbSBesiVN01dt165te5S9fZfe+WKf+l3XWJK0d+9eXXXVVZJUJihLJ8LyP//5T3Xs2PFHNaOiojRixAjNnj274i8AAAAAjnDlkIklW9L05OKvlJZToCsiW6rE1tD9T/1B//o8RUuXLvU+evds7r//ft19992Kjz/xI8WMjAwZY/T9999r+PDhWrZsmVatWqWSkhIdO3ZMa9eu1b59+yrr0gAAAPATuTIQT1+1XQVFJZL+9+jdw1tWaWCnlpo/f6gYj6cAABI4SURBVL569+59zkfoPvLII4qLi9PXX38tj8dT5kEKUVFRWrp0qaZOnaqwsDBFRUVp+vTpKi0trZTrAgAAwE/nynmIm45fobNdtZG0+9leat++ve6//36NHj36rMfPmzdPc+bM0YYNG04cZ4y+++47NWvWrOKaBgAAQLkxD/EZIkJOPA0s99N/KeOdqTqW+pVK8g8r3HOFunfvrs2bN6tDhw66++67FR4ersjISE2cOFElJSUXrJ2bm6uRI0cqLCxM0dHRmjJlivcOcXR0tDZv3ixJmj9/vowx+uabbyRJc+bMUd++fSvoigEAAHAurgzE4xJbKCjAT7Xa3KyC3ZtVeHCn9s97WF9M+YXWrl2rGTNm6De/+Y38/f31/fffa8uWLVq9erXmzJlzwdoPP/ywcnNztWvXLiUlJenVV1/V3LlzJUkJCQneB3WsW7dOsbGxSkpK8i4nJCRU2DUDAADg7FwZiPvGRWpa/58punGkrmz8/1QvuI7eXv8fvf3WIjVv3lyDBw/WypUr9ec//1m1atVSgwYN9Nhjj2nhwoXnrVtSUqI333xT06ZNk8fjUUxMjJ544gm99tprkk4E4lMBeP369XryySe9y0lJSQRiAAAAH3BlIJZOhOKPx3fTP6f+WpFZyeobF6n58+drxIgRSklJUVFRkcLDwxUSEqKQkBDdd999OnTo0HlrZmZmqrCwUNHR0d510dHRSktLk3QiEK9fv14HDhxQSUmJBg0apI8//lh79uxRbm6u2rZtW6HXDAAAgB9z3TzEp+YfTs8pUERIkB7p2k7btj2g//znP1q+fLmef/55BQQEKDAwUJmZmfL3v/i3qH79+goICFBKSopat24tSUpNTVVkZKQkqVmzZqpZs6ZmzpypLl26yOPxqFGjRpo1a5Y6depU5mEfAAAAqByuSmCnzz9sJaXlFOiZFd8p/uYeGjp0qG644QY1adJE4eHhuvXWW/XEE0/oyJEjKi0t1c6dO73DG87Fz89PAwcO1IQJE5SXl6eUlBT98Y9/1PDhw737JCQk6C9/+Yt3eETXrl3LLAMAAKByuSoQnz7/8CkFRSU60KC9vvrqK40YMcK7/tVXX1VhYaFat26tunXr6s4779T+/fsveI4XX3xRtWrVUmxsrDp16qShQ4fqrrvu8m5PSEhQXl6eunTpctZlAAAAVC5XzUN8rvmHS44cUta8h3TgwAHVqVOnUnoBAABA5WEe4pNOzT98OmtLVfLlcg0ePJgwDAAA4EKuCsSn5h8+pbTwmPb+aaD8D/xHkydP9mFnAAAA8BVXzTLRN+7EbA+nZpmIalBXMz/93rseAAAA7uOqQCydCMUEYAAAAJziqiETAAAAwJkIxAAAAHA1AjEAAABcjUAMAAAAV3MkEBtjxhhjko0xx40x85yoCQAAAFQGp2aZSJc0RVKipB8//QIAAACoohwJxNbaxZJkjImX1NiJmgAAAEBl8MkYYmPMvSeHWCRnZGT4ogUAAABAko8CsbV2lrU23lobHxYW5osWAAAAAEkXEYiNMWuNMfYcrw2V0SQAAABQUS44htha27US+gAAAAB8wpEf1Rlj/E/W8pPkZ4y5UlKxtbbYifoAAABARXFqDPFESQWSxksafvLviQ7VBgAAACqMU9OuTZI0yYlaAAAAQGXi0c0AAABwNQIxAAAAXI1ADAAAAFcjEAMAAMDVCMQAAABwNQIxAAAAXI1ADAAAAFcjEAMAAMDVCMQAAABwNQIxAAAAXI1ADAAAAFcjEAMAAMDVCMQAAABwNQIxAAAAXI1ADAAAAFcjEAMAAMDVCMQAAABwNQIxAAAAXI1ADAAAAFcjEAMAAMDVCMQAAABwNQIxAAAAXI1ADAAAAFcjEAMAAMDVCMQAAABwNQIxAAAAXI1ADAAAAFcjEAMAAMDVLjkQG2MCjTEvG2NSjDF5xpgtxpgeTjQHAAAAVDQn7hD7S9orKUFSsKSnJS0yxsQ4UBsAAACoUP6XWsBae1TSpNNWLTfG7JZ0vaQ9l1ofAAAAqEiOjyE2xjSU1FzS107XBgAAAJzmaCA2xgRIWiDpFWvtt+fZ715jTLIxJjkjI8PJFgAAAICf5IKB2Biz1hhjz/HacNp+NSS9JqlQ0pjz1bTWzrLWxltr48PCwi75IgAAAIDyuuAYYmtt1wvtY4wxkl6W1FBST2tt0aW3BgAAAFS8S/5R3Ul/l9RKUndrbYFDNQEAAIAK58Q8xNGS7pPUVtIBY0z+ydewS+4OAAAAqGBOTLuWIsk40AsAAABQ6Xh0MwAAACpVTEyMPvzwwx+tb9OmjdauXVvp/Tg1hhgAAAC4JF9/7ZvHWHCHGAAAAK5GIAYAAIDPfPvtt2ratKkWLlxYZijFpEmTNHDgQI0cOVIej0dt2rRRcnKy97gvvvhCcXFx8ng8GjBggAYNGqSJEyeWqwcCMQAAAHziiy++0K233qoXX3xRgwcP/tH2d999V4MHD1ZOTo769OmjMWNOPPutsLBQ/fr106hRo5Sdna0hQ4bonXfeKXcfBGIAAABUuvXr16tPnz565ZVX1Lt377Pu06lTJ/Xs2VN+fn4aMWKEvvzyS0nSJ598ouLiYo0dO1YBAQHq37+/brjhhnL3QiAGAABApXvppZfUoUMH3Xzzzefcp1GjRt6/a9asqWPHjqm4uFjp6emKjIzUiYclnxAVFVXuXgjEAAAAqHBLtqSp47Nr1HT8Ch3IPabR//cHpaam6rHHHvvJtcLDw5WWliZrrXfd3r17y90bgRgAAAAVasmWND25+Cul5RTISioutXr9iww9/Pw/tW7dOo0fP/4n1bvpppvk5+env/zlLyouLtbSpUv12Weflbs/5iEGAABAhZq+arsKikrKrDteUqq/bTygDz74QDfffLMCAgIuut4VV1yhxYsX61e/+pWefPJJ9ejRQ71791ZgYGC5+jOn32r2hfj4eHv6FBoAAAC4vDQdv0JnS5xG0u5nezlyjvbt2+v+++/X6NGjz7rdGLPZWht/tm0MmQCAy9ykSZM0fPhwX7cBwMUiQoJ+0vqLkZSUpAMHDqi4uFivvPKKtm3bpttuu61ctQjEAAAAqFDjElsoKMCvzLqgAD+NS2xR7prbt2/Xtddeq+DgYM2YMUNvv/22wsPDy1WLMcQAUM0UFxfL35+vbwDVR9+4SEknxhKn5xQoIiRI4xJbeNeXx7333qt7773Xkf64QwwA1UBMTIyee+45XXPNNapVq5Y2bNigDh06KCQkRNdee63Wrl3r3Xf37t1KSEiQx+PRLbfcoszMTN81DgAn9Y2L1Mfju2n3s7308fhulxSGncYtBgCoJt544w2tWLFCNWrU0DXXXKPXXntNt912mz766CPdcccd+vbbbxUWFqahQ4fqpptu0urVq/Xpp5+qV69e+sUvfuHr9gGgyuIOMQBUE2PHjlVUVJTmz5+vnj17qmfPnqpRo4ZuueUWxcfH67333lNqaqo+//xz/f73v1dgYKC6dOmi22+/3detA0CVxh1iAKiilmxJ8463O5B7TKnHT/waOyUlRW+99ZaWLVvm3beoqEg333yz0tPTVbduXdWqVcu7LTo6+pKe4AQAlzsCMQBUQaee6nRqIvviUqt/btijtjemKSoqSiNGjNDs2bN/dFxKSooOHz6so0ePekNxamqqjDGV2j8AVCcMmQCAKuhcT3Wavmq7hg8frmXLlmnVqlUqKSnRsWPHtHbtWu3bt0/R0dGKj4/XM888o8LCQm3YsKHMneSz6dq1q+bMmeNdnjhxourXr69GjRpVyLUBQFVDIAaAKig9p+Cc66OiorR06VJNnTpVYWFhioqK0vTp01VaWipJev311/Xpp5+qXr16mjx5skaOHHnR5927d69mzJihb775RgcOHHDkWgCgqmPIBABUQREhQUo7LRQ3fuCf3vXSiUeUJiUlnfXY2NhYrV+/vlznTUlJUWhoqBo0aFCu4wGgOuIOMQBUQRf7VKe5c+eWmUWiWbNmGjhwoHc5KipKW7du1caNG9WuXTsFBwerXbt22rhx44/O+eGHH+qWW25Renq6ateurVGjRjl7UQBQRRGIAaAK6hsXqWn9f6bIkCAZSZEhQZrW/2c/msg+ISFB69evV2lpqfbv36+ioiJ9/PHHkqRdu3YpPz9fTZo0Ua9evTR27FhlZWXp8ccfV69evZSVlVWmVvfu3bVy5UpFREQoPz9f8+bNq6SrBQDfYsgEAFRRfeMiL/gkp9jYWHk8Hm3dulU7duxQYmKitm7dqm+//VabNm1S586dtWLFCl199dUaMWKEJGnIkCGaOXOmli1bxl1gABCBGACqpdPnKD5Wv4VeXLBUQQUZSkhIUEhIiJKSkrRp0yYlJCQoPT1d0dHRZY6Pjo5WWlqaj7oHgKqFIRMAUM2cmqM4LadAVlJpw1Z6e9kqrfhgjRISEpSQkKCkpCQlJSUpISFBERERSklJKVMjNTVVkZHnv/sMAG5BIAaAaubMOYqvbPIzHU3ZpkPZeWrcuLE6d+6s999/X1lZWYqLi1PPnj21Y8cOvf766youLtabb76pb775Rr179/bhVQBA1cGQCQCoZs6cozigXqRqBFwp/4jWkqQ6deooNjZWYWFh8vPzU2hoqJYvX65HHnlEDzzwgJo1a6bly5erfv36vmgfAKocY6299CLGzJf0c0m1JB2Q9Ly1ds75jzohPj7eJicnX3IPAOAWHZ9dU2aO4lMiQ4L08fhuPugIAKo+Y8xma2382bY5NWRimqQYa20dSX0kTTHGXO9QbQDAaS52jmIAwMVxZMiEtfbr0xdPvq6StNmJ+gCA/zk1FdupWSYiQoI0LrHFBadoAwCcnSNDJiTJGPM3SaMkBUnaIqmLtTb/HPveK+leSWrSpMn1Z/76GQAAAHBSZQyZkLX2QUkeSZ0lLZZ0/Dz7zrLWxltr48PCwpxqAQAAAPjJLhiIjTFrjTH2HK8Np+9rrS2x1m6Q1FjSAxXVNAAAAOCUC44httZ2LWfdq8pxHAAAAFCpLnnIhDGmgTFmsDGmtjHGzxiTKGmIpDWX3h4AAABQsZyYZcLqxPCIl3QiYKdIetRau9SB2gAAAECFuuRAbK3NkJTgQC8AAABApXNslgkAAACgOiIQAwAAwNUIxAAAAHA1AjEAAABczbFHN5e7AWMydGJmigupLymzgtvB5YnPDsqDzw3Ki88OyoPPTcWLttae9RHJPg/EF8sYk3yu508D58NnB+XB5wblxWcH5cHnxrcYMgEAAABXIxADAADA1apTIJ7l6wZQbfHZQXnwuUF58dlBefC58aFqM4YYAAAAqAjV6Q4xAAAA4DgCMQAAAFyt2gZiY8zVxphjxpj5vu4FVZsxJtAY87IxJsUYk2eM2WKM6eHrvlA1GWPqGWPeMcYcPfmZGerrnlD18T2DS0Wu8a1qG4gl/VXS575uAtWCv6S9khIkBUt6WtIiY0yMD3tC1fVXSYWSGkoaJunvxpg2vm0J1QDfM7hU5BofqpaB2BgzWFKOpI983QuqPmvtUWvtJGvtHmttqbV2uaTdkq73dW+oWowxtSTdIelpa22+tXaDpHcljfBtZ6jq+J7BpSDX+F61C8TGmDqSfifpCV/3gurJGNNQUnNJX/u6F1Q5zSWVWGt3nLbuS0ncIcZPwvcMLha5pmqodoFY0u8lvWyt3evrRlD9GGMCJC2Q9Iq19ltf94Mqp7ak3DPW5Ury+KAXVFN8z+AnItdUAVUqEBtj1hpj7DleG4wxbSV1l/QnX/eKquNCn5vT9qsh6TWdGB86xmcNoyrLl1TnjHV1JOX5oBdUQ3zP4Kcg11Qd/r5u4HTW2q7n226MeVRSjKRUY4x04m6OnzGmtbX2ugpvEFXShT43kmROfGBe1okfSvW01hZVdF+olnZI8jfGXG2t/e7kumvFP3vjIvA9g3LoKnJNlVCtnlRnjKmpsndvfq0TH6QHrLUZPmkK1YIx5iVJbSV1t9bm+7ofVF3GmIWSrKRf6cRn5j1JHay1hGKcF98z+KnINVVHlbpDfCHW2h8k/XBq2RiTL+kYHxqcjzEmWtJ9ko5LOnDy/8Il6T5r7QKfNYaq6kFJ/5R0SFKWTvyHiTCM8+J7BuVBrqk6qtUdYgAAAMBpVepHdQAAAEBlIxADAADA1QjEAAAAcDUCMQAAAFyNQAwAAABXIxADAADA1QjEAAAAcDUCMQAAAFyNQAwAAABX+/+YaZ5x4TsHfwAAAABJRU5ErkJggg==\n",
+ "text/plain": [
+ ""
+ ]
+ },
+ "metadata": {
+ "needs_background": "light"
+ },
+ "output_type": "display_data"
+ }
+ ],
+ "source": [
+ "embedding2 = embedding * 1.0 + pos_encoding[:,:,:] * 1.0\n",
+ "\n",
+ "plot_words(embedding2, sequences, 0)\n",
+ "plot_words(embedding2, sequences, 1)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Wow look at the big difference between the plots! Both plots have changed drastically compared to their original counterparts. Notice that in the second image, which corresponds to the sentence in which similar words are not together, very dissimilar words such as `red` and `wolf` appear more close.\n",
+ "\n",
+ "Now you can try different relative weights and see how this strongly impacts the vector representation of the words in the sentence."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 19,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "image/png": "iVBORw0KGgoAAAANSUhEUgAAAtMAAAFoCAYAAACVEXcaAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO3df3QW1YH/8fcloTFCIAKRHyEGAQVlraKhqCChiiKg1IIiKqhoVbpFFK3fRcEutopaStu17W6rdqFqLZSKWEFFqSWCiooCulihQAmQoBIggfA7ZL5/AE8BQcoYEhLer3NyTubemTt3MufJ88nNnfuEKIqQJEmSdPhqVXUHJEmSpOrKMC1JkiTFZJiWJEmSYjJMS5IkSTEZpiVJkqSYDNOSJElSTMlV3YEv06hRo6hFixZV3Q1JkiTVcO+//35RFEUZh3vcUR2mW7Rowdy5c6u6G5IkSarhQgj5cY5zmockSZIUk2FakiRJiskwLUmSJMVkmJakY1TXrl158sknE9sjR46kUaNGNGnSpAp7JUnVy1H9AKIkqXKsXLmSsWPHkp+fz4knnljV3ZGkasORaUkS+fn5NGzY0CAtSYfJMC1J1dC4ceO4/PLLE9utW7emX79+ie2srCzmz5/PW2+9RYcOHahfvz4dOnTgrbfe+kJbM2bM4OKLL6awsJC6dety4403VsYlSFKNYJiWpGooNzeXWbNmUV5ezurVq9mxYwdvvvkmAMuWLaO0tJSTTjqJXr16MXToUNauXctdd91Fr169WLt27T5tdevWjZdffplmzZpRWlrK+PHjq+CKJKl6MkxLUjXUsmVL0tLSmD9/Pnl5eXTv3p3MzEw++eQT8vLyuOCCC5g2bRqnnHIKAwcOJDk5mWuuuYa2bdvy4osvVnX3JanG8AFESaompswrYMz0RRQWb6FZeiotz+jAzJkzWbJkCbm5uaSnp5OXl8fbb79Nbm4uhYWFZGdn79NGdnY2BQUFVXQFklTzODItSdXAlHkF3Dv5IwqKtxABBcVbWByymPDCK8yaNYvc3Fxyc3PJy8sjLy+P3NxcmjVrRn7+vp+Ou2LFCjIzM6vmIiSpBjJMS1I1MGb6Irbs2LlPWa3Mdrw/ZzZbtmyhefPmXHDBBbzyyiusXbuW9u3b07NnTxYvXsyzzz5LWVkZEydO5OOPP+ayyy6roquQpJrHaR6SVA0UFm/5QlntBpmE2sdxwQUXAFCvXj1atmxJRkYGSUlJNGzYkKlTp3LHHXfw3e9+l9atWzN16lQaNWpU2d2XpBorRFFU1X04qJycnGju3LlV3Q1JqnKdHnmdggME6sz0VN4cfmEV9EiSapYQwvtRFOUc7nFO85CkauCe7m1IrZ20T1lq7STu6d6minokSQKneUhStXBF+10PDe69msc93dskyiVJVcMwLUnVxBXtMw3PknSUcZqHJEmSFJNhWpIkSYrJMC1JkiTFZJiWJEmSYjJMS5IkSTEZpiVJkqSYDNOSJElSTIZpSZIkKSbDtCRJkhSTYVqSJEmKyTAtSZIkxWSYliRJkmIyTEuSJEkxGaYlSZKkmAzTkiRJUkyGaUmSJCkmw7QkSZIUk2FakiRJiskwLUmSJMVkmJYkSZJiMkxLkiRJMRmmJUmSpJgM05IkSVJMhmlJkiQpJsO0JEk10I033sjIkSOruhtSjWeYlo4iixYton379qSlpfHYY49VWLvjx4+nc+fOsY6dOXMmzZs3T2y3aNGCGTNmVFTXJEmq1pKrugOS/unHP/4xXbt2Zd68ebHbWL58OSeffDI7duwgOdmXuCRJR5Ij09JRJD8/n3bt2lV1NyRVQ/PmzePss88mLS2Nq6++mq1btybqnnjiCVq3bk2DBg3o3bs3hYWFibpXX32VNm3aUL9+ff793/+d3Nxcnnzyyaq4BKlaMkxLR4kLL7yQv/71rwwZMoS6deuyYMECrr/+ejIyMsjOzubBBx+kvLwcgPLych588EGys7M58cQTuf766ykpKQGgS5cuAKSnp1O3bl3efvttAKIo4vbbb6d+/fq0bduWv/zlL4lzjxs3jtNOO420tDRatmzJb37zm0q+eklfxfbt27niiisYOHAg69at46qrruK5554D4PXXX+fee+/lj3/8I6tXryY7O5v+/fsDUFRUxJVXXsnDDz/M2rVradOmDW+99VZVXopU7RimpaPE66+/zgUXXMAvf/lLSktLGTt2LCUlJSxbtoy8vDyeeuopxo0bB+yaAz1+/Hj++te/smzZMkpLSxkyZAgAb7zxBgDFxcWUlpZy3nnnAfDOO+/QsmVLioqKeOCBB+jTpw/r1q0D4MQTT2Tq1Kls2LCBcePGMWzYMD744IMq+ClIimPOnDns2LGDO++8k9q1a3PllVfSoUMHAH7/+99z0003cfbZZ5OSksLDDz/M22+/zfLly3nppZdo164dffr0ITk5maFDh9KkSZMqvhqpejFMS0ehnTt3MnHiRB5++GHS0tJo0aIFd999N08//TSw683xrrvuomXLltStW5eHH36YCRMmUFZWdtA2TzzxxMQb7dVXX02bNm2YNm0aAL169aJVq1aEEMjNzeWSSy5h1qxZlXKtkr66wsJCMjMzCSEkyrKzsxN1e74HqFu3Lg0bNqSgoIDCwkKysrISdSGEfR44lnRoFRamQwgpIYTfhhDyQwgbQwjzQgg99qq/KITwSQhhcwjhryGE7C9rTzpWTJlXQKdHXufk4dOYt6KYeSvWU1RUxPbt2/d5A8zOzqagoAD44ptjdnY2ZWVlfPbZZwc9z4HeaPfMm3z55Zc599xzadCgAenp6bz00ksUFRVV9KVKqmB7fn8M+/NyFiz6B89/sCpRt2LFCgCaNWtGfn5+onzTpk2sXbuWzMxMmjZtyqpV/zwmiqJ9tiUdWkWOTCcDK4FcoD5wP/DHEEKLEEIjYPLusgbAXGBiBZ5bqpamzCvg3skfUVC8hQjYVraTyR8UMHvlNmrXrr3PG+CKFSvIzMwEvvjmuGLFCpKTk2ncuPE+gXlvBQUFRFG0zzHNmjVj27Zt9O3bl+9///t89tlnFBcX07Nnz332lXT02fv3x9cy21JOLQbf+yDPvZfP5MmTeffddwG49tprGTduHPPnz2fbtm3cd999dOzYkRYtWtCrVy8++ugjpkyZQllZGb/61a/49NNPq/jKpOqlwsJ0FEWboigaFUXR8iiKyqMomgr8AzgH6AMsjKJoUhRFW4FRwJkhhLYVdX6pOhozfRFbduzcp2z7znJ+OmMJ/fr1Y8SIEWzcuJH8/Hx++tOfMmDAAACuueYafvazn/GPf/yD0tJS7rvvPq6++mqSk5PJyMigVq1aLFu2bJ92P//8cx577DF27NjBpEmT+Nvf/kbPnj3Zvn0727ZtIyMjg+TkZF5++WVeffXVSvsZSIpn798fIak2Gd++j/XzX6Nf59OYOHEiffr0AeCiiy7iRz/6EX379qVp06YsXbqUCRMmANCoUSMmTZrE//t//4+GDRvy8ccfk5OTQ0pKSpVdl1TdHLFFaEMIjYFTgYXAd4EFe+qiKNoUQlgKtAM+OVJ9kI52hcVbDlo+9Re/4Pbbb6dly5Ycd9xx3HLLLdx0000A3HTTTRQWFtKlSxe2bt1K9+7d+cUvfgHA8ccfz4gRI+jUqRM7duzglVdeAaBjx478/e9/p1GjRjRu3Jg//elPNGzYEIDHHnuMfv36sW3bNi6//HJ69+5dCVcv6avY//dHStNTaDboMQIw8ZFe+9QNHjyYwYMHH7CdSy+9lMWLFwO7Vgpq3ry586alwxCOxL9yQwi1gZeBpVEU3RZC+C2wJoqi4Xvt8ybwRBRF4/c79lbgVoCTTjrpnL3/lS3VNJ0eeZ2CAwTqzPRU3hx+YRX0SFJ1UVG/P6ZPn07Hjh1JTU1lzJgx/OpXv2LZsmWkpqZWZHelo14I4f0oinIO97gKX80jhFALeBrYDgzZXVwK1Ntv13rAxv2Pj6Lo8SiKcqIoysnIyKjo7klHlXu6tyG1dtI+Zam1k7ine5sq6pGk6qKifn+8/fbbtGrVikaNGvHiiy8yZcoUg7R0GCp0ZDrsevLpf4EWQM8oirbsLr8VuCGKok67t+sAa4Czoyg66DSPnJycaO7cuRXWP+loNGVeAWOmL6KweAvN0lO5p3sbrmifWdXdklQN+PtDqjhxR6YrOkz/GjgL6BZFUele5RnAEuAmYBrwAJAbRdG5X9aeYVqSJEmVocqneexeN/o2doXpT0MIpbu/rouiaA3QF3gIWA90BPpX1LklSZKkqlBhq3lEUZQPHHiB2131MwCXwpMkSVKN4ceJS5IkSTEZpiVJkqSYDNOSJElSTIZpSZIkKSbDtCRJkhSTYVqSJEmKyTAtSZIkxWSYliRJkmIyTEuSJEkxGaYlSZKkmAzTkiRJUkyGaUmSJCkmw7QkSZIUk2FakiRJiskwLUmSJMVkmJYkSZJiMkxLkiRJMRmmJUmSpJgM05IkSVJMhmlJkiQpJsO0JEmSFJNhWpIkSYrJMC1JkiTFZJiWJEmSYjJMS5IkSTEZpiVJkqSYDNOSJElSTIZpSZIkKSbDtCRJkhSTYVqSJEmKyTAtSZIkxWSYliRJkmIyTEuSJEkxGaYlSZKkmAzTkiRJUkyGaUmSJCkmw7QkSZIUk2FakiRJiskwLUmSJMVkmJYkSZJiMkxLkiRJMRmmJUmSpJgM05IkSVJMhmlJkiQpJsO0JEmSFJNhWpIkSYrJMC1JkiTFZJiWJEmSYjJMS5IkSTEZpiVJkqSYDNOSJElSTIZpSZIkKSbDtCRJkhSTYVqSJEmKyTAtSZIkxWSYliRJkmIyTEuSJEkxVWiYDiEMCSHMDSFsCyGM36/uohDCJyGEzSGEv4YQsivy3JIkSVJlq+iR6ULgQeB/9y4MITQCJgP3Aw2AucDECj63JEmSVKmSK7KxKIomA4QQcoDme1X1ARZGUTRpd/0ooCiE0DaKok8qsg+SJElSZamsOdPtgAV7NqIo2gQs3V0uSZIkVUuVFabrAiX7lZUAafvvGEK4dfe867lr1qyplM5JkiRJcVRWmC4F6u1XVg/YuP+OURQ9HkVRThRFORkZGZXSOUmSJCmOygrTC4Ez92yEEOoArXaXS5IkSdVSRS+NlxxCOA5IApJCCMeFEJKB54F/CyH03V3/A+BDHz6UJElSdVbRI9MjgS3AcGDA7u9HRlG0BugLPASsBzoC/Sv43JIkSVKlquil8UYBow5SNwNoW5HnkyRJkqqSHycuSZIkxWSYliRJkmIyTEuSJEkxGaaV0KJFC2bMmFFh7Y0fP57OnTtXSFs33ngjI0eOjHXsqFGjGDBgAADLly8nhEBZWVmF9EuSJB3bDNM66lRkCJckSTqSDNOSJElSTIZp7eO9997j9NNP54QTTmDQoEFs3bqV9evXc9lll5GRkcEJJ5zAZZddxqpVqxLHjB8/npYtW5KWlsbJJ5/M73//+wO2fc8999C5c2dKSkooKSnh5ptvpmnTpmRmZjJy5Eh27tzJ3/72NwYPHszbb79N3bp1SU9PTxxfVFTExRdfTFpaGrm5ueTn5yfq7rjjDrKysqhXrx7nnHMOs2bNOnI/JEmSpN0M09rH73//e6ZPn87SpUtZvHgxDz74IOXl5QwaNIj8/HxWrFhBamoqQ4YMAWDTpk0MHTqUl19+mY0bN/LWW29x1lln7dNmeXk5t9xyCx9++CGvvvoq9evX54YbbiA5OZklS5Ywb948Xn31VZ588klOO+00fv3rX3PeeedRWlpKcXHxPn27//77KSoq4qyzzuK6665L1HXo0IH58+ezbt06rr32Wq666iq2bt1aOT80SZJ0zDJMax9DhgwhKyuLBg0aMGLECP7whz/QsGFD+vbty/HHH09aWhojRowgLy8vcUytWrX4v//7P7Zs2ULTpk1p165dom7Hjh1cc801rFu3jhdffJHjjz+ezz77jJdffpmf//zn1KlThxNPPJFhw4YxYcKEL+1br1696NKlCykpKTz00EO8/fbbrFy5EoABAwbQsGFDkpOTufvuu9m2bRuLFi06Mj8kSZKk3Sr0ExBV/UyZV8CY6YsoLN7CpyVbWbX9+ERddnY2hYWFbN68mWHDhvHKK6+wfv16ADZu3MjOnTupU6cOEydO5Cc/+Qk333wznTp1YuzYsbRtu+vDLpcsWcKCBQt49913+drXvgZAfn4+O3bsoGnTpolzlZeXk5WV9aV93bu+bt26NGjQgMLCQrKyshg7dixPPvkkhYWFhBDYsGEDRUVFFfZzkiRJOhBHpo9hU+YVcO/kjygo3kIElJVHPPnKe0yZVwDAihUraNasGWPHjmXRokW88847bNiwgTfeeAOAKIoA6N69O6+99hqrV6+mbdu23HLLLYlznHbaaYwbN44ePXokRoqzsrJISUmhqKiI4uJiiouL2bBhAwsXLgQghHDA/u4ZhQYoLS1l3bp1NGvWjFmzZvHoo4/yxz/+kfXr11NcXEz9+vUT/ZMk6Wg3btw4Lr/88sR269at6devX2I7KyuL+fPn89Zbb9GhQwfq169Phw4deOuttxL7dO3alZEjR3L++edTt25dLr/8ctauXct1111HvXr16NChA8uXL0/s/2XPG40aNYp+/fpx/fXXk5aWRrt27Zg7d+6R/SFUU4bpY9iY6YvYsmPnPmXr3nuRB/84m3Xr1jF69GiuvvpqNm7cSGpqKunp6axbt44HHnggsf9nn33Gn//8ZzZt2kRKSgp169YlKSlpnzavueYaRo8eTbdu3Vi6dClNmzblkksu4e6772bDhg2Ul5ezdOnSxNSRxo0bs2rVKrZv375POy+99BKzZ89m+/bt3H///XTs2JGsrCw2btxIcnIyGRkZlJWV8cMf/pANGzYcoZ+aJEkVLzc3l1mzZlFeXs7q1avZsWMHb775JgDLli2jtLSUk046iV69ejF06FDWrl3LXXfdRa9evVi7dm2inQkTJvD0009TUFDA0qVLOe+88xg0aBDr1q3jtNNO2+c9/FDPG/35z3+mf//+FBcX07t378TzUtqXYfoYVli85QtldU7P5cMn7qFly5a0bNmSkSNHcuedd7JlyxYaNWrEueeey6WXXprYv7y8nLFjx9KsWTMaNGhAXl4e//3f//2Fdm+44QZ+8IMfcOGFF7J8+XKeeuoptm/fnlg55Morr2T16tUAXHjhhbRr144mTZrQqFGjRBvXXnstDzzwAA0aNOD9999PrBrSvXt3evTowamnnkp2djbHHXfcIaeMSJJ0NNmzKtb8+fPJy8uje/fuZGZm8sknn5CXl8cFF1zAtGnTOOWUUxg4cCDJyclcc801tG3blhdffDHRzqBBg2jVqhX169enR48etGrVim7dupGcnMxVV13FvHnzEvse6nmjzp0707NnT5KSkhg4cCALFiyo1J9JdRGO5n+F5+TkRP5L4cjp9MjrFBwgUGemp/Lm8AuroEeSJB27BgwYwNlnn82SJUvo1KkTCxYsoFWrVrz99tucccYZlJWVMXfuXCZNmpQ4pn///pxxxhmMGDGCrl27MmDAAL7zne8AMHLkSFatWsX48eMBmDFjBoMHD2bJkiUAB3ze6LXXXuOiiy5i1KhRLFmyhGeeeQbY9QnCJ598Mjt27CA5uWY+chdCeD+KopzDPc6R6WPYPd3bkFp73ykZqbWTuKd7myrqkSRJx5Yp8wro9MjrnDx8GnM2N2bCC68wa9YscnNzyc3NJS8vj7y8PHJzc2nWrNk+n7EAu55vyszMPOzz+rxRxTFMH8OuaJ/Jw33OIDM9lcCuEemH+5zBFe0P/0UpSZIOz/4LAWxp1Ib358ymqHgjzZs354ILLuCVV15h7dq1tG/fnp49e7J48WKeffZZysrKmDhxIh9//DGXXXbZYZ/b540qTs0cp9e/7Ir2mYZnSZKqwP4LAdRukEmofRw7G+9aXrZevXq0bNmSjIwMkpKSaNiwIVOnTuWOO+7gu9/9Lq1bt2bq1Kn7PF/0r9r7eaM6deowbNgwnzeKyTnTkiRJVeDk4dM4UAoLwD8e6VXZ3TnmOWdakiSpGmmWnnpY5To6GaYlSZKqgAsB1AzOmZYkSaoCe55ZGjN9EYXFW2iWnso93dv4LFM1Y5iWJEmqIi4EUP05zUOSJEmKyTAtSZIkxWSYliRJkmIyTEuSJEkxGaYlSZKkmAzTkiRJUkyGaUmSJCkmw7QkSZIUk2FakiRJiskwLUmSJMVkmJYkSZJiMkxLkiRJMRmmJUmSpJgM05IkSVJMhmlJkiQpJsO0JEmSFJNhWpIkSYrJMC1JkiTFZJiWJEmSYjJMS5IkSTEZpiVJkqSYDNOSJElSTIZpSZIkKSbDtCRJkhSTYVqSJEmKyTAtSZIkxWSYliRJkmIyTEuSJEkxGaYlSZKkmAzTkiRJUkyGaUmSJCkmw7QkSZIUk2FakiRJiskwLUmSJMVkmJYkSZJiqtQwHUJoEEJ4PoSwKYSQH0K4tjLPL0mSJFWk5Eo+36+A7UBj4CxgWghhQRRFCyu5H5IkSdJXVmkj0yGEOkBf4P4oikqjKJoN/BkYWFl9kCRJkipSZU7zOBXYGUXR4r3KFgDt9t4phHBrCGFuCGHumjVrKrF7kiRJ0uGpzDBdFyjZr6wESNu7IIqix6MoyomiKCcjI6PSOidJkiQdrsoM06VAvf3K6gEbK7EPkiRJUoWpzDC9GEgOIZyyV9mZgA8fSpIkqVqqtDAdRdEmYDLwwxBCnRBCJ+BbwNOV1QdJkiSpIlX2h7b8O5AKfA78Afiuy+JJkiSpuqrUdaajKFoHXFGZ55QkSZKOFD9OXJIkSYrJMC1JkiTFZJiWJEmSYjJMS5IkSTEZpiVJkqSYDNOSJElSTIZpSZIkKSbDtCRJkhSTYVqSJEmKyTAtSZIkxWSYliRJkmIyTEuSJEkxGaYlSZKkmAzTkiRJUkyGaUmSJCkmw7QkSZIUk2FakiRJiskwLUmSJMVkmJYkSZJiMkxLkiRJMRmmJUmSpJgM05IkSVJMhmlJkiQpJsO0JEmSFJNhWpIkSYrJMC1JkiTFZJiWJEmSYjJMS5IkSTEZpo+gUaNGMWDAgKruhiRJko4Qw7QkSZKqTIsWLRgzZgxf//rXqVOnDjfffDOfffYZPXr0IC0tjW7durF+/XoArrrqKpo0aUL9+vXp0qULCxcuTLRz44038r3vfY9evXqRlpZGx44dWbp06RHvv2H6MJSVlVV1FyRJkmqc5557jtdee43Fixfz4osv0qNHD0aPHk1RURHl5eU89thjAPTo0YO///3vfP7555x99tlcd911+7Tzhz/8gf/8z/9k/fr1tG7dmhEjRhzxvhumD6FFixY8+uijib+WZs+ezfnnn096ejpnnnkmM2fOTOz7j3/8g9zcXNLS0rj44ospKiqquo5LkiRVE7fffjuNGzcmMzOTCy64gI4dO9K+fXtSUlL49re/zbx58wC46aabSEtLIyUlhVGjRrFgwQJKSkoS7fTp04dvfOMbJCcnc9111zF//vwj3vfkI36GGuAPf/gD06ZNo1atWnz961/n6aef5tJLL+Uvf/kLffv25ZNPPiEjI4Nrr72W8847j1dffZV33nmHXr168a1vfauquy9JknTUmDKvgDHTF1FYvIVm6als3r6Txo0bJ+pTU1O/sF1aWsrOnTsZMWIEkyZNYs2aNdSqtWtMuKioiPr16wPQpEmTxHHHH388paWlR/x6HJn+FwwdOpSsrCyeeeYZevbsSc+ePalVqxYXX3wxOTk5vPTSS6xYsYL33nuPH/3oR6SkpNClSxcuv/zyqu66JEnSUWPKvALunfwRBcVbiICC4i2s37ydN5cc+r/5zz77LC+88AIzZsygpKSE5cuXAxBF0ZHt9CEYpg9gyrwCOj3yOicPn8anJVtZsS0VgPz8fCZNmkR6enria/bs2axevZrCwkJOOOEE6tSpk2gnOzu7qi5BkiTpqDNm+iK27Ni5T1kUwR/nrjzksRs3biQlJYWGDRuyefNm7rvvviPVzcNimN7P/n8xlZVH/O/s5UyZV0BWVhYDBw6kuLg48bVp0yaGDx9O06ZNWb9+PZs2bUq0tWLFiqq7EEmSpKNMYfGWA5avLd1+yGOvv/56srOzyczM5PTTT+fcc8+t6O7FEqp6aPzL5OTkRHPnzq3Uc3Z65HUK9rrRq/7nJhr2GErrs85jwnWn0KFDB373u9/RrVs3duzYwZw5c2jdujXNmzfn3HPPpXPnzowePZp3332Xnj170rt3b5555plKvQZJkqSj0f45a4/M9FTeHH5hFfTon0II70dRlHO4xzkyvZ+D/cVUWLyFrKwsXnjhBUaPHk1GRgZZWVmMGTOG8vJyYNdcnnfeeYcGDRrwwAMPcP3111dm1yVJko5q93RvQ2rtpH3KUmsncU/3NlXUo6/Oken9HM1/MUmSJFV3+6/mcU/3NlzRPrOquxV7ZNql8fZzT/c23Dv5o30mx1f3v5gkSZKOFle0zzwqwnNFMUzvZ8/NPRr/YpIkSdLRxTB9ADXtLyZJkiQdGT6AKEmSJMVkmJYkSZJiMkxLkiRJMRmmJUmSpJgM05IkSVJMhmlJkiQpJsO0JEmSFJNhWpIkSYrJMC1JkiTFZJiWJEmSYjJMS5IkSTEZpiVJkqSYDNOSJElSTIZpSZIkKSbDtCRJkhRThYTpEMKQEMLcEMK2EML4A9RfFEL4JISwOYTw1xBCdkWcV5IkSapKFTUyXQg8CPzv/hUhhEbAZOB+oAEwF5hYQeeVJEmSqkxyRTQSRdFkgBBCDtB8v+o+wMIoiibt3mcUUBRCaBtF0ScVcX5JkiSpKlTGnOl2wII9G1EUbQKW7i7/ghDCrbunjMxds2ZNJXRPkiRJiqcywnRdoGS/shIg7UA7R1H0eBRFOVEU5WRkZBzxzkmSJElxHTJMhxBmhhCig3zN/hfOUQrU26+sHrAxToclSZKko8Uh50xHUdT1K55jIXDDno0QQh2g1e5ySZIkqdqqqKXxkkMIxwFJQFII4bgQwp6g/jzwbyGEvrv3+QHwoQ8fSpIkqbqrqDnTI8V196gAABH6SURBVIEtwHBgwO7vRwJEUbQG6As8BKwHOgL9K+i8kiRJUpWpqKXxRgGjvqR+BtC2Is4lSZIkHS38OHFJkiQpJsO0JEmSFJNhWpIkSYrJMC1JkiTFZJiWJEmSYjJMS5IkSTEZpiVJkqSYDNOSJElSTIZpSZIkKSbDtCRJkhSTYVqSJEmKyTAtSZIkxWSYliRJkmIyTEuSJEkxGaYlSZKkmAzTkiRJUkyGaUmSJCkmw7QkSZIUk2FakiRJiskwLUmSJMVkmJYkSZJiMkxLkiRJMRmmJUmSpJgM05IkSVJMhukKUlZWVtVdkCRJUiUzTB/CBx98QPv27UlLS+Oqq67i6quvZuTIkcycOZPmzZvz6KOP0qRJEwYNGkR5eTmPPPIIrVq1omHDhvTr149169Yl2pozZw7nn38+6enpnHnmmcycOTNR17VrV+6//346depEWloal1xyCUVFRVVwxZIkSfpXGaa/xPbt2/n2t7/NjTfeyLp167jmmmt4/vnnE/Wffvop69atIz8/n8cff5zHHnuMKVOmkJeXR2FhISeccALf+973ACgoKKBXr16MHDmSdevW8ZOf/IS+ffuyZs2aRHvPPvss48aN4/PPP2f79u385Cc/qfRrliRJqmrz5s3j7LPPJi0tjauvvpr+/fszcuRIxo8fT+fOnffZN4TAkiVLANi2bRvf//73Oemkk2jcuDGDBw9my5YtiX2nTp3KWWedRXp6Oueffz4ffvjh3k2dEUL4fgjhwxBCSQhhYgjhuEP11TD9JebMmUNZWRlDhw6ldu3a9OnTh2984xuJ+lq1avHAAw+QkpJCamoqv/nNb3jooYdo3rw5KSkpjBo1ij/96U+UlZXxzDPP0LNnT3r27EmtWrW4+OKLycnJ4aWXXkq0N2jQIE499VRSU1Pp168f8+fPr4rLliRJqjLbt2/niiuuYODAgaxbt46rrrqK55577l869j/+4z9YvHgx8+fPZ8mSJRQUFPDDH/4Q2DXb4KabbuI3v/kNa9eu5bbbbqN3795s27Zt7yb6AZcCJwNfB2481DkN0/uZMq+ATo+8zsnDp3HL/7zK8SdkEEJI1GdlZSW+z8jI4Ljj/vkHS35+Pt/+9rdJT08nPT2d0047jaSkJD777DPy8/OZNGlSoi49PZ3Zs2ezevXqxPFNmjRJfH/88cdTWlp6hK9WkiTp6DJnzhx27NjBnXfeSe3atbnyyivp0KHDIY+LoognnniCn/3sZzRo0IC0tDTuu+8+JkyYAMATTzzBbbfdRseOHUlKSuKGG24gJSWFOXPm7N3MY1EUFUZRtA54ETjrUOc1TO9lyrwC7p38EQXFW4iAklCX5StW8fwHqxL7rFy5MvH93iEbdgXtl19+meLi4sTX1q1byczMJCsri4EDB+5Tt2nTJoYPH15ZlydJko5RLVq0YMaMGV8ob9eu3T7PcFWlPQOafX8ylQ216vHC/MJEXXZ29iGPX7NmDZs3b+acc85JDFxeeumliSm1+fn5jB07dp+BzZUrV1JYWLh3M5/u9f1moO6hzmuY3suY6YvYsmNnYjslsy0Rtbh71KOUlZXxwgsv8O677x70+MGDBzNixAjy8/OBXTf1hRdeAGDAgAG8+OKLTJ8+nZ07d7J161ZmzpzJqlWrDtqeJEnSkbRw4UK6du1a1d3YZ0AzqW4DtpasYfhzHzJlXgEAK1asAKBOnTps3rw5cdynn/4z+zZq1IjU1FQWLlyYGLgsKSlJ/Kc/KyuLESNG7DOwuXnzZq655pqv1HfD9F4Ki7fssx2SapPx7ftY9c400tPTeeaZZ7jssstISUk54PF33HEHvXv35pJLLiEtLY1zzz2Xd955B9h1A1944QVGjx5NRkYGWVlZjBkzhvLy8iN+XZIkSUezvQc0UzLbQkji8znP8+OXPmby5MmJwcwzzzyThQsXMn/+fLZu3cqoUaMSbdSqVYtbbrmFYcOG8fnnnwO7FoCYPn06ALfccgu//vWveeedd4iiiE2bNjFt2jQ2btz4lfpumN5Ls/TUL5SlND2FDsOepLS0lEmTJlFQUEDz5s3p2rXrF0aVa9WqxV133cWiRYvYuHEjS5cuZfTo0Yn6jh07kpeXx7p161izZg3Tpk3jpJNOAmDmzJl85zvfSex74403Mnv27CN0pZIk6Vj1ySefcPLJJzNhwoR9pn+MGjWKfv36cf3115OWlka7du2YO3du4riDLRdcEfYe0NwzmFn60V+Y88NvMXHiRPr06QPAqaeeyg9+8AO6devGKaec8oWVPR599FFat27NueeeS7169ejWrRuLFi0CICcnhyeeeIIhQ4Zwwgkn0Lp1a8aPH/+V+26Y3ss93duQWjtpn7KocCHfOecEysrK+N3vfseHH37IpZdeWkU9lCRJiu+DDz7gkksu4Re/+AX9+/f/Qv2f//xn+vfvT3FxMb1792bIkCHAoZcL/qr2H9BMaXoKzQY9xnkPTGXixImkpv6zfsSIERQVFbFy5UoGDBhAFEW0bt0agOOOO47Ro0ezbNkyNmzYwN/+9jeGDh2aOPbSSy/lvffeo7i4mNWrVzNp0iTS0tL2VH8URVFiYnkURaOiKBpwqL4bpvdyRftMHu5zBpnpqQQgMz2VXidFDB9wKfXr12fs2LH86U9/omnTplXdVUmSpMMya9Ysevfuze9+9zsuu+yyA+7TuXNnevbsSVJSEgMHDmTBggXAoZcL/qoONKCZWjuJe7q3qbBzHCnJVd2Bo80V7TO5on3mXiUX8t8P/keV9UeSJCmOKfMKGDN9EYXFW/i0ZCv/9cv/5pKLvsk3v/nNgx6z/zK9W7dupaysjMLCQjIzMw+6XPBXtSd77elvs/RU7uneZr9MdnQyTEuSJNUwe1bH2PNQX1l5RN1ug1nwt2kMGzaMn/3sZ4fVXtOmTSkoKCCKokSgXrlyJa1ataqwPn9xQPOfKmJu85HiNA9JkqQaZv/lfgF2JB1Hep9RvPHGG4f9ORfnnXceSUlJ/PKXv/yXlgs+ljgyLUmSVMPsv9zvHp9vS+L9117jm9/8JrVr1/6X2/va177G5MmT+c53vsO9995Ljx49vnS54GNJiKKoqvtwUDk5OdHeS7JIkiTp0Do98joFBwjUmempvDn8wgo5R8eOHRk8eDCDBg2qkPaqWgjh/SiKcg73OKd5SJIk1TBHYnWMvLw8Pv30U5cL3o9hupoZP378PguUhxBYsmRJFfZIkiQdbQ603O/Dfc74SqtjLFq0iDPPPNPlgvfjnGlJkqQa6MtWx4jj1ltv5dZbb62w9moKR6YlSZKkmAzTVWTMmDH07dt3n7Lbb7+dO++8k5KSEm6++WaaNm1KZmYmI0eOZOfOnQdp6Z9KSkq4/vrrycjIIDs7mwcffJDy8nIAsrOzef/99wF45plnCCHw8ccfA/Dkk09yxRVXVPAVSpIk1XyG6SoyYMAAXnnlFYqLiwEoKytj4sSJDBw4kBtuuIHk5GSWLFnCvHnzePXVV3nyyScP2ebtt99OSUkJy5YtIy8vj6eeeopx48YBkJuby8yZMwF44403aNmyJXl5eYnt3NzcI3OhkiRJNZhhuoo0bdqULl26MGnSJABeeeUVGjVqRPPmzXn55Zf5+c9/Tp06dTjxxBMZNmwYEyZM+NL2du7cycSJE3n44YdJS0ujRYsW3H333Tz99NPArjC9JzzPmjWLe++9N7Gdl5dnmJYkSYrBMF3JpswroNMjr3Py8GksSz+Hn//6t8CuqRcDBw4kPz+fHTt20LRpU9LT00lPT+e2227j888//9J2i4qK2L59O9nZ2Ymy7OxsCgoKgF1hetasWXz66afs3LmTq6++mjfffJPly5dTUlLCWWeddeQuWpIkqYZyNY9KNGVeAfdO/ijx8Z5bM89h1ZT/4r8m/YWpU6fy4x//mNq1a5OSkkJRURHJyf/67WnUqBG1a9cmPz+f008/HYAVK1aQmbnrKd7WrVtz/PHH89hjj9GlSxfS0tJo0qQJjz/+OJ07d6ZWLf+ukiRJOlwmqEo0ZvqiRJAGCMlf4/hTz+e+obfwjW98g5NOOommTZtyySWXcPfdd7NhwwbKy8tZunRpYkrGwSQlJdGvXz9GjBjBxo0byc/P56c//SkDBgxI7JObm8svf/nLxJSOrl277rMtSZKkw2OYrkSFB/hYzzpnXMTmT//BwIEDE2VPPfUU27dv5/TTT+eEE07gyiuvZPXq1Yds/xe/+AV16tShZcuWdO7cmWuvvZabbropUZ+bm8vGjRvp0qXLAbclSZJ0eEIURVXdh4PKycmJ5s6dW9XdqDCdHnmdgv0CddmGz1n95HdZX/Q59erVq6KeSZIkHdtCCO9HUZRzuMc5Ml2J7unehtTaSYntKCpn8/sv0LXHFQZpSZKkasgHECvRno/0HDN9Eas+X8+qXw0gs/lJjP/TjCrumSRJkuJwmockSZKOeU7zkCRJkirZVw7TIYSUEMJvQwj5IYSNIYR5IYQe++1zUQjhkxDC5hDCX0MI2QdrT5IkSaouKmJkOhlYCeQC9YH7gT+GEFoAhBAaAZN3lzcA5gITK+C8kiRJUpX6yg8gRlG0CRi1V9HUEMI/gHOA5UAfYGEURZMAQgijgKIQQtsoij75queXJEmSqkqFz5kOITQGTgUW7i5qByzYU787fC/dXS5JkiRVWxUapkMItYHfA7/ba9S5LlCy364lQNpB2rg1hDA3hDB3zZo1Fdk9SZIkqUIdMkyHEGaGEKKDfM3ea79awNPAdmDIXk2UAvt/Ikk9YOOBzhdF0eNRFOVEUZSTkZFx2BckSZIkVZZDzpmOoqjrofYJIQTgt0BjoGcURTv2ql4I3LDXvnWAVvxzGogkSZJULVXUNI//AU4DLo+iaMt+dc8D/xZC6BtCOA74AfChDx9KkiSpuquIdaazgduAs4BPQwilu7+uA4iiaA3QF3gIWA90BPp/1fNKkiRJVe2o/jjxEMIaIL+q+3EMaAQUVXUndMR4f2s272/N5b2t2by/R5/sKIoO+4G9ozpMq3KEEObG+Sx6VQ/e35rN+1tzeW9rNu9vzVHh60xLkiRJxwrDtCRJkhSTYVoAj1d1B3REeX9rNu9vzeW9rdm8vzWEc6YlSZKkmByZliRJkmIyTEuSJEkxGaaPUSGElBDCb0MI+SGEjSGEeSGEHvvtc1EI4ZMQwuYQwl93f0CPqokQwpAQwtwQwrYQwvgD1Ht/q7EQQoMQwvMhhE27X8fXVnWfFM+XvVZ9nVZvh3qv9f7WDIbpY1cysBLIBeoD9wN/DCG0AAghNAIm7y5vAMwFJlZFRxVbIfAg8L/7V3h/a4RfAduBxsB1wP+EENpVbZcU0wFfq75Oa4SDvtd6f2sOH0BUQgjhQ+CBKIqeCyHcCtwYRdH5u+vqsOuTmtpHUfRJVfZThyeE8CDQPIqiG/cq8/5WY7vv13rg36IoWry77GmgIIqi4VXaOcW2/2vV12nNtOe9FmiI97dGcGRaAIQQGgOnAgt3F7UDFuypj6JoE7B0d7mqP+9v9XYqsHNPkN5tAd6/msbXaQ2z33ut97eGMEyLEEJt4PfA7/b6a7guULLfriVAWmX2TUeM97d68/4dG7zPNcgB3mu9vzWEYbqGCiHMDCFEB/mavdd+tYCn2TX3csheTZQC9fZrth6w8Yh3Xof0r97fL+H9rd68f8cG73MNcZD3Wu9vDWGYrqGiKOoaRVE4yFdngBBCAH7LrgeY+kZRtGOvJhYCZ+7Z2D2XqxX/nAaiKvSv3N9D8P5Wb4uB5BDCKXuVnYn3r6bxdVoDfMl7rfe3hjBMH9v+BzgNuDyKoi371T0P/FsIoW8I4TjgB8CHPhRRfYQQknffuyQgKYRwXAgheXe197ca2z23cjLwwxBCnRBCJ+Bb7Br5UjXzJa9VX6c1w8Hea72/NYRh+hi1ey3L24CzgE9DCKW7v64DiKJoDdAXeIhdqwZ0BPpXVX8Vy0hgCzAcGLD7+5Hg/a0h/h1IBT4H/gB8N4oiR7SqpwO+Vn2dVn9f9l7r/a05XBpPkiRJismRaUmSJCkmw7QkSZIUk2FakiRJiskwLUmSJMVkmJYkSZJiMkxLkiRJMRmmJUmSpJgM05IkSVJMhmlJkiQppv8PknoilZ+8Gd8AAAAASUVORK5CYII=\n",
+ "text/plain": [
+ ""
+ ]
+ },
+ "metadata": {
+ "needs_background": "light"
+ },
+ "output_type": "display_data"
+ },
+ {
+ "data": {
+ "image/png": "iVBORw0KGgoAAAANSUhEUgAAAswAAAFoCAYAAABDtK1/AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO3de3gV1b3/8fcioRAgECCRS4hBQFFpq9ggKmjwioBaBLkKFuyxtS1qrbVHBVu03pWe/rQ99UIL3kUsYkEFay0RFLUoiEcrFpQgiSghJBAMhJD5/QHsAsIoNCQE36/nyfPsWWv2zHf2uM2HyZo1IYoiJEmSJO1evdouQJIkSTqQGZglSZKkGAZmSZIkKYaBWZIkSYphYJYkSZJiGJglSZKkGMm1XUCc9PT0qH379rVdhiRJkg5yb775ZlEURRm76zugA3P79u1ZsGBBbZchSZKkg1wIIX9PfQ7JkCRJkmIYmCVJkqQYBmZJkiQphoFZkg5Co0aNYty4cbVdhiQdFAzMkiRJUgwDsyRJkhTDwCxJB4GFCxdy3HHHkZqaypAhQ9i4cWOi74EHHqBTp060aNGC8847j8LCwkTfCy+8QOfOnWnWrBk//vGPyc3NZeLEibVxCJJ0wDIwS1IdV1FRQf/+/Rk5ciTFxcUMGjSIP//5zwC89NJLXHvttTz55JN88sknZGdnM3ToUACKioq44IILuPXWW1mzZg2dO3fm1Vdfrc1DkaQDkoFZkuq41157jc2bN/PTn/6U+vXrc8EFF9CtWzcAHn30US6++GKOO+44GjRowK233sr8+fNZvnw5zz33HF26dGHAgAEkJydz+eWX07p161o+Gkk68BiYJamOmr6wgB63vcTAu2ayrl5Tnln076EW2dnZABQWFiZeAzRp0oSWLVtSUFBAYWEhWVlZib4QAu3atau5A5CkOsLALEl10PSFBVw77R0KSspJatKCjaWruebPi5m+sACAFStWANC2bVvy8//9tNcNGzawZs0aMjMzadOmDStXrkz0RVG007IkaSsDsyTVQXfOXkL55i0ANMg8EkISn732NHc89x7Tpk3jjTfeAGD48OFMmjSJRYsWsWnTJq677jq6d+9O+/bt6devH++88w7Tp0+nsrKS3//+96xatao2D0uSDkgGZkmqgwpLyhOvQ1J9Ms6/jrJ3/sZrN36XKVOmMGDAAABOP/10fv3rXzNw4EDatGnDsmXLeOKJJwBIT09n6tSp/OIXv6Bly5a899575OTk0KBBg1o5Jkk6UIUoimq7hj3KycmJFixYUNtlSNIBp8dtL1GwQ2jeLjMthVeuOW2ftllVVUW7du149NFHOfXUU//TEiWpTgkhvBlFUc7u+rzCLEl10NW9O5NSP2mntpT6SVzdu/NebWf27NmUlJSwadMmbrnlFqIo4oQTTqjOUiWpzkuu7QIkSXuvf9dMYOtY5sKSctqmpXB1786J9q9q/vz5DB8+nIqKCo4++mimT59OSkrK/ihZkuosh2RIkiTpa88hGZIkSdI+MjBLkiRJMQzMkiRJUgwDsyRJkhTDwCxJkiTFMDBLkiRJMQzMkiRJUgwDsyRJkhTDwCxJkiTFMDBLkiRJMQzMkiRJUgwDsyRJkhTDwCxJkiTFMDBLkiRJMQzMkiRJUgwDsyRJkhTDwCxJkiTFMDBLkiRJMQzMkiRJUgwDs/Q1UVlZWdslSJJUJxmYpTrurbfeomvXrqSmpjJo0CCGDBnCuHHjmDNnDu3ateP222+ndevWjB49mqqqKm677TY6duxIy5YtGTx4MMXFxYltvfbaa5x00kmkpaVxzDHHMGfOnERfr169uP766+nRowepqamcddZZFBUV1cIRS5JUswzMUh1WUVHB+eefz6hRoyguLmbYsGE8/fTTif5Vq1ZRXFxMfn4+999/P3fffTfTp08nLy+PwsJCmjdvzk9+8hMACgoK6NevH+PGjaO4uJi77rqLgQMHsnr16sT2HnvsMSZNmsRnn31GRUUFd911V40fsyRJNc3ALNVhr732GpWVlVx++eXUr1+fAQMGcPzxxyf669Wrxw033ECDBg1ISUnhvvvu4+abb6Zdu3Y0aNCA8ePH89RTT1FZWckjjzxC37596du3L/Xq1ePMM88kJyeH5557LrG90aNHc8QRR5CSksLgwYNZtGhRbRy2JEk1Krm2C5C07woLC8nMzCSEkGjLyspKvM7IyKBhw4aJ5fz8fM4//3zq1fv3v5WTkpL49NNPyc/PZ+rUqcyYMSPRt3nzZk499dTEcuvWrROvGzVqRFlZWbUfkyRJBxoDs1THTF9YwJ2zl1BYUk7j4k/4ZPkKoihKhOaPP/6Yjh07AuwUpGFrmP7Tn/5Ejx49vrDdrKwsRo4cyQMPPLD/D0KSpDrEIRlSHTJ9YQHXTnuHgpJyImB9s8NYt6mKH1xzE5WVlTzzzDO88cYbe3z/pZdeytixY8nPzwdg9erVPPPMMwCMGDGCGTNmMHv2bLZs2cLGjRuZM2cOK1eurIlDkyTpgGVgluqQO2cvoXzzlsRySKpPev/reOzhB0lLS+ORRx7hnHPOoUGDBrt9/xVXXMF5553HWWedRWpqKieccAKvv/46sPUK8zPPPMMtt9xCRkYGWVlZ3HnnnVRVVdXIsUmSdKAKURTVdg17lJOTEy1YsKC2y5AOGIdd8yy7+8YG4KPb+gHQvXt3Lr30UkaPHl2jtUmSVJeFEN6Moihnd31eYZbqkLZpKV9o27jiHdKTyqmsrOTBBx9k8eLFnH322bVQnSRJB6dqC8whhAYhhD+GEPJDCOtDCAtDCH126D89hPB+COHzEMLfQwjZ1bVv6evi6t6dSamftFNbKC3kX/f+iGbNmjFhwgSeeuop2rRpU0sVSpJ08KnOWTKSgY+BXGAF0Bd4MoTwLaAMmAb8FzAD+DUwBTihGvcvHfT6d80ESMyS0TYthatv+m/6d727liuTJOngtV/HMIcQFgM3AC2BUVEUnbStvTFQBHSNouj9Pb3fMcySJEmqCbUyhjmE0Ao4AngX6AK8vb0viqINwLJt7ZIkSdIBa78E5hBCfeBR4MFtV5CbAKW7rFYKpO7mvT8IISwIISxYvXr1/ihPkiRJ+sqqPTCHEOoBDwMVwJhtzWVA011WbQqs3/X9URTdH0VRThRFORkZGdVdniRJkrRXqjUwh63P4f0j0AoYGEXR5m1d7wLH7LBeY6DjtnZJkiTpgFXdV5j/ABwFnBtFUfkO7U8D3wwhDAwhNAR+CSyOu+FPkiTFmzRpEueee25iuVOnTgwePDixnJWVxaJFi3j11Vfp1q0bzZo1o1u3brz66quJdXr16sW4ceM46aSTaNKkCeeeey5r1qzhwgsvpGnTpnTr1o3ly5cn1r/iiivIysqiadOmfOc732Hu3LmJvvHjxzN48GAuuugiUlNT6dKlC968r4NBdc7DnA38EDgWWBVCKNv2c2EURauBgcDNwFqgOzC0uvYtSdLXUW5uLnPnzqWqqopPPvmEzZs388orrwDw4YcfUlZWxqGHHkq/fv24/PLLWbNmDT/72c/o168fa9asSWzniSee4OGHH6agoIBly5Zx4oknMnr0aIqLiznqqKO44YYbEut269aNRYsWUVxczPDhwxk0aBAbN25M9P/lL39h6NChlJSUcN555zFmzBikuq7aAnMURflRFIUoihpGUdRkh59Ht/W/GEXRkVEUpURR1CuKouXVtW9Jkr6OOnToQGpqKosWLSIvL4/evXuTmZnJ+++/T15eHieffDLPPvsshx9+OCNHjiQ5OZlhw4Zx5JFHMmPGjMR2Ro8eTceOHWnWrBl9+vShY8eOnHHGGSQnJzNo0CAWLlyYWHfEiBG0bNmS5ORkrrrqKjZt2sSSJUsS/T179qRv374kJSUxcuRI3n77baS6rjofXCJJkmrA9IUFiQcYbUzvzD2PPkNK+Wpyc3NJS0sjLy+P+fPnk5ubS2FhIdnZOz9cNzs7m4KCgsRyq1atEq9TUlK+sFxWVpZYnjBhAhMnTqSwsJAQAuvWraOoqCjR37p168TrRo0asXHjRiorK0lONnKo7tpv8zBLkqTqN31hAddOe4eCknIioKrVUTw1YzbP/vUlcnNzyc3NJS8vj7y8PHJzc2nbti35+fk7bWPFihVkZmbu9b7nzp3L7bffzpNPPsnatWspKSmhWbNm7M+HoEkHAgOzJEl1yJ2zl1C+eUtiueGh32JD/mI+K15Pu3btOPnkk5k1axZr1qyha9eu9O3blw8++IDHHnuMyspKpkyZwnvvvcc555yz1/tev349ycnJZGRkUFlZyY033si6deuq8/CkA5KBWZKkOqSwpHyn5fotMqlXvyHJbY8GoGnTpnTo0IEePXqQlJREy5YtmTlzJhMmTKBly5bccccdzJw5k/T09L3ed+/evenTpw9HHHEE2dnZNGzYkKysrGo5LulAFg7kP6Pk5ORETkcjSdK/9bjtJQp2Cc0AmWkpvHLNabVQkXRwCCG8GUVRzu76vMIsSVIdcnXvzqTUT9qpLaV+Elf37lxLFUkHP29ZlSSpDunfdevNettnyWiblsLVvTsn2iVVPwOzJEl1TP+umQZkqQY5JEOSJEmKYWCWJEmSYhiYJUmSpBgGZkmSJCmGgVmSJEmKYWCWJEmSYhiYJUmSpBgGZkmSJCmGgVmSJEmKYWCWJEmSYhiYJUmSpBgGZkmSJCmGgVmSJEmKYWCWJEmSYhiYJUmSpBgGZkmSJCmGgVmSJEmKYWCWJEmSYhiYJUmSpBgGZkmSJCmGgVmSJEmKYWCWJEmSYhiYJUmSpBgGZkmSJCmGgVmSJEmKYWCWJEmSYhiYJUmSpBgGZkmSJCmGgVmSJEmKYWCWJEmSYhiYJUmSpBgGZkmSJCmGgVmSJEmKYWCWJEmSYhiYJUmSpBgGZkmSJCmGgVmSJEmKYWCWJEmSYhiYJUmSpBgGZkmSJCmGgVmSJEmKYWCWJEmSYhiYJUmSpBgGZkmSJCmGgVmSJEmKYWCWJEmSYhiYJUmSpBgGZkmSJCmGgVmSJEmKYWCWJEmSYhiYJUmSpBjVGphDCGNCCAtCCJtCCJN36Ts9hPB+COHzEMLfQwjZ1blvSZIkaX+o7ivMhcBNwJ92bAwhpAPTgOuBFsACYEo171uSJEmqdsnVubEoiqYBhBBygHY7dA0A3o2iaOq2/vFAUQjhyCiK3q/OGiRJkqTqVFNjmLsAb29fiKJoA7BsW7skSZJ0wKqpwNwEKN2lrRRI3XXFEMIPto2DXrB69eoaKU6SJEnak5oKzGVA013amgLrd10xiqL7oyjKiaIoJyMjo0aKkyRJkvakpgLzu8Ax2xdCCI2BjtvaJUmSpANWdU8rlxxCaAgkAUkhhIYhhGTgaeCbIYSB2/p/CSz2hj9JkiQd6Kr7CvM4oBy4Bhix7fW4KIpWAwOBm4G1QHdgaDXvW5IkSap21T2t3Hhg/B76XgSOrM79SZIkSfubj8aWJEmSYhiY9ZW1b9+eF198sdq2N3nyZHr27Fkt2xo1ahTjxo3bp/eOHz+eESNGALB8+XJCCFRWVlZLXZIkqe4zMKvOqc6gLUmS9GUMzJIkSVIMA7P2yj/+8Q+OPvpomjdvzujRo9m4cSNr167lnHPOISMjg+bNm3POOeewcuXKxHsmT55Mhw4dSE1N5bDDDuPRRx/d7bavvvpqevbsSWlpKaWlpXz/+9+nTZs2ZGZmMm7cOLZs2cI///lPLr30UubPn0+TJk1IS0tLvL+oqIgzzzyT1NRUcnNzyc/PT/RdccUVZGVl0bRpU77zne8wd+7c/fchSZKkg4qBWXvl0UcfZfbs2SxbtowPPviAm266iaqqKkaPHk1+fj4rVqwgJSWFMWPGALBhwwYuv/xynn/+edavX8+rr77Kscceu9M2q6qquOSSS1i8eDEvvPACzZo143vf+x7JycksXbqUhQsX8sILLzBx4kSOOuoo7r33Xk488UTKysooKSnZqbbrr7+eoqIijj32WC688MJEX7du3Vi0aBHFxcUMHz6cQYMGsXHjxpr50CRJUp1mYNZeGTNmDFlZWbRo0YKxY8fy+OOP07JlSwYOHEijRo1ITU1l7Nix5OXlJd5Tr149/u///o/y8nLatGlDly5dEn2bN29m2LBhFBcXM2PGDBo1asSnn37K888/z29/+1saN27MIYccwpVXXskTTzwRW1u/fv045ZRTaNCgATfffDPz58/n448/BmDEiBG0bNmS5ORkrrrqKjZt2sSSJUv2z4ckSdJXsGTJErp27Upqaip33313tW33P7nXZ86cObRr1y6xXN03/NdV1ToPsw5+WVlZidfZ2dkUFhby+eefc+WVVzJr1izWrl0LwPr169myZQuNGzdmypQp3HXXXXz/+9+nR48eTJgwgSOP3Dol99KlS3n77bd54403+MY3vgFAfn4+mzdvpk2bNol9VVVV7bTvL6utSZMmtGjRgsLCQrKyspgwYQITJ06ksLCQEALr1q2jqKio2j4XSZL21h133EGvXr1YuHDhPm9j+fLlHHbYYWzevJnkZGPd/uIVZsWavrCAHre9xGHXPMuq0o3MePWdRN+KFSto27YtEyZMYMmSJbz++uusW7eOl19+GYAoigDo3bs3f/3rX/nkk0848sgjueSSSxLbOOqoo5g0aRJ9+vRJXPHNysqiQYMGFBUVUVJSQklJCevWrePdd98FIISw21q3X00GKCsro7i4mLZt2zJ37lxuv/12nnzySdauXUtJSQnNmjVL1CdJUm3Iz8/f6a+uOnAZmLVH0xcWcO20dygoKScCKqsiJj9wHxNn/YPi4mJuueUWhgwZwvr160lJSSEtLY3i4mJuuOGGxDY+/fRT/vKXv7BhwwYaNGhAkyZNSEpK2mk/w4YN45ZbbuGMM85g2bJltGnThrPOOourrrqKdevWUVVVxbJlyxLDPFq1asXKlSupqKjYaTvPPfcc8+bNo6Kiguuvv57u3buTlZXF+vXrSU5OJiMjg8rKSm688UbWrVu33z8/SZL25LTTTuPvf/87Y8aMoUmTJrz99ttcdNFFZGRkkJ2dnbhHCLb+lfWmm24iOzubQw45hIsuuojS0lIATjnlFADS0tJo0qQJ8+fPB7ZetLrsssto1qwZRx55JH/7298S+540aRJHHXUUqampdOjQgfvuu6+Gj77uMTBrj+6cvYTyzVt2aks56hQu/94FdOjQgQ4dOjBu3Dh++tOfUl5eTnp6OieccAJnn312Yv2qqiomTJhA27ZtadGiBXl5efzv//7vF/b1ve99j1/+8pecdtppLF++nIceeoiKiorEjBwXXHABn3zyCbD1fzJdunShdevWpKenJ7YxfPhwbrjhBlq0aMGbb76ZmI2jd+/e9OnThyOOOILs7GwaNmz4pcM7JEnan1566SVOPvlkfve731FWVsaECRMoLS3lww8/JC8vj4ceeohJkyYBW8ckT548mb///e98+OGHlJWVJW6u3/5X3ZKSEsrKyjjxxBMBeP311+nQoQNFRUXccMMNDBgwgOLiYgAOOeQQZs6cybp165g0aRJXXnklb731Vi18CnVHOJD/LJ2TkxMtWLCgtsv42jrsmmfZ3X8dAfjotn41XY4kSQeVXr16MWLECEaPHk2jRo1YuHAhRx99NAD33Xcfjz/+OHPmzOH0009n4MCB/PjHPwa23iz4zW9+k/LyclauXPmFMcyTJ0/muuuuo6CgIDGM8fjjj+eyyy5j5MiRX6ijf//+nHrqqVxxxRXMmTOHESNGJKaHbd++PRMnTuSMM86oiY+kVoUQ3oyiKGd3fV5h1h61TUvZq3ZJkrT3ioqKqKioIDs7O9GWnZ1NQUEBAIWFhV/oq6ys5NNPP93jNjMzM3e652f7jfoAzz//PCeccAItWrQgLS2N5557zhvhv4SBWXt0de/OpNTfebxxSv0kru7duZYqkiSpbtvxZvqFK0pYuGIt6enp1K9ff6cHbq1YsYLMzEwA2rZt+4W+5ORkWrVqtccb4QsKCna6uX37jfqbNm1i4MCB/PznP+fTTz+lpKSEvn37eiP8lzAwa4/6d83k1gHfIjMthQBkpqVw64Bv0b9rZm2XJklSnbPrzfSbKrcw7a0CZixexeDBgxk7dizr168nPz+f3/zmN4wYMQLYenP8//zP//DRRx9RVlbGddddx5AhQxI3tNerV48PP/xwp3199tln3H333WzevJmpU6fyz3/+k759+1JRUcGmTZvIyMggOTmZ559/nhdeeKEWPo26xQn7FKt/10wDsiRJ1WB3N9NXbKniztlLmHnPPVx22WV06NCBhg0bcskll3DxxRcDcPHFF1NYWMgpp5zCxo0b6d27N/fccw8AjRo1YuzYsfTo0YPNmzcza9YsALp3786//vUv0tPTadWqFU899RQtW7YE4O6772bw4MFs2rSJc889l/POO68GP4W6yZv+JEmSaoA30x/YvOlPkiSplnkzfd1lYJYkSaoB3kxfdzmGWZIkqQZsvyfoztlLKCwpp21aClf37uy9QnWAgVmSJKmGeDN93eSQDEmSJCmGgVmSJEmKYWCWJEmSYhiYJUmSpBgGZkmSJCmGgVmSJEmKYWCWJEmSYhiYJUmSpBgGZkmSJCmGgVmSJEmKYWCWJEmSYhiYJUmSpBgGZkmSJCmGgVmSJEmKYWCWJEmSYhiYJUmSpBgGZkmSJCmGgVmSJEmKYWCWJEmSYhiYJUmSpBgGZkmSJCmGgVmSJEmKYWCWJEmSYhiYJUmSpBgGZkmSJCmGgVmSJEmKYWCWJEmSYhiYJUmSpBgGZkmSJCmGgVmSJEmKYWCWJEmSYhiYJUmSpBgGZkmSJCmGgVmSJEmKYWCWJEmSYhiYJUmSpBgGZkmSJCmGgVmSJEmKUaOBOYTQIoTwdAhhQwghP4QwvCb3L0mSJO2t5Bre3++BCqAVcCzwbAjh7SiK3q3hOiRJkqSvpMauMIcQGgMDgeujKCqLomge8BdgZE3VIEmSJO2tmhyScQSwJYqiD3ZoexvosuNKIYQfhBAWhBAWrF69ugbLkyRJkr6oJgNzE6B0l7ZSIHXHhiiK7o+iKCeKopyMjIwaK06SJEnanZoMzGVA013amgLra7AGSZIkaa/UZGD+AEgOIRy+Q9sxgDf8SZIk6YBVY4E5iqINwDTgxhBC4xBCD+C7wMM1VYMkSZK0t2r6wSU/BlKAz4DHgR85pZwkSZIOZDU6D3MURcVA/5rcpyRJkvSf8NHYkiRJUgwDsyRJkhTDwCxJkiTFMDBLkiRJMQzMkiRJUgwDsyRJkhTDwCxJkiTFMDBLkiRJMQzMkiRJUgwDsyRJkhTDwCxJkiTFMDBLkiRJMQzMkiRJUgwDsyRJkhTDwCxJkiTFMDBLkiRJMQzMkiRJUgwDsyRJkhTDwCxJkiTFMDBLkiRJMQzMkiRJUgwDsyRJkhTDwCxJkiTFMDBLkiRJMQzMkiRJUgwDsyRJkhTDwCxJkiTFMDDXgsmTJ9OzZ8/EcgiBpUuX1mJFkiRJ2hMDsyRJkhTDwCxJkiTFMDD/B+68804GDhy4U9tll13GT3/6U0pLS/n+979PmzZtyMzMZNy4cWzZsuVLt1laWspFF11ERkYG2dnZ3HTTTVRVVQGQnZ3Nm2++CcAjjzxCCIH33nsPgIkTJ9K/f/9qPkJJkiQZmP8DI0aMYNasWZSUlABQWVnJlClTGDlyJN/73vdITk5m6dKlLFy4kBdeeIGJEyd+6TYvu+wySktL+fDDD8nLy+Ohhx5i0qRJAOTm5jJnzhwAXn75ZTp06EBeXl5iOTc3d/8cqCRJ0teYgfk/0KZNG0455RSmTp0KwKxZs0hPT6ddu3Y8//zz/Pa3v6Vx48YccsghXHnllTzxxBOx29uyZQtTpkzh1ltvJTU1lfbt23PVVVfx8MMPA1sD8/aAPHfuXK699trEcl5enoFZkiRpPzAw74PpCwvocdtLHHbNs3yY9h1+e+8fga3DJEaOHEl+fj6bN2+mTZs2pKWlkZaWxg9/+EM+++yz2O0WFRVRUVFBdnZ2oi07O5uCggJga2CeO3cuq1atYsuWLQwZMoRXXnmF5cuXU1payrHHHrv/DlqSJOlrKrm2C6hrpi8s4Npp71C+eet45I2Z32Hl9P/H/5v6N2bOnMkdd9xB/fr1adCgAUVFRSQnf/WPOD09nfr165Ofn8/RRx8NwIoVK8jMzASgU6dONGrUiLvvvptTTjmF1NRUWrduzf3330/Pnj2pV89//0iSJFU3E9ZeunP2kkRYBgjJ36DRESdx3eWXcPzxx3PooYfSpk0bzjrrLK666irWrVtHVVUVy5YtSwyf2JOkpCQGDx7M2LFjWb9+Pfn5+fzmN79hxIgRiXVyc3P53e9+lxh+0atXr52WJUmSVL0MzHupsKT8C22Nv3U6n6/6iJEjRybaHnroISoqKjj66KNp3rw5F1xwAZ988smXbv+ee+6hcePGdOjQgZ49ezJ8+HAuvvjiRH9ubi7r16/nlFNO2e2yJElSXde+fXtefPHFL7R36dIlMQFCTQpRFNX4Tr+qnJycaMGCBbVdxk563PYSBbuE5sp1n/HJxB+xtugzmjZtWkuVSZIkHRzat2/PxIkTOeOMM2psnyGEN6Moytldn1eY99LVvTuTUj8psRxFVXz+5jP06tPfsCxJknQQMjDvpf5dM7l1wLfITEshqtjIyt8OpknRP5n8uztruzRJkqSDzvvvv89hhx3GE088sdNQjfHjxzN48GAuuugiUlNT6dKlCzuOTHjrrbfo2rUrqampDBo0iCFDhjBu3Lh9qsHAvA/6d83klWtOI/83A9myqZwVy5aQlZVV22VJkiQdVN566y3OOuss7rnnHoYOHfqF/r/85S8MHTqUkpISzjvvPMaMGQNARUUF559/PqNGjaK4uJhhw4bx9NNP73MdBmZJkiQdcObOnct5553Hgw8+yDnnnLPbdXr27Enfvn1JSkpi5MiRvP322wC89tprVFZWcvnll1O/fn0GDBjA8ccfv8+1GJglSZJ0wLn33ns56aSTOPXUU/e4TuvWrROvGzVqxIc0EO8AABDcSURBVMaNG6msrKSwsJDMzExCCIn+/2Q0gIFZkiRJtW7HJymvKt3I6F/czIoVK7jyyiv3eltt2rShoKCAHWeD+/jjj/e5NgOzJEmSatX2JykXlJQTAZVVEY+9tZrL7vgTL7/8Mtdcc81ebe/EE08kKSmJ3/3ud1RWVvLMM8/wxhtv7HN9PhpbkiRJtWrXJykDbNpSxf++uoq//vWvnHrqqdSvX/8rb+8b3/gG06ZN47/+67+49tpr6dOnD+eccw4NGjTYp/p8cIkkSZJq1WHXPMvuEmkAPrqtX7Xso3v37lx66aWMHj16t/0+uESSJEkHrLZpKXvV/lXk5eWxatUqKisrefDBB1m8eDFnn332Pm3LwCxJkqRateuTlAFS6idxde/O+7zNJUuWcMwxx9CsWTMmTJjAU089RZs2bfZpWw7JkCRJUq2bvrCAO2cvobCknLZpKVzduzP9u2bW2P7jhmR4058kSZJqXf+umTUakPeGQzIkSZKkGAZmSZIkKYaBWZIkSYphYJYkSZJiGJglSZKkGAZmSZIkKYaBWZIkSYphYJYkSZJiVEtgDiGMCSEsCCFsCiFM3k3/6SGE90MIn4cQ/h5CyK6O/UqSJEn7W3VdYS4EbgL+tGtHCCEdmAZcD7QAFgBTqmm/kiRJ0n5VLY/GjqJoGkAIIQdot0v3AODdKIqmbltnPFAUQjgyiqL3q2P/kiRJ0v5SE2OYuwBvb1+IomgDsGxb+xeEEH6wbXjHgtWrV9dAeZIkSdKe1URgbgKU7tJWCqTubuUoiu6PoigniqKcjIyM/V6cJEmSFOdLA3MIYU4IIdrDz7yvsI8yoOkubU2B9ftSsCRJklSTvnQMcxRFvf7DfbwLfG/7QgihMdBxW7skSZJ0QKuuaeWSQwgNgSQgKYTQMISwPYw/DXwzhDBw2zq/BBZ7w58kSZLqguoawzwOKAeuAUZsez0OIIqi1cBA4GZgLdAdGFpN+5UkSZL2q+qaVm48MD6m/0XgyOrYlyRJklSTfDS2JEmSFMPALEmSJMUwMEuSJEkxDMySJElSDAOzJEmSFMPALEmSJMUwMEuSJEkxDMySJElSDAOzJEmSFMPALEmSJMUwMEuSJEkxDMySJElSDAOzJEmSFMPALEmSJMUwMEuSJEkxDMySJElSDAOzJEmSFMPALEmSJMUwMEuSJEkxDMySJEmqk8aPH8+IESP2+34MzJIkSVIMA7MkSZIOCJWVlbVdwm4ZmCVJklRr2rdvz+233863v/1tGjduzLx58zjppJNIS0vjmGOOYc6cOYl1P/roI3Jzc0lNTeXMM8+kqKioRmpMrpG9SJIkSXvw+OOP8+yzz1KvXj2+/e1v8/DDD3P22Wfzt7/9jYEDB/L++++TkZHB8OHDOfHEE3nhhRd4/fXX6devH9/97nf3e31eYZYkSVKtuvzyy8nKyuKRRx6hb9++9O3bl3r16nHmmWeSk5PDc889x4oVK/jHP/7Br3/9axo0aMApp5zCueeeWyP1GZglSZK0R7169WLixImJ5XHjxpGenk7r1q33eZvTFxbQ47aXOOyaZ1lVupEVm1IAyM/PZ+rUqaSlpSV+5s2bxyeffEJhYSHNmzencePGie1kZ2fv+4HtBYdkSJIk6Sv5+OOPmTBhAvn5+RxyyCH7tI3pCwu4dto7lG/eAkBlVcSf5i3n2BMKyMrKYuTIkTzwwANfeF9+fj5r165lw4YNidC8YsUKQgj7fkBfkVeYJUmS9JXk5+fTsmXLfQ7LAHfOXpIIy9tt2lLFnbOXMGLECGbMmMHs2bPZsmULGzduZM6cOaxcuZLs7GxycnL41a9+RUVFBfPmzWPGjBn/6SF9JQZmSZKkg9SkSZN2GufbqVMnBg8enFjOyspi0aJFvPrqq3Tr1o1mzZrRrVs3Xn311S9s68UXX+TMM8+ksLCQJk2aMGrUqH2qqbCkfI/tWVlZPPPMM9xyyy1kZGSQlZXFnXfeSVVVFQCPPfYYr7/+Oi1atOCGG27goosu2qca9pZDMiRJkg5Subm5XHnllVRVVfHpp5+yefNmXnnlFQA+/PBDysrKOPTQQ+nYsSN33303w4YNY+rUqfTr14+lS5fSsmXLxLbOOOMMnn/+eUaMGMHKlSv3uaa2aSkU7BCa2/3oT4l2gO7du5OXl7fb93bo0IG5c+fu8773lVeYJUmSDlIdOnQgNTWVRYsWkZeXR+/evcnMzOT9998nLy+Pk08+mWeffZbDDz+ckSNHkpyczLBhwzjyyCP323CHq3t3JqV+0k5tKfWTuLp35/2yv+rgFWZJkqSDWG5uLnPmzGHp0qXk5uaSlpZGXl4e8+fPJzc3l8LCwi/MNpGdnU1BQcF+qad/10xg61jmwpJy2qalcHXvzon2A5GBWZIk6SAzfWFBIpAmfd6KD56ZRXnxJ1x33XWkpaXx6KOPMn/+fMaMGcM///lP8vPzd3r/ihUrOPvss/dbff27Zh7QAXlXDsn4EgsXLuS4444jNTWVIUOGMHToUMaNG8fkyZPp2bPnTuuGEFi6dCkAmzZt4uc//zmHHnoorVq14tJLL6W8/N/jdWbOnMmxxx5LWloaJ510EosXL070tW/fnrvuuotvf/vbNGvWjCFDhrBx48aaOWBJklSnbZ+2raCknAgoT+/Mm6/No6hkPe3atePkk09m1qxZrFmzhq5du9K3b18++OADHnvsMSorK5kyZQrvvfce55xzTm0fygHDwByjoqKC/v37M3LkSIqLixk0aBB//vOfv9J7//u//5sPPviARYsWsXTpUgoKCrjxxhsBeOutt7j44ou57777WLNmDT/84Q8577zz2LRpU+L9Tz75JLNmzeKjjz5i8eLFTJ48eX8coiRJOsjsOm1b/RaZhPoN2dLqSACaNm1Khw4d6NGjB0lJSbRs2ZKZM2cyYcIEWrZsyR133MHMmTNJT0+vrUM44IQoimq7hj3KycmJFixYUGv7f/nllxk6dCgFBQWJSbFPOukkTjvtNDp16sTEiROZN29eYv0QAv/617/o2LEjTZo0YfHixXTs2BGA+fPnM3z4cD766CN+9KMfkZ6ezq9//evEezt37sz9999Pbm4u7du356abbmLEiBEA/OIXv2DdunXce++9NXj0kiSpLjrsmmfZXboLwEe39avpcuqMEMKbURTl7K7PMcy72HHMzzfy59O4+SE7PUHmqzyCcfXq1Xz++ed85zvfSbRFUcSWLVv/tZefn8+DDz7IPffck+ivqKigsLAwsbzj4yYbNWq0U58kSdKe7Dpt247t2jcOydjBrmN+SkMTlq/4mKff+vdcgytWrACgcePGfP7554n2VatWJV6np6eTkpLCu+++S0lJCSUlJZSWllJWVgZsnSR87Nixib6SkhI+//xzhg0bVjMHKkmSDlp1cdq2A52BeQe7jvlpkHkkUUjiZ7+6lcrKSqZNm8Ybb7wBwDHHHMO7777LokWL2LhxI+PHj0+8r169elxyySVceeWVfPbZZwAUFBQwe/ZsAC655BLuvfdeXn/9daIoYsOGDTz77LOsX7++5g5WkiQdlPp3zeTWAd8iMy2FAGSmpXDrgG/VqVkpDjQG5h3s+qjGkFSfjPOvo/CNWTRv3pwpU6YwYMAAAI444gh++ctfcsYZZ3D44Yd/YcaM22+/nU6dOnHCCSfQtGlTzjjjDJYsWQJATk4ODzzwAGPGjKF58+Z06tTJm/okSVK16d81k1euOY2PbuvHK9ecZlj+D3nT3w563PbSbsf8ZKal8Mo1pwEwatQo2rVrx0033VQt+2zfvj0/+clPePjhh1m2bBlDhw7llltuYdSoUcybN4/u3bszdepUmjdvzqBBg5g7dy7l5eUcc8wx/OEPf6BLly6Juho3bszy5ct5+eWXOfroo3nssccSNx1KkiRpz+Ju+vMK8w5qa8zPn//8Z/7617/ywQcfMGPGDPr06cMtt9xCUVERVVVV3H333QD06dOHf/3rX3z22Wccd9xxXHjhhTtt5/HHH+dXv/oVa9eupVOnTowdO3a/1i1JkvR14CwZO6itRzVedtlltGrVCoCTTz6ZQw45hK5duwJw/vnn87e//Q2Aiy++OPGe8ePH07x5c0pLS2nWrBkAAwYM4Pjjjwfgwgsv5Gc/+9l+rVuSJOnrwMC8iy97VGN1jDXeceq6VaUbWbr+31e1U1JSEuF5+3JZWRlbtmxh7NixTJ06ldWrV1Ov3tY/DhQVFSUC865T0W2flUOSJEn7ziEZNWzXqesqqyL+OPcjpi8siH3fY489xjPPPMOLL75IaWkpy5cvB7bO7yxJkqT9x8Bcw3adug5g05Yq7py9JPZ969evp0GDBrRs2ZLPP/+c6667bn+WKUmSpG0MzDVs16nrvqx9u4suuojs7GwyMzM5+uijOeGEE/ZHeZIkSdqF08rVsK8ydZ0kSZJqltPKHUB8XKUkSVLd4iwZNay2pq6TJEnSvjEw14Ivm7pOkiRJBw6HZEiSJEkxDMySJElSDAOzJEmSFMPALEmSJMUwMEuSJEkxDMySJElSDAOzJEmSFMPALEmSJMUwMEuSJEkxDMySJElSjBBFUW3XsEchhNVAfm3X8TWWDhTVdhGqdp7Xg5Pn9eDkeT04eV4PTNlRFGXsruOADsyqXSGEBVEU5dR2HapenteDk+f14OR5PTh5Xuseh2RIkiRJMQzMkiRJUgwDs+LcX9sFaL/wvB6cPK8HJ8/rwcnzWsc4hlmSJEmK4RVmSZIkKYaBWZIkSYphYNZOQggNQgh/DCHkhxDWhxAWhhD67LLO6SGE90MIn4cQ/h5CyK6tevXVhRDGhBAWhBA2hRAm76bf81oHhRBahBCeDiFs2Pa9HV7bNWnvxX0//W7WTV/2+9TzWrcYmLWrZOBjIBdoBlwPPBlCaA8QQkgHpm1rbwEsAKbURqHaa4XATcCfdu3wvNZpvwcqgFbAhcAfQghdarck7YPdfj/9btZpe/x96nmte7zpT18qhLAYuCGKoj+HEH4AjIqi6KRtfY3Z+rSirlEUvV+bdeqrCSHcBLSLomjUDm2e1zpo23laC3wziqIPtrU9DBREUXRNrRanfbLr99Pv5sFl++9ToCWe1zrFK8yKFUJoBRwBvLutqQvw9vb+KIo2AMu2tavu8rzWTUcAW7aH5W3exvN2MPG7eZDY5fep57WOMTBrj0II9YFHgQd3+BdvE6B0l1VLgdSarE3VzvNaN3neDn6e44PAbn6fel7rGAPz10wIYU4IIdrDz7wd1qsHPMzWsZFjdthEGdB0l802Bdbv9+K1R1/1vMbwvNZNnreDn+e4jtvD71PPax1jYP6aiaKoVxRFYQ8/PQFCCAH4I1tvIhoYRdHmHTbxLnDM9oVt46468u8hG6oFX+W8fgnPa930AZAcQjh8h7Zj8LwdTPxu1mExv089r3WMgVm78wfgKODcKIrKd+l7GvhmCGFgCKEh8EtgsTcpHPhCCMnbzlkSkBRCaBhCSN7W7Xmtg7aNe5wG3BhCaBxC6AF8l61Xs1SHxHw//W7WbXv6fep5rWMMzNrJtnkgfwgcC6wKIZRt+7kQIIqi1cBA4Ga23p3fHRhaW/Vqr4wDyoFrgBHbXo8Dz2sd92MgBfgMeBz4URRFXqWqe3b7/fS7WXfF/T71vNY9TisnSZIkxfAKsyRJkhTDwCxJkiTFMDBLkiRJMQzMkiRJUgwDsyRJkhTDwCxJkiTFMDBLkiRJMQzMkiRJUgwDsyRJkhTj/wOfuY8g173t4AAAAABJRU5ErkJggg==\n",
+ "text/plain": [
+ ""
+ ]
+ },
+ "metadata": {
+ "needs_background": "light"
+ },
+ "output_type": "display_data"
+ }
+ ],
+ "source": [
+ "W1 = 1 # Change me\n",
+ "W2 = 10 # Change me\n",
+ "embedding2 = embedding * W1 + pos_encoding[:,:,:] * W2\n",
+ "plot_words(embedding2, sequences, 0)\n",
+ "plot_words(embedding2, sequences, 1)\n",
+ "\n",
+ "# For reference\n",
+ "#['king queen man woman dog wolf football basketball red green yellow',\n",
+ "# 'man queen yellow basketball green dog woman football king red wolf']"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "If you set `W1 = 1` and `W2 = 10`, you can see how arrangement of the words begins to take on a clockwise or anti-clockwise order depending on the position of the words in the sentence. Under these parameters, the positional encoding vectors have dominated the embedding. \n",
+ "\n",
+ "Now try inverting the weights to `W1 = 10` and `W2 = 1`. Observe that under these parameters, the plot resembles the original embedding visualizations and there are only a few changes between the positions of the plotted words.\n",
+ "\n",
+ "In the previous Transformer assignment, the word embedding is multiplied by `sqrt(EMBEDDING_DIM)`. In this case, it will be equivalent using `W1 = sqrt(EMBEDDING_DIM) = 10` and `W2 = 1`. \n",
+ "\n",
+ "### Congratulations!\n",
+ " \n",
+ "You've completed this notebook, and have a better sense of the inputs of the Transformer network! \n",
+ "\n",
+ "By now, you've: \n",
+ "\n",
+ "* Created positional encoding matrices to visualize the relational properties of the vectors\n",
+ "* Plotted embeddings and positional encodings on a Cartesian plane to observe how they affect each other \n",
+ "\n",
+ "\n",
+ "What you should remember:\n",
+ "- Positional encodings can be expressed as linear functions of each other, which allow the model to learn according to the relative positions of words.\n",
+ "- Positional encodings can affect the word embeddings, but if the relative weight of the positional encoding is small, the sum will retain the semantic meaning of the words. \n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": []
+ }
+ ],
+ "metadata": {
+ "accelerator": "GPU",
+ "colab": {
+ "collapsed_sections": [],
+ "name": "Transformer Assignment - Subclass.ipynb",
+ "provenance": []
+ },
+ "kernelspec": {
+ "display_name": "Python 3",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.7.6"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 1
+}
diff --git a/Transformer Mechanism/Transformer Pre-Processing/home/jovyan/work/W4A4_UGL_POS/glove/glove.6B.100d.txt b/Transformer Mechanism/Transformer Pre-Processing/home/jovyan/work/W4A4_UGL_POS/glove/glove.6B.100d.txt
new file mode 100644
index 0000000000000000000000000000000000000000..f7bdbecc60d705c68a7d2204407e2ffe2f6d6ea5
--- /dev/null
+++ b/Transformer Mechanism/Transformer Pre-Processing/home/jovyan/work/W4A4_UGL_POS/glove/glove.6B.100d.txt
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:95dde4dfd627ab26608d33e76d1195ec059734bd29089ea52cadb08d07c64544
+size 347116733
diff --git a/Transformer Mechanism/Transformer Pre-Processing/home/jovyan/work/W4A4_UGL_POS/preprocessing.png b/Transformer Mechanism/Transformer Pre-Processing/home/jovyan/work/W4A4_UGL_POS/preprocessing.png
new file mode 100644
index 0000000000000000000000000000000000000000..c1d6918ecc023e9feee9f51a32fa31283017ad15
--- /dev/null
+++ b/Transformer Mechanism/Transformer Pre-Processing/home/jovyan/work/W4A4_UGL_POS/preprocessing.png
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:81afb9ffa07ebc9b585731c19a3fea007b7adb285101b92119ade4c03a5914cc
+size 196641