Small_Language_Model_Scratch_moviescript.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/pranikz/SLM-Movie-Script/blob/main/Small_Language_Model_Scratch_moviescript.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ijDG3IDqkjKx"
      },
      "source": [
        "# Introduction\n",
        "\n",
        "Let us build a Small Language Model (SLM) from scratch. We will try to keep the parameter size to 50-60 million.\n",
        "\n",
        "Our goal is to generate creative and coherent text based on the input data."
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Step 1: Import the Dataset\n",
        "\n",
        "Movie Scripts Dataset\n",
        "The Movie Scripts Dataset consists of scripts from 1,172 movies, providing a comprehensive collection of movie dialogues and narratives. This dataset is designed to support various natural language processing (NLP) tasks, including dialogue generation, script summarization, and text analysis.\n",
        "\n",
        "Details\n",
        "The dataset contains 2 columns:\n",
        "\n",
        "Name: The title of the movie.\n",
        "Script: The full script of the movie in English.\n",
        "Usage\n",
        "The Movie Scripts Dataset is ideal for training and evaluating models for dialogue generation, script summarization, and other text analysis tasks. It offers a diverse set of scripts that can be used to explore various aspects of movie dialogues and narratives."
      ],
      "metadata": {
        "id": "tPl23BNsRqfm"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!pip install datasets"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "BSeXbOztRtKN",
        "outputId": "a7329063-0407-4bd7-ee51-00cddc39bdd5"
      },
      "execution_count": 46,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Requirement already satisfied: datasets in /usr/local/lib/python3.12/dist-packages (4.1.1)\n",
            "Requirement already satisfied: filelock in /usr/local/lib/python3.12/dist-packages (from datasets) (3.19.1)\n",
            "Requirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.12/dist-packages (from datasets) (2.0.2)\n",
            "Requirement already satisfied: pyarrow>=21.0.0 in /usr/local/lib/python3.12/dist-packages (from datasets) (21.0.0)\n",
            "Requirement already satisfied: dill<0.4.1,>=0.3.0 in /usr/local/lib/python3.12/dist-packages (from datasets) (0.3.8)\n",
            "Requirement already satisfied: pandas in /usr/local/lib/python3.12/dist-packages (from datasets) (2.2.2)\n",
            "Requirement already satisfied: requests>=2.32.2 in /usr/local/lib/python3.12/dist-packages (from datasets) (2.32.4)\n",
            "Requirement already satisfied: tqdm>=4.66.3 in /usr/local/lib/python3.12/dist-packages (from datasets) (4.67.1)\n",
            "Requirement already satisfied: xxhash in /usr/local/lib/python3.12/dist-packages (from datasets) (3.5.0)\n",
            "Requirement already satisfied: multiprocess<0.70.17 in /usr/local/lib/python3.12/dist-packages (from datasets) (0.70.16)\n",
            "Requirement already satisfied: fsspec<=2025.9.0,>=2023.1.0 in /usr/local/lib/python3.12/dist-packages (from fsspec[http]<=2025.9.0,>=2023.1.0->datasets) (2025.3.0)\n",
            "Requirement already satisfied: huggingface-hub>=0.24.0 in /usr/local/lib/python3.12/dist-packages (from datasets) (0.34.4)\n",
            "Requirement already satisfied: packaging in /usr/local/lib/python3.12/dist-packages (from datasets) (25.0)\n",
            "Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.12/dist-packages (from datasets) (6.0.2)\n",
            "Requirement already satisfied: aiohttp!=4.0.0a0,!=4.0.0a1 in /usr/local/lib/python3.12/dist-packages (from fsspec[http]<=2025.9.0,>=2023.1.0->datasets) (3.12.15)\n",
            "Requirement already satisfied: typing-extensions>=3.7.4.3 in /usr/local/lib/python3.12/dist-packages (from huggingface-hub>=0.24.0->datasets) (4.15.0)\n",
            "Requirement already satisfied: hf-xet<2.0.0,>=1.1.3 in /usr/local/lib/python3.12/dist-packages (from huggingface-hub>=0.24.0->datasets) (1.1.9)\n",
            "Requirement already satisfied: charset_normalizer<4,>=2 in /usr/local/lib/python3.12/dist-packages (from requests>=2.32.2->datasets) (3.4.3)\n",
            "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.12/dist-packages (from requests>=2.32.2->datasets) (3.10)\n",
            "Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.12/dist-packages (from requests>=2.32.2->datasets) (2.5.0)\n",
            "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.12/dist-packages (from requests>=2.32.2->datasets) (2025.8.3)\n",
            "Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.12/dist-packages (from pandas->datasets) (2.9.0.post0)\n",
            "Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.12/dist-packages (from pandas->datasets) (2025.2)\n",
            "Requirement already satisfied: tzdata>=2022.7 in /usr/local/lib/python3.12/dist-packages (from pandas->datasets) (2025.2)\n",
            "Requirement already satisfied: aiohappyeyeballs>=2.5.0 in /usr/local/lib/python3.12/dist-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]<=2025.9.0,>=2023.1.0->datasets) (2.6.1)\n",
            "Requirement already satisfied: aiosignal>=1.4.0 in /usr/local/lib/python3.12/dist-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]<=2025.9.0,>=2023.1.0->datasets) (1.4.0)\n",
            "Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.12/dist-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]<=2025.9.0,>=2023.1.0->datasets) (25.3.0)\n",
            "Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.12/dist-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]<=2025.9.0,>=2023.1.0->datasets) (1.7.0)\n",
            "Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.12/dist-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]<=2025.9.0,>=2023.1.0->datasets) (6.6.4)\n",
            "Requirement already satisfied: propcache>=0.2.0 in /usr/local/lib/python3.12/dist-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]<=2025.9.0,>=2023.1.0->datasets) (0.3.2)\n",
            "Requirement already satisfied: yarl<2.0,>=1.17.0 in /usr/local/lib/python3.12/dist-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]<=2025.9.0,>=2023.1.0->datasets) (1.20.1)\n",
            "Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.12/dist-packages (from python-dateutil>=2.8.2->pandas->datasets) (1.17.0)\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "from datasets import load_dataset\n",
        "\n",
        "# Load the dataset\n",
        "ds = load_dataset(\"IsmaelMousa/movies\")\n",
        "\n",
        "# Split the dataset into train and validation sets\n",
        "# Assuming the loaded dataset is named 'train', we split it\n",
        "if 'train' in ds:\n",
        "    ds = ds['train'].train_test_split(test_size=0.1, seed=42) # Using 10% for validation\n",
        "    # Rename the test split to validation to match the rest of the code\n",
        "    ds['validation'] = ds.pop('test')\n",
        "else:\n",
        "    # If the dataset has other splits, you might need to adjust this logic\n",
        "    print(\"Dataset does not contain a 'train' split. Please check the dataset splits.\")\n",
        "\n",
        "# Print the dataset splits to confirm\n",
        "print(\"Dataset splits after processing:\", ds.keys())"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "3r34TTsnR3GI",
        "outputId": "e898e802-1bae-46f7-87ec-93746082d566"
      },
      "execution_count": 47,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Dataset splits after processing: dict_keys(['train', 'validation'])\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Step 2: Tokenize the Dataset\n",
        "\n",
        "In this step, we will do the following:\n",
        "\n",
        "(1) Tokenize the dataset into tokenIDs.\n",
        "\n",
        "(2) Create a file called \"train.bin\" and \"validtion.bin\" where we will store the tokenIDs from the entire dataset.\n",
        "\n",
        "(3) We make sure the tokenIDs are stored on a disk, rather than on the RAM for efficient computations."
      ],
      "metadata": {
        "id": "vccyr4qKR6OH"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!pip install tiktoken\n",
        "import tiktoken\n",
        "import os\n",
        "import numpy as np\n",
        "from tqdm.auto import tqdm\n",
        "\n",
        "enc = tiktoken.get_encoding(\"gpt2\")\n",
        "\n",
        "# Some functions from https://github.com/karpathy/nanoGPT/blob/master/data/openwebtext/prepare.py\n",
        "\n",
        "def process(example):\n",
        "    # Use the 'Script' column instead of 'text'\n",
        "    ids = enc.encode_ordinary(example['Script']) # encode_ordinary ignores any special tokens\n",
        "    out = {'ids': ids, 'len': len(ids)}\n",
        "    return out\n",
        "\n",
        "if not os.path.exists(\"train.bin\"):\n",
        "    print(\"Tokenizing dataset splits...\")\n",
        "    tokenized = ds.map(\n",
        "        process,\n",
        "        # Remove the 'Name' and 'Script' columns\n",
        "        remove_columns=['Name', 'Script'],\n",
        "        desc=\"tokenizing the splits\",\n",
        "        num_proc=8,\n",
        "        )\n",
        "    # concatenate all the ids in each dataset into one large file we can use for training\n",
        "    print(\"Writing tokenized data to .bin files...\")\n",
        "    for split, dset in tokenized.items():\n",
        "        print(f\"Processing split: {split}\")\n",
        "        arr_len = np.sum(dset['len'], dtype=np.uint64)\n",
        "        filename = f'{split}.bin'\n",
        "        print(f\"Attempting to create file: {filename} with length {arr_len}\")\n",
        "        dtype = np.uint16 # (can do since enc.max_token_value == 50256 is < 2**16)\n",
        "        arr = np.memmap(filename, dtype=dtype, mode='w+', shape=(arr_len,))\n",
        "        # Use a number of batches that is at most the dataset size\n",
        "        num_batches_for_split = min(len(dset), 1024) # Use 1024 or the number of examples if smaller\n",
        "\n",
        "        idx = 0\n",
        "        for batch_idx in tqdm(range(num_batches_for_split), desc=f'writing {filename}'):\n",
        "            # Batch together samples for faster write\n",
        "            batch = dset.shard(num_shards=num_batches_for_split, index=batch_idx, contiguous=True).with_format('numpy')\n",
        "            arr_batch = np.concatenate(batch['ids'])\n",
        "            # Write into mmap\n",
        "            arr[idx : idx + len(arr_batch)] = arr_batch\n",
        "            idx += len(arr_batch)\n",
        "        arr.flush()\n",
        "        print(f\"Finished writing {filename}\")"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 433,
          "referenced_widgets": [
            "df380f8545ea45019e3f682131871023",
            "51f762c34cb94a3db8b7cccb564e42e4",
            "2862c5ef39be4c5d9c9e8a0e067b6a2a",
            "b9437f36fa9c492f91c9bd60e7c02a32",
            "14d085e122ca4443bf510e1f5dbb09ea",
            "946b71da99ea4c9299002ce3e33c85f8",
            "c595061cb534452b85b6b06a273034c0",
            "1afd256c93af4b3c9f2fe2abe6cfc03f",
            "a055e9383a8443f9a5be1b04deb5fe3f",
            "902c50861737414a97ab4597b24ccf2b",
            "cfb3691906e64b139671b5f4266d9183",
            "dcaa3317d5b2458b8c587e37416f4556",
            "b4a40fd07c6b4bd9a30c093533cf442a",
            "705b30b3d60e40029538205adef72bc6",
            "e0949efeccdb42d183190f066fd3eae9",
            "6206c991506141b0886a8a68378e1570",
            "170a94f8b45c42eabeb8fa9742e9e6ad",
            "19234232222247d495155341cb88dbcd",
            "79765c0ad7fa4e2695221e75ffdffc97",
            "2d033811ed5c47b7955b7a10f10babd6",
            "c162c0d72559484fb9d6045b8d49b78e",
            "ce03a6bbf7d840cca3f883877e5eced3",
            "108bcaa337b3469ead4edd3460a1ff69",
            "0d88c6cf77bb4b49b81b76ca96487806",
            "536c18036bd04b9ea49f933348742d1f",
            "3ddf0a19bef643d0bb407a3c0c95c891",
            "391f9e3fc7fc4ee293999823e49966af",
            "a12d4123f63e44ca8f32ce6f329c9656",
            "0f1e25c41b13445bbf37c02fc72eb9da",
            "6d8f95cd8a1f48be9c739e0f2421b97c",
            "6b353f12fa8c4deb8da3309ada39bf5e",
            "6d2127ed2eca435782ae796f5f782dcc",
            "76103cd6b60847eb8ad31a0d8c58bb65",
            "e85791c0c84746e3a249ebd9def7e819",
            "d00860d39d1d4a32ac9774a4bb4288a1",
            "1a0dbd4a6c234f6ab926280aad42e9cb",
            "a4f375caf2e44daca655b3314e23f3c1",
            "b5c6fb4f54b04086b0eaeb2442d33101",
            "e8809ebfc7fd49dfaaea4291ef97a151",
            "b2b85a7f2c2f4cfca44000f07a2f3d6c",
            "c13ca3cdae2448e98513f43cc5192862",
            "09eb329d9c9945a1b985e671e02c4eda",
            "72cb236d4a0c42ee858cd7f5db0130ab",
            "db2c4e93f88c4fcc955d2a7524d2e3e9"
          ]
        },
        "id": "vFkgAjyMR8fa",
        "outputId": "302f43d9-4a8f-4b76-9b37-a437878dde73"
      },
      "execution_count": 48,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Requirement already satisfied: tiktoken in /usr/local/lib/python3.12/dist-packages (0.11.0)\n",
            "Requirement already satisfied: regex>=2022.1.18 in /usr/local/lib/python3.12/dist-packages (from tiktoken) (2024.11.6)\n",
            "Requirement already satisfied: requests>=2.26.0 in /usr/local/lib/python3.12/dist-packages (from tiktoken) (2.32.4)\n",
            "Requirement already satisfied: charset_normalizer<4,>=2 in /usr/local/lib/python3.12/dist-packages (from requests>=2.26.0->tiktoken) (3.4.3)\n",
            "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.12/dist-packages (from requests>=2.26.0->tiktoken) (3.10)\n",
            "Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.12/dist-packages (from requests>=2.26.0->tiktoken) (2.5.0)\n",
            "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.12/dist-packages (from requests>=2.26.0->tiktoken) (2025.8.3)\n",
            "Tokenizing dataset splits...\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "tokenizing the splits (num_proc=8):   0%|          | 0/1054 [00:00<?, ? examples/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "df380f8545ea45019e3f682131871023"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "tokenizing the splits (num_proc=8):   0%|          | 0/118 [00:00<?, ? examples/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "dcaa3317d5b2458b8c587e37416f4556"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Writing tokenized data to .bin files...\n",
            "Processing split: train\n",
            "Attempting to create file: train.bin with length 114600149\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "writing train.bin:   0%|          | 0/1024 [00:00<?, ?it/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "108bcaa337b3469ead4edd3460a1ff69"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Finished writing train.bin\n",
            "Processing split: validation\n",
            "Attempting to create file: validation.bin with length 12409610\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "writing validation.bin:   0%|          | 0/118 [00:00<?, ?it/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "e85791c0c84746e3a249ebd9def7e819"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Finished writing validation.bin\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Step 3: Create Input-Output batches for the dataset"
      ],
      "metadata": {
        "id": "X_qRtn_WSbV4"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Some functions from https://github.com/karpathy/nanoGPT/blob/master/train.py with slight modifications\n",
        "#block size = context window\n",
        "def get_batch(split):\n",
        "    # We recreate np.memmap every batch to avoid a memory leak, as per\n",
        "    # https://stackoverflow.com/questions/45132940/numpy-memmap-memory-usage-want-to-iterate-once/61472122#61472122\n",
        "    if split == 'train':\n",
        "        data = np.memmap('train.bin', dtype=np.uint16, mode='r')\n",
        "    else:\n",
        "        data = np.memmap('validation.bin', dtype=np.uint16, mode='r')\n",
        "    ix = torch.randint(len(data) - block_size, (batch_size,))\n",
        "    x = torch.stack([torch.from_numpy((data[i:i+block_size]).astype(np.int64)) for i in ix])\n",
        "    y = torch.stack([torch.from_numpy((data[i+1:i+1+block_size]).astype(np.int64)) for i in ix])\n",
        "    if device_type == 'cuda':\n",
        "        # pin arrays x,y, which allows us to move them to GPU asynchronously (non_blocking=True)\n",
        "        x, y = x.pin_memory().to(device, non_blocking=True), y.pin_memory().to(device, non_blocking=True)\n",
        "    else:\n",
        "        x, y = x.to(device), y.to(device)\n",
        "    return x, y\n"
      ],
      "metadata": {
        "id": "gak79CZESkjN"
      },
      "execution_count": 30,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kA1SVp1hkjKy"
      },
      "source": [
        "## Step 4: Define the SLM Model Architecture"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "import torch\n",
        "import torch.nn as nn\n",
        "import torch.nn.functional as F\n",
        "import math\n",
        "from dataclasses import dataclass\n",
        "import numpy as np\n",
        "from tqdm.auto import tqdm\n",
        "from contextlib import nullcontext\n",
        "import os\n",
        "\n",
        "class LayerNorm(nn.Module):\n",
        "    def __init__(self, ndim, bias):\n",
        "        super().__init__()\n",
        "        self.weight = nn.Parameter(torch.ones(ndim))\n",
        "        self.bias = nn.Parameter(torch.zeros(ndim)) if bias else None\n",
        "    def forward(self, x):\n",
        "        return F.layer_norm(x, self.weight.shape, self.weight, self.bias, 1e-5)\n",
        "\n",
        "class CausalSelfAttention(nn.Module):\n",
        "    def __init__(self, config):\n",
        "        super().__init__()\n",
        "        assert config.n_embd % config.n_head == 0\n",
        "        self.c_attn = nn.Linear(config.n_embd, 3 * config.n_embd, bias=config.bias)\n",
        "        self.c_proj = nn.Linear(config.n_embd, config.n_embd, bias=config.bias)\n",
        "        self.attn_dropout = nn.Dropout(config.dropout)\n",
        "        self.resid_dropout = nn.Dropout(config.dropout)\n",
        "        self.n_head = config.n_head\n",
        "        self.n_embd = config.n_embd\n",
        "        self.flash = hasattr(F, 'scaled_dot_product_attention')\n",
        "        if not self.flash:\n",
        "            self.register_buffer(\"bias\", torch.tril(torch.ones(config.block_size, config.block_size))\n",
        "                                       .view(1, 1, config.block_size, config.block_size))\n",
        "\n",
        "    def forward(self, x):\n",
        "        B, T, C = x.size()\n",
        "        q, k, v = self.c_attn(x).split(self.n_embd, dim=2)\n",
        "        k = k.view(B, T, self.n_head, C // self.n_head).transpose(1, 2)\n",
        "        q = q.view(B, T, self.n_head, C // self.n_head).transpose(1, 2)\n",
        "        v = v.view(B, T, self.n_head, C // self.n_head).transpose(1, 2)\n",
        "\n",
        "        if self.flash:\n",
        "            y = F.scaled_dot_product_attention(q, k, v, attn_mask=None, dropout_p=self.attn_dropout.p if self.training else 0.0, is_causal=True)\n",
        "        else:\n",
        "            att = (q @ k.transpose(-2, -1)) * (1.0 / math.sqrt(k.size(-1)))\n",
        "            att = att.masked_fill(self.bias[:, :, :T, :T] == 0, float('-inf'))\n",
        "            att = F.softmax(att, dim=-1)\n",
        "            att = self.attn_dropout(att)\n",
        "            y = att @ v\n",
        "\n",
        "        y = y.transpose(1, 2).contiguous().view(B, T, C)\n",
        "        y = self.resid_dropout(self.c_proj(y))\n",
        "        return y\n",
        "\n",
        "class MLP(nn.Module):\n",
        "    def __init__(self, config):\n",
        "        super().__init__()\n",
        "        self.c_fc = nn.Linear(config.n_embd, 4 * config.n_embd, bias=config.bias)\n",
        "        self.gelu = nn.GELU()\n",
        "        self.c_proj = nn.Linear(4 * config.n_embd, config.n_embd, bias=config.bias)\n",
        "        self.dropout = nn.Dropout(config.dropout)\n",
        "    def forward(self, x):\n",
        "        return self.dropout(self.c_proj(self.gelu(self.c_fc(x))))\n",
        "\n",
        "class Block(nn.Module):\n",
        "    def __init__(self, config):\n",
        "        super().__init__()\n",
        "        self.ln1 = LayerNorm(config.n_embd, config.bias)\n",
        "        self.attn = CausalSelfAttention(config)\n",
        "        self.ln2 = LayerNorm(config.n_embd, config.bias)\n",
        "        self.mlp = MLP(config)\n",
        "    def forward(self, x):\n",
        "        x = x + self.attn(self.ln1(x))\n",
        "        x = x + self.mlp(self.ln2(x))\n",
        "        return x\n",
        "\n",
        "@dataclass\n",
        "class GPTConfig:\n",
        "    block_size: int\n",
        "    vocab_size: int\n",
        "    n_layer: int\n",
        "    n_head: int\n",
        "    n_embd: int\n",
        "    dropout: float = 0.0\n",
        "    bias: bool = True\n",
        "\n",
        "class GPT(nn.Module):\n",
        "    def __init__(self, config):\n",
        "        super().__init__()\n",
        "        self.config = config\n",
        "        self.transformer = nn.ModuleDict(dict(\n",
        "            wte=nn.Embedding(config.vocab_size, config.n_embd),\n",
        "            wpe=nn.Embedding(config.block_size, config.n_embd),\n",
        "            drop=nn.Dropout(config.dropout),\n",
        "            h=nn.ModuleList([Block(config) for _ in range(config.n_layer)]),\n",
        "            ln_f=LayerNorm(config.n_embd, config.bias),\n",
        "        ))\n",
        "        self.lm_head = nn.Linear(config.n_embd, config.vocab_size, bias=False)\n",
        "        self.transformer.wte.weight = self.lm_head.weight  # weight tying\n",
        "\n",
        "        self.apply(self._init_weights)\n",
        "        for pn, p in self.named_parameters():\n",
        "            if pn.endswith('c_proj.weight'):\n",
        "                nn.init.normal_(p, mean=0.0, std=0.02 / math.sqrt(2 * config.n_layer))\n",
        "\n",
        "    def _init_weights(self, module):\n",
        "        if isinstance(module, nn.Linear):\n",
        "            nn.init.normal_(module.weight, mean=0.0, std=0.02)\n",
        "            if module.bias is not None:\n",
        "                nn.init.zeros_(module.bias)\n",
        "        elif isinstance(module, nn.Embedding):\n",
        "            nn.init.normal_(module.weight, mean=0.0, std=0.02)\n",
        "\n",
        "    def forward(self, idx, targets=None):\n",
        "        device = idx.device\n",
        "        b, t = idx.size()\n",
        "        assert t <= self.config.block_size\n",
        "        pos = torch.arange(0, t, dtype=torch.long, device=device)\n",
        "\n",
        "        tok_emb = self.transformer.wte(idx)\n",
        "        pos_emb = self.transformer.wpe(pos)\n",
        "        x = self.transformer.drop(tok_emb + pos_emb)\n",
        "        for block in self.transformer.h:\n",
        "            x = block(x)\n",
        "        x = self.transformer.ln_f(x)\n",
        "\n",
        "        if targets is not None:\n",
        "            logits = self.lm_head(x)\n",
        "            loss = F.cross_entropy(logits.view(-1, logits.size(-1)), targets.view(-1), ignore_index=-1)\n",
        "            return logits, loss\n",
        "        else:\n",
        "            logits = self.lm_head(x[:, [-1], :])\n",
        "            return logits, None\n",
        "\n",
        "    @torch.no_grad()\n",
        "    def generate(self, idx, max_new_tokens, temperature=1.0, top_k=None):\n",
        "        \"\"\"\n",
        "        Generate tokens given a conditioning sequence.\n",
        "        idx: Tensor of shape (B, T)\n",
        "        \"\"\"\n",
        "        for _ in range(max_new_tokens):\n",
        "            idx_cond = idx if idx.size(1) <= self.config.block_size else idx[:, -self.config.block_size:]\n",
        "            logits, _ = self(idx_cond)\n",
        "            logits = logits[:, -1, :] / temperature\n",
        "            if top_k is not None:\n",
        "                v, _ = torch.topk(logits, min(top_k, logits.size(-1)))\n",
        "                logits[logits < v[:, [-1]]] = -float('Inf')\n",
        "            probs = F.softmax(logits, dim=-1)\n",
        "            idx_next = torch.multinomial(probs, num_samples=1)\n",
        "            idx = torch.cat((idx, idx_next), dim=1)\n",
        "        return idx\n",
        "\n"
      ],
      "metadata": {
        "id": "9friaxWABOA-"
      },
      "execution_count": 49,
      "outputs": []
    },
    {
      "cell_type": "code",
      "execution_count": 50,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2024-06-20T15:36:54.88378Z",
          "iopub.status.busy": "2024-06-20T15:36:54.883342Z",
          "iopub.status.idle": "2024-06-20T15:37:07.278493Z",
          "shell.execute_reply": "2024-06-20T15:37:07.277342Z",
          "shell.execute_reply.started": "2024-06-20T15:36:54.883753Z"
        },
        "id": "uRm6WlvfkjKz",
        "trusted": true
      },
      "outputs": [],
      "source": [
        "config = GPTConfig(\n",
        "    vocab_size=50257,     # use the tokenizer's vocab size\n",
        "    block_size=512,       # or whatever context size you're training with\n",
        "    n_layer=6,\n",
        "    n_head=6,\n",
        "    n_embd=384,\n",
        "    dropout=0.1,\n",
        "    bias=True\n",
        ")\n",
        "\n",
        "model = GPT(config)"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Step 5: Define the loss function"
      ],
      "metadata": {
        "id": "C_a8Rd-0S_WC"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "def estimate_loss(model):\n",
        "    out = {}\n",
        "    model.eval()\n",
        "    with torch.inference_mode():\n",
        "        for split in ['train', 'val']:\n",
        "            losses = torch.zeros(eval_iters)\n",
        "            for k in range(eval_iters):\n",
        "                X, Y = get_batch(split)\n",
        "                with ctx:\n",
        "                    logits, loss = model(X, Y)\n",
        "                losses[k] = loss.item()\n",
        "            out[split] = losses.mean()\n",
        "    model.train()\n",
        "    return out"
      ],
      "metadata": {
        "id": "La2Aun_nTBzk"
      },
      "execution_count": 51,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Step 6: Define SLM Training Configuration Part 1"
      ],
      "metadata": {
        "id": "UvqWPUstTRXO"
      }
    },
    {
      "cell_type": "code",
      "execution_count": 52,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "8QQyayhnkjK5",
        "outputId": "5b62b0c2-8cb0-4b76-c3e6-553fd9e979ad",
        "trusted": true
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<torch._C.Generator at 0x7a62898b60b0>"
            ]
          },
          "metadata": {},
          "execution_count": 52
        }
      ],
      "source": [
        "# Training Config\n",
        "import torch\n",
        "from contextlib import nullcontext\n",
        "\n",
        "learning_rate = 1e-4 #more stable training, earlier 1e-4\n",
        "max_iters = 20000 #increase from 25000\n",
        "warmup_steps = 1000 #smoother initial train, earlier 100\n",
        "min_lr = 5e-4 #lower rate, earlier 5e-4\n",
        "eval_iters = 500 # increased from 100\n",
        "batch_size = 32 # changed from 16, better gradient estimate\n",
        "block_size = 128 #changed from 64, capture longer range dependencies\n",
        "\n",
        "gradient_accumulation_steps = 32 # reduced from 50\n",
        "\n",
        "device =  \"cuda\" if torch.cuda.is_available() else \"cpu\"\n",
        "device_type = 'cuda' if 'cuda' in device else 'cpu' # for later use in torch.autocast\n",
        "# note: float16 data type will automatically use a GradScaler\n",
        "\n",
        "# How to use autocast https://wandb.ai/wandb_fc/tips/reports/How-To-Use-Autocast-in-PyTorch--VmlldzoyMTk4NTky\n",
        "#dtype = 'bfloat16' if torch.cuda.is_available() and torch.cuda.is_bf16_supported() else 'float16' # 'float32', 'bfloat16', or 'float16', the latter will auto implement a GradScaler\n",
        "dtype = 'bfloat16' if torch.cuda.is_available() and torch.cuda.is_bf16_supported() else 'float16' # 'float32', 'bfloat16', or 'float16', the latter will auto implement a GradScaler\n",
        "ptdtype = {'float32': torch.float32, 'bfloat16': torch.bfloat16, 'float16': torch.float16}[dtype]\n",
        "\n",
        "ctx = nullcontext() if device_type == 'cpu' else torch.amp.autocast(device_type=device_type, dtype=ptdtype)\n",
        "\n",
        "torch.set_default_device(device)\n",
        "torch.manual_seed(42)"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Step 7: Define SLM Training Configuration Part 2"
      ],
      "metadata": {
        "id": "cmWj6YcKTW_z"
      }
    },
    {
      "cell_type": "code",
      "execution_count": 53,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "JMkO3FlNkjK6",
        "outputId": "e38f644a-d941-4529-f8b5-3c62a8915d14",
        "trusted": true
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "/tmp/ipython-input-2132813893.py:11: FutureWarning: `torch.cuda.amp.GradScaler(args...)` is deprecated. Please use `torch.amp.GradScaler('cuda', args...)` instead.\n",
            "  scaler = torch.cuda.amp.GradScaler(enabled=(dtype == 'float16'))\n"
          ]
        }
      ],
      "source": [
        "from torch.optim.lr_scheduler import LinearLR,SequentialLR, CosineAnnealingLR\n",
        "\n",
        "##PUT IN WEIGHT DECAY, CHANGED BETA2 to 0.95\n",
        "optimizer =  torch.optim.AdamW(model.parameters(), lr=learning_rate, betas=(0.9, 0.95), weight_decay=0.1, eps=1e-9) #weight decay for regularization\n",
        "\n",
        "scheduler_warmup = LinearLR(optimizer, total_iters = warmup_steps) #Implement linear warmup\n",
        "scheduler_decay = CosineAnnealingLR(optimizer,T_max = max_iters - warmup_steps, eta_min = min_lr) #Implement lr decay\n",
        "scheduler = SequentialLR(optimizer, schedulers=[scheduler_warmup, scheduler_decay], milestones=[warmup_steps]) #Switching from warmup to decay\n",
        "\n",
        "# https://stackoverflow.com/questions/72534859/is-gradscaler-necessary-with-mixed-precision-training-with-pytorch\n",
        "scaler = torch.cuda.amp.GradScaler(enabled=(dtype == 'float16'))"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Step 8: Pre-train the SLM"
      ],
      "metadata": {
        "id": "Nz8fPSKNTY3W"
      }
    },
    {
      "cell_type": "code",
      "execution_count": 54,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000,
          "referenced_widgets": [
            "1b7ef9eeaf524b0a83cc3a7202f439b0",
            "f25a6677e26b49659d4a44754ac54cc9",
            "31ac912fc6ee40ce85a8c765d2f1ecc2",
            "a8d6c756e7e042eea821c4719dda02e5",
            "fec71188445848c39c68714d05826f87",
            "ad6d7f7729104fd9a3963527bfa80edc",
            "3e3cf11e619f40ce97a87a0129b89254",
            "b8c47443b6ca43deadb57efbb8100f45",
            "c94b8b9fd94f4543aca2a0fdc283f67d",
            "25e135fe47b94e75931eae4f5ae42f94",
            "8db5ee2efbdc4e7ea72d9d1f4bd1928a"
          ]
        },
        "id": "t0l-YhockjK6",
        "outputId": "ec50b31d-18da-4dfa-bdc1-eb091b3a45bc",
        "trusted": true
      },
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "  0%|          | 0/20000 [00:00<?, ?it/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "1b7ef9eeaf524b0a83cc3a7202f439b0"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "/usr/local/lib/python3.12/dist-packages/torch/optim/lr_scheduler.py:192: UserWarning: Detected call of `lr_scheduler.step()` before `optimizer.step()`. In PyTorch 1.1.0 and later, you should call them in the opposite order: `optimizer.step()` before `lr_scheduler.step()`.  Failure to do this will result in PyTorch skipping the first value of the learning rate schedule. See more details at https://pytorch.org/docs/stable/optim.html#how-to-adjust-learning-rate\n",
            "  warnings.warn(\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 500: train loss 6.0358, val loss 6.0601\n",
            "The current learning rate: 0.00007\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "/usr/local/lib/python3.12/dist-packages/torch/optim/lr_scheduler.py:209: UserWarning: The epoch parameter in `scheduler.step()` was not necessary and is being deprecated where possible. Please use `scheduler.step()` to step the scheduler. During the deprecation, if epoch is different from None, the closed form is used instead of the new chainable form, where available. Please open an issue if you are unable to replicate your use case: https://github.com/pytorch/pytorch/issues/new/choose.\n",
            "  warnings.warn(EPOCH_DEPRECATION_WARNING, UserWarning)\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1000: train loss 5.0690, val loss 5.1143\n",
            "The current learning rate: 0.00010\n",
            "Epoch 1500: train loss 4.3162, val loss 4.3407\n",
            "The current learning rate: 0.00010\n",
            "Epoch 2000: train loss 3.5948, val loss 3.6099\n",
            "The current learning rate: 0.00010\n",
            "Epoch 2500: train loss 3.0460, val loss 3.0569\n",
            "The current learning rate: 0.00011\n",
            "Epoch 3000: train loss 2.7518, val loss 2.7398\n",
            "The current learning rate: 0.00011\n",
            "Epoch 3500: train loss 2.5606, val loss 2.5574\n",
            "The current learning rate: 0.00012\n",
            "Epoch 4000: train loss 2.4583, val loss 2.4691\n",
            "The current learning rate: 0.00012\n",
            "Epoch 4500: train loss 2.3943, val loss 2.3969\n",
            "The current learning rate: 0.00013\n",
            "Epoch 5000: train loss 2.3428, val loss 2.3513\n",
            "The current learning rate: 0.00014\n",
            "Epoch 5500: train loss 2.2667, val loss 2.2981\n",
            "The current learning rate: 0.00015\n",
            "Epoch 6000: train loss 2.2141, val loss 2.2155\n",
            "The current learning rate: 0.00016\n",
            "Epoch 6500: train loss 2.1810, val loss 2.1883\n",
            "The current learning rate: 0.00018\n",
            "Epoch 7000: train loss 2.1389, val loss 2.1577\n",
            "The current learning rate: 0.00019\n",
            "Epoch 7500: train loss 2.1077, val loss 2.0999\n",
            "The current learning rate: 0.00020\n",
            "Epoch 8000: train loss 2.0570, val loss 2.0703\n",
            "The current learning rate: 0.00022\n",
            "Epoch 8500: train loss 2.0355, val loss 2.0365\n",
            "The current learning rate: 0.00024\n",
            "Epoch 9000: train loss 2.0062, val loss 2.0210\n",
            "The current learning rate: 0.00025\n",
            "Epoch 9500: train loss 1.9713, val loss 1.9841\n",
            "The current learning rate: 0.00027\n",
            "Epoch 10000: train loss 1.9604, val loss 1.9715\n",
            "The current learning rate: 0.00028\n",
            "Epoch 10500: train loss 1.9310, val loss 1.9621\n",
            "The current learning rate: 0.00030\n",
            "Epoch 11000: train loss 1.9001, val loss 1.9256\n",
            "The current learning rate: 0.00032\n",
            "Epoch 11500: train loss 1.8820, val loss 1.9022\n",
            "The current learning rate: 0.00033\n",
            "Epoch 12000: train loss 1.8580, val loss 1.8924\n",
            "The current learning rate: 0.00035\n",
            "Epoch 12500: train loss 1.8437, val loss 1.8755\n",
            "The current learning rate: 0.00036\n",
            "Epoch 13000: train loss 1.8197, val loss 1.8531\n",
            "The current learning rate: 0.00038\n",
            "Epoch 13500: train loss 1.8071, val loss 1.8378\n",
            "The current learning rate: 0.00040\n",
            "Epoch 14000: train loss 1.7954, val loss 1.8284\n",
            "The current learning rate: 0.00041\n",
            "Epoch 14500: train loss 1.7799, val loss 1.8035\n",
            "The current learning rate: 0.00042\n",
            "Epoch 15000: train loss 1.7781, val loss 1.8102\n",
            "The current learning rate: 0.00044\n",
            "Epoch 15500: train loss 1.7555, val loss 1.8028\n",
            "The current learning rate: 0.00045\n",
            "Epoch 16000: train loss 1.7369, val loss 1.7937\n",
            "The current learning rate: 0.00046\n",
            "Epoch 16500: train loss 1.7308, val loss 1.7586\n",
            "The current learning rate: 0.00047\n",
            "Epoch 17000: train loss 1.7163, val loss 1.7610\n",
            "The current learning rate: 0.00048\n",
            "Epoch 17500: train loss 1.6914, val loss 1.7499\n",
            "The current learning rate: 0.00048\n",
            "Epoch 18000: train loss 1.6901, val loss 1.7314\n",
            "The current learning rate: 0.00049\n",
            "Epoch 18500: train loss 1.6936, val loss 1.7349\n",
            "The current learning rate: 0.00049\n",
            "Epoch 19000: train loss 1.6864, val loss 1.7399\n",
            "The current learning rate: 0.00050\n",
            "Epoch 19500: train loss 1.6594, val loss 1.7216\n",
            "The current learning rate: 0.00050\n"
          ]
        }
      ],
      "source": [
        "best_val_loss = float('inf')\n",
        "best_model_params_path = \"best_model_params.pt\"\n",
        "train_loss_list, validation_loss_list = [], []\n",
        "\n",
        "# Ensure model is on the correct device\n",
        "model = model.to(device)\n",
        "\n",
        "# In your training loop\n",
        "for epoch in tqdm(range(max_iters)):\n",
        "    if epoch % eval_iters == 0 and epoch != 0:\n",
        "        # Ensure estimate_loss uses the correct device\n",
        "        losses = estimate_loss(model)\n",
        "        print(f\"Epoch {epoch}: train loss {losses['train']:.4f}, val loss {losses['val']:.4f}\")\n",
        "        print(f\"The current learning rate: {optimizer.param_groups[0]['lr']:.5f}\")\n",
        "        train_loss_list += [losses['train']]\n",
        "        validation_loss_list += [losses['val']]\n",
        "\n",
        "        if losses['val'] < best_val_loss:\n",
        "            best_val_loss = losses['val']\n",
        "            torch.save(model.state_dict(), best_model_params_path)\n",
        "\n",
        "    # Ensure X and y are on the correct device\n",
        "    X, y = get_batch(\"train\")\n",
        "    X, y = X.to(device), y.to(device)\n",
        "\n",
        "    with ctx:\n",
        "        logits, loss = model(X, y)\n",
        "        loss = loss / gradient_accumulation_steps\n",
        "        scaler.scale(loss).backward()\n",
        "\n",
        "    if ((epoch + 1) % gradient_accumulation_steps == 0) or (epoch + 1 == max_iters):\n",
        "        torch.nn.utils.clip_grad_norm_(model.parameters(), max_norm=0.5)\n",
        "        scaler.step(optimizer)\n",
        "        scaler.update()\n",
        "        optimizer.zero_grad(set_to_none=True)\n",
        "    scheduler.step()"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Step 9: Plot the SLM Loss Function"
      ],
      "metadata": {
        "id": "pdzhSo_7TcgI"
      }
    },
    {
      "cell_type": "code",
      "execution_count": 55,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 449
        },
        "id": "KSWpvAnakjK6",
        "outputId": "5f8363ee-8fb7-452b-a7b2-8028734f89cb",
        "trusted": true
      },
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 640x480 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAioAAAGwCAYAAACHJU4LAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjAsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvlHJYcgAAAAlwSFlzAAAPYQAAD2EBqD+naQAAVQ5JREFUeJzt3Xlc1HXiP/DXDDDDfd9yqoCIgICKeGReaamZ1mbpbvrNLrN73d3cysw2LTu2c2utTaufadaGuZl5lOCFF4KiIgoioFwKct8z798fIyOjyDnwmRlez8fj82COz8zn9ZmPu/PqM+/P5yMTQggQERERGSC51AGIiIiIboVFhYiIiAwWiwoREREZLBYVIiIiMlgsKkRERGSwWFSIiIjIYLGoEBERkcEylzpAd6jVauTn58POzg4ymUzqOERERNQBQghUVlbC29sbcnnb+0yMuqjk5+fD19dX6hhERETUBXl5efDx8WlzHqMuKnZ2dgA0K2pvby9xGiIiIuqIiooK+Pr6ar/H22LURaX55x57e3sWFSIiIiPTkWEbHExLREREBotFhYiIiAwWiwoREREZLKMeo0JERPqhUqnQ2NgodQwyERYWFjAzM9PLe7GoEBH1YUIIFBYWoqysTOooZGIcHR3h6enZ7fOcsagQEfVhzSXF3d0d1tbWPHkmdZsQAjU1NSguLgYAeHl5dev9WFSIiPoolUqlLSkuLi5SxyETYmVlBQAoLi6Gu7t7t34G4mBaIqI+qnlMirW1tcRJyBQ1/7vq7tgnFhUioj6OP/dQT9DXvysWFSIiIjJYLCpERERksFhUiIioTwsICMD777+vl/dKSEiATCbj4d56xKN+WiMEUFAA1NYCAwZInYaIiG5w++23Y+jQoXopGEeOHIGNjU33Q1GP4B6V1vzrX0C/fsCSJVInISKiLhBCoKmpqUPzurm58cgnA8ai0orKAG8AQO2JZImTEBH1HiEEqhuqJZmEEB3OuWDBAiQmJuKDDz6ATCaDTCbDunXrIJPJsG3bNsTExECpVGLfvn3IysrCzJkz4eHhAVtbWwwfPhy7du3Seb8bf/qRyWT44osvMGvWLFhbWyMoKAhbtmzp8uf63//+F2FhYVAqlQgICMC7776r8/y//vUvBAUFwdLSEh4eHrjvvvu0z/3www8IDw+HlZUVXFxcMGnSJFRXV3c5izGS/KefS5cu4W9/+xu2bduGmpoaDBw4EGvXrsWwYcMkyxQvz8BDABTZeUB9PaBUSpaFiKi31DTWwHaVrSTLrlpaBRtFx35++eCDD3D27FkMGTIEK1asAACcOnUKAPDiiy/inXfeQf/+/eHk5IS8vDzcddddeOONN6BUKvH1119jxowZyMjIgJ+f3y2X8dprr2H16tV4++238dFHH2HevHnIycmBs7Nzp9YrOTkZ999/P5YvX445c+bgwIEDePLJJ+Hi4oIFCxbg6NGjeOaZZ/DNN99g1KhRKC0txd69ewEABQUFePDBB7F69WrMmjULlZWV2Lt3b6dKnSmQtKhcvXoVo0ePxvjx47Ft2za4ubnh3LlzcHJykjIW/AeNRJkScKwHcPYsEB4uaR4iIrrOwcEBCoUC1tbW8PT0BACcOXMGALBixQpMnjxZO6+zszMiIyO1919//XXEx8djy5YteOqpp265jAULFuDBBx8EAKxcuRIffvghDh8+jKlTp3Yq63vvvYeJEyfilVdeAQAEBwfj9OnTePvtt7FgwQLk5ubCxsYG06dPh52dHfz9/REVFQVAU1Sampowe/Zs+Pv7AwDC++D3kaRF5a233oKvry/Wrl2rfSwwMFDCRBqD3cNw2g0YdRGoP5ECZR/8h0FEfY+1hTWqllZJtmx9uHFvfFVVFZYvX46tW7dqv/hra2uRm5vb5vtERERob9vY2MDe3l577ZrOSE9Px8yZM3UeGz16NN5//32oVCpMnjwZ/v7+6N+/P6ZOnYqpU6dqf3KKjIzExIkTER4ejilTpuCOO+7AfffdJ/l/zPc2SceobNmyBcOGDcMf/vAHuLu7IyoqCp9//vkt56+vr0dFRYXO1BPcbNxw3kvzc09p8r4eWQYRkaGRyWSwUdhIMunrLKY3Hr2zZMkSxMfHY+XKldi7dy9SU1MRHh6OhoaGNt/HwsLips9GrVbrJWNLdnZ2OHbsGDZs2AAvLy8sW7YMkZGRKCsrg5mZGXbu3Ilt27Zh8ODB+OijjxASEoLs7Gy95zBkkhaV8+fP49NPP0VQUBC2b9+ORYsW4ZlnnsFXX33V6vyrVq2Cg4ODdvL19e2xbGX9NQNqG08e77FlEBFR1ygUCqhUqnbn279/PxYsWIBZs2YhPDwcnp6euHDhQs8HvCY0NBT79++/KVNwcLD2Qn3m5uaYNGkSVq9ejRMnTuDChQv4/fffAWgK0ujRo/Haa68hJSUFCoUC8fHxvZbfEEj6049arcawYcOwcuVKAEBUVBROnjyJzz77DPPnz79p/qVLl+KFF17Q3q+oqOixsqIaFAIgG5YZ53vk/YmIqOsCAgJw6NAhXLhwAba2trfc2xEUFIQff/wRM2bMgEwmwyuvvNIje0Zu5c9//jOGDx+O119/HXPmzEFSUhI+/vhj/Otf/wIA/Pzzzzh//jxuu+02ODk54ZdffoFarUZISAgOHTqE3377DXfccQfc3d1x6NAhXL58GaGhob2W3xBIukfFy8sLgwcP1nksNDT0lr8dKpVK2Nvb60w9xSZyOADA+VIJ0M4uQiIi6l1LliyBmZkZBg8eDDc3t1t+b7z33ntwcnLCqFGjMGPGDEyZMgXR0dG9ljM6OhqbNm3Cxo0bMWTIECxbtgwrVqzAggULAACOjo748ccfMWHCBISGhuKzzz7Dhg0bEBYWBnt7e+zZswd33XUXgoOD8fLLL+Pdd9/FnXfe2Wv5DYFMSHic09y5c5GXl6c9FAsAnn/+eRw6dAgHDhxo9/UVFRVwcHBAeXm53kvLr+e2YfSQu2DXAODUKeCGQkVEZOzq6uqQnZ2NwMBAWFpaSh2HTExb/7468/0t6R6V559/HgcPHsTKlSuRmZmJb7/9FmvWrMHixYuljAXg+pE/ANB08oS0YYiIiPooSYvK8OHDER8fjw0bNmDIkCF4/fXX8f7772PevHlSxgIA+Nr74qyHZgjP1WP725mbiIj6gieeeAK2tratTk888YTU8UyS5GemnT59OqZPny51jJvIZDKUBngAxy6h/kSK1HGIiMgArFixAktucR24nhw32ZdJXlQMWeOgIACXoMjIlDoKEREZAHd3d7i7u0sdo0/hRQnbYBURAwBwyr0MdPAqnERERKQ/LCpt8B4ShxpzwKJJDZzn+VSIiIh6G4tKG0I9wpB+7cgf9amT0oYhIiLqg1hU2jDAaQDOuGuuP1F+7KDEaYiIiPoeFpU2WJhZoNhfs0ul9sRRidMQERH1PSwq7WgIHgAAMDtzVuIkRESkLwEBAXj//fe192UyGTZv3nzL+S9cuACZTIbU1NRuLVdf79MZ7a2boePhye1QhEcBSILThUJApQKuXe2SiIhMR0FBAZycnPT6ngsWLEBZWZlOSfD19UVBQQFcXV31uixTxj0q7fAMH4k6M0DRoAJycqSOQ0REPcDT0xNKpbLHl2NmZgZPT0+Ym3M/QUexqLRjkOcQZFwrvuLUKWnDEBH1JCGA6mpppk5cH3fNmjXw9vaGWq3WeXzmzJl4+OGHkZWVhZkzZ8LDwwO2trYYPnw4du3a1eZ73vjzyOHDhxEVFQVLS0sMGzYMKSm6ZyhXqVRYuHAhAgMDYWVlhZCQEHzwwQfa55cvX46vvvoKP/30E2QyGWQyGRISElr96ScxMREjRoyAUqmEl5cXXnzxRTS1OHfX7bffjmeeeQZ//etf4ezsDE9PTyxfvrzDn9eN0tLSMGHCBFhZWcHFxQWPPfYYqqqqtM8nJCRgxIgRsLGxgaOjI0aPHo2ca/+hfvz4cYwfPx52dnawt7dHTEwMjh7t2TGcLCrtCHEN0V6csDr1sLRhiIh6Uk0NYGsrzVRT0+GYf/jDH1BSUoLdu3drHystLcWvv/6KefPmoaqqCnfddRd+++03pKSkYOrUqZgxYwZyc3M79P5VVVWYPn06Bg8ejOTkZCxfvvym0+ar1Wr4+Pjg+++/x+nTp7Fs2TL8/e9/x6ZNmwAAS5Yswf3334+pU6eioKAABQUFGDVq1E3LunTpEu666y4MHz4cx48fx6effor//Oc/+Mc//qEz31dffQUbGxscOnQIq1evxooVK7Bz584Of2bNqqurMWXKFDg5OeHIkSP4/vvvsWvXLjz11FMAgKamJtxzzz0YN24cTpw4gaSkJDz22GOQyTRHwM6bNw8+Pj44cuQIkpOT8eKLL8LCwqLTOTpFGLHy8nIBQJSXl/foct65y1EIQBTMntKjyyEi6k21tbXi9OnTora2VvNAVZUQmn0bvT9VVXUq+8yZM8XDDz+svf/vf/9beHt7C5VK1er8YWFh4qOPPtLe9/f3F//85z+19wGI+Ph47Xu5uLhc/1yEEJ9++qkAIFJSUm6ZafHixeLee+/V3p8/f76YOXOmzjzZ2dk67/P3v/9dhISECLVarZ3nk08+Eba2ttp1GTdunBgzZozO+wwfPlz87W9/u2WWllqu25o1a4STk5OoavF5b926VcjlclFYWChKSkoEAJGQkNDqe9nZ2Yl169Z1aLk3/ftqoTPf39yj0gF1wf0BAPL0MxInISLqQdbWQFWVNJO1daeizps3D//9739RX18PAFi/fj0eeOAByOVyVFVVYcmSJQgNDYWjoyNsbW2Rnp7e4T0q6enpiIiIgKWlpfaxuLi4m+b75JNPEBMTAzc3N9ja2mLNmjUdXkbLZcXFxWn3WADA6NGjUVVVhYsXL2ofi4iI0Hmdl5cXiouLO7Ws5uVFRkbCxsZGZ3lqtRoZGRlwdnbGggULMGXKFMyYMQMffPABCgoKtPO+8MILeOSRRzBp0iS8+eabyMrK6nSGzmJR6QDzMM0/EIfsfOCG30SJiEyGTAbY2Egztfii7ogZM2ZACIGtW7ciLy8Pe/fuxbx58wBofnaJj4/HypUrsXfvXqSmpiI8PBwNDQ16+6g2btyIJUuWYOHChdixYwdSU1Pxf//3f3pdRks3/rwik8luGqOjL2vXrkVSUhJGjRqF7777DsHBwTh4UHPS0+XLl+PUqVOYNm0afv/9dwwePBjx8fE9kqMZi0oHuIWPRIMcUNY1Anl5UschIurzLC0tMXv2bKxfvx4bNmxASEgIoqOjAQD79+/HggULMGvWLISHh8PT0xMXLlzo8HuHhobixIkTqKur0z7W/EXdbP/+/Rg1ahSefPJJREVFYeDAgTftXVAoFFCpVO0uKykpCaLFYOL9+/fDzs4OPj4+Hc7cUaGhoTh+/Diqq6t1lieXyxESEqJ9LCoqCkuXLsWBAwcwZMgQfPvtt9rngoOD8fzzz2PHjh2YPXs21q5dq/ecLbGodMAg73Ccdbl25/RpSbMQEZHGvHnzsHXrVnz55ZfavSkAEBQUhB9//BGpqak4fvw45s6d26m9D3PnzoVMJsOjjz6K06dP45dffsE777yjM09QUBCOHj2K7du34+zZs3jllVdw5MgRnXkCAgJw4sQJZGRk4MqVK2hsbLxpWU8++STy8vLw9NNP48yZM/jpp5/w6quv4oUXXoBcrv+v6Hnz5sHS0hLz58/HyZMnsXv3bjz99NP405/+BA8PD2RnZ2Pp0qVISkpCTk4OduzYgXPnziE0NBS1tbV46qmnkJCQgJycHOzfvx9HjhxBaGio3nO2xKLSAaGuodojf+pOHJM2DBERAQAmTJgAZ2dnZGRkYO7cudrH33vvPTg5OWHUqFGYMWMGpkyZot3b0hG2trb43//+h7S0NERFReGll17CW2+9pTPP448/jtmzZ2POnDmIjY1FSUkJnnzySZ15Hn30UYSEhGDYsGFwc3PD/v37b1pWv3798Msvv+Dw4cOIjIzEE088gYULF+Lll1/u5KfRMdbW1ti+fTtKS0sxfPhw3HfffZg4cSI+/vhj7fNnzpzBvffei+DgYDz22GNYvHgxHn/8cZiZmaGkpAQPPfQQgoODcf/99+POO+/Ea6+91iNZm8lEy/1NRqaiogIODg4oLy+Hvb19jy7rnTtssWRnNS7PmQG3jVt6dFlERL2hrq4O2dnZCAwM1Bk4SqQPbf376sz3N/eodFB1UAAAQPCnHyIiol7DotJBZoOHAADss/I6dQZFIiKinrJ+/XrY2tq2OoWFhUkdTy94sYEOco6MRZPsO1jWNAD5+UC/flJHIiKiPu7uu+9GbGxsq8/1+BljewmLSgeFeEcg0xkYVALNkT8sKkREJDE7OzvY2dlJHaNH8aefDgp1u37kT2PacWnDEBHpUU+dOIz6Nn39u+IelQ7ysvVClpcCONOAitRDcGn/JUREBk2hUEAulyM/Px9ubm5QKBQ6p3In6gohBBoaGnD58mXI5XIoFIpuvR+LSgfJZDJUDfQDdmdCfeqk1HGIiLpNLpcjMDAQBQUFyM/PlzoOmRhra2v4+fl1+8R1LCqdEToYQCZsz13QHPnD//IgIiOnUCjg5+eHpqamdk/3TtRRZmZmMDc318seOhaVTnCMjIVKtgVWlXVAURHg6Sl1JCKibpPJZLCwsDCZo0TItHAwbScE+UTgvNO1OzzxGxERUY9jUemEltf84TgVIiKinsei0gkBjgE4624GAKhMOSRxGiIiItPHotIJZnIzlA3wBgA0njwhcRoiIiLTx6LSSSI0FABgdS5b4iRERESmj0Wlk+wihgMAbMqqgcuXJU5DRERk2lhUOmmgbySyHa/dSU+XMgoREZHJY1HppJbX/BGnTkkbhoiIyMSxqHRSkHMQ0q8VlerjR6QNQ0REZOJYVDpJaa7ElQB3AEB9Wqq0YYiIiEwci0oXqEJDAADKs1kSJyEiIjJtLCpdYBM+DABge6UCuHpV4jRERESmi0WlC/oHDEWe/bU7PPKHiIiox7CodEHLa/7w4oREREQ9h0WlCwa5DsKpa0Wl7sQxacMQERGZMBaVLrBT2qHAzwkAUHs8WeI0REREpotFpYsaQwYCACwyzkmchIiIyHSxqHSRVXg0AMC26CpQUSFxGiIiItPEotJFAYFRyLe9dodH/hAREfUIFpUuannNHx75Q0RE1DNYVLqo5SHKDWnHpQ1DRERkolhUusjNxg25/WwAALUnjkqchoiIyDSxqHRDXXB/AIBZ+lmJkxAREZkmFpVuUIQPBQDY5l8GqqulDUNERGSCWFS6wW9ANIqtr905c0bSLERERKaIRaUbeM0fIiKinsWi0g0tD1FWnUyTNgwREZEJYlHpBl97X2R5KgAANSd4zR8iIiJ9Y1HpBplMhqrgAM1t/vRDRESkdywq3WQ+JBwAYH2xCKitlTgNERGRaWFR6aZ+A6JQYgXI1QLIyJA6DhERkUlhUemmwe5hPPKHiIioh7CodFPLQ5TFyZPShiEiIjIxLCrdNMB5ANI9NB9j7bHDEqchIiIyLSwq3WQuN8flUD8AgPzYMYnTEBERmRYWFT0QkZFQyQDLy1eB/Hyp4xAREZkMFhU9GOgTcX1AbTJP/EZERKQvLCp6EOUZhWSva3eOHpU0CxERkSmRtKgsX74cMplMZxo0aJCUkbpkeL/hSPbW3FYd5YBaIiIifTGXOkBYWBh27dqlvW9uLnmkTutn1w/ZA5wBlEJ15DDMhABkMqljERERGT3JW4G5uTk8PT2ljtEtMpkMljGxUMm2QXG5VDOgtl8/qWMREREZPcnHqJw7dw7e3t7o378/5s2bh9zc3FvOW19fj4qKCp3JUEQEjsQpDqglIiLSK0mLSmxsLNatW4dff/0Vn376KbKzszF27FhUVla2Ov+qVavg4OCgnXx9fXs58a0N974+ToUDaomIiPRDJoQQUodoVlZWBn9/f7z33ntYuHDhTc/X19ejvr5ee7+iogK+vr4oLy+Hvb19b0a9yZWaK3j1D2745BegcepkWGzbIWkeIiIiQ1VRUQEHB4cOfX9LPkalJUdHRwQHByMzM7PV55VKJZRKZS+n6hhXa1fkB3sCvxRCHDkCcEAtERFRt0k+RqWlqqoqZGVlwcvLq/2ZDZD1sFFokgGKkjLg0iWp4xARERk9SYvKkiVLkJiYiAsXLuDAgQOYNWsWzMzM8OCDD0oZq8siA2Jxyv3aHY5TISIi6jZJi8rFixfx4IMPIiQkBPfffz9cXFxw8OBBuLm5tf9iAzTce/j1M9TyyB8iIqJuk3SMysaNG6VcvN7FeMfgB2/g4VSg/nASDHM0DRERkfEwqDEqxs5eaY8rg/01d44e1QyoJSIioi5jUdEz25hRaJQDytJy4OJFqeMQEREZNRYVPRsaGHf9DLUcUEtERNQtLCp61vJKyoJFhYiIqFtYVPQs0iMSKf00H2vdof0SpyEiIjJuLCp6ZmVhhbKwAQAA+bEUDqglIiLqBhaVHmA/fIxmQO3VCiAvT+o4RERERotFpQdEBcThJM9QS0RE1G0sKj1geL/hOMoBtURERN3GotIDwtzCcMJHc9LfmoN7JU5DRERkvFhUeoCFmQVqwkMBAGYpqRxQS0RE1EUsKj3EYcRYNMgBy7IqIDdX6jhERERGiUWlh0T5j+SAWiIiom5iUekhLQfUqo8ekTYMERGRkWJR6SHBLsE47WcJAKg+kChxGiIiIuPEotJD5DI5aocOAQBYHE/jgFoiIqIuYFHpQS7DbtMMqC2vBi5ckDoOERGR0WFR6UFRASOR5nHtTnKypFmIiIiMEYtKD2o5oLbpyCFpwxARERkhFpUe5O/gjwx/WwBAdRIH1BIREXUWi0oPkslkaIyKAAAoj5/igFoiIqJOYlHpYW4jxqPeDLCsqAGys6WOQ0REZFRYVHpYdMBInOCAWiIioi5hUelhw7yHIdlLc7v+cJK0YYiIiIwMi0oP87T1xPn+jgCAGg6oJSIi6hQWlV6gio4CAFidSOeAWiIiok5gUekF7s0DaitrgfPnpY5DRERkNFhUekFMQByOc0AtERFRp7Go9IJh3sOQfO0MtbVJe6UNQ0REZERYVHqBo6Uj8oLcAQA1h/ZJnIaIiMh4sKj0EhEdDQCwSTvDAbVEREQdxKLSSzxjJ6LODLCsqgOysqSOQ0REZBRYVHpJjP9IHPe8docDaomIiDqERaWXRHlG4di1AbWV+3dLG4aIiMhIsKj0EhuFDQpC+gEA6g7vlzgNERGRcWBR6UWymGEAALuT5wC1WuI0REREho9FpRf1GzkZteaAZXU9B9QSERF1AItKL4r2i9WeoVYcPSptGCIiIiPAotKLIjwikNpP85GX7/9N4jRERESGj0WlFynMFCgO9QMA1B9OkjgNERGR4WNR6WVmw0YAABxOZXJALRERUTtYVHqZz8g7NANqaxqAzEyp4xARERk0FpVeNsxvJFKvnaFWffiwtGGIiIgMHItKLxvkOgjHfcwBAKX7dkqchoiIyLCxqPQyM7kZrgzpDwBQHTwgcRoiIiLDxqIiAfO40QAAp/RsoLFR4jRERESGi0VFAgOGT8FVS0DRoALS0qSOQ0REZLBYVCQQ6xeHw5rrE6IhaZ+0YYiIiAwYi4oEfO19cTrQBgBwNfFXidMQEREZLhYVCchkMlRHDQEAmB05JnEaIiIiw8WiIhG7MRMBAM45RUBFhcRpiIiIDBOLikQiwich2xGQCwC8kjIREVGrWFQkMsx7mHZAbcXeXdKGISIiMlAsKhKxU9ohN0RzLv2qvb9LnIaIiMgwsahISDU8BgBgk3pK4iRERESGiUVFQu5jp6JJBjiUVAEXL0odh4iIyOCwqEho2MDbkOahua06eFDaMERERAaIRUVCYW5hSPG9diXlxF8kTkNERGR4WFQkZCY3w5XwAQCAxiReSZmIiOhGLCoSsxipuZKy8+nzgEolcRoiIiLDwqIisYC4O1GpACxrG4H0dKnjEBERGRQWFYnF+o/CUW/N7br9idKGISIiMjAsKhLztvNGeqAtAKCUV1ImIiLSwaJiAKqjwwEAZkeTJU5CRERkWFhUDIDdmAkAANesQqCmRuI0REREhoNFxQCERU3BJTvATC0gkrlXhYiIqJnBFJU333wTMpkMzz33nNRRel2MdwyOXLuScnniDmnDEBERGRCDKCpHjhzBv//9b0REREgdRRLWFtbIHeQFAKjc/5vEaYiIiAyH5EWlqqoK8+bNw+effw4nJ6c2562vr0dFRYXOZCpUw65dSTmFV1ImIiJq1qWikpeXh4strvZ7+PBhPPfcc1izZk2n32vx4sWYNm0aJk2a1O68q1atgoODg3by9fXt9PIMldttd0INwLmoAigqkjoOERGRQehSUZk7dy52794NACgsLMTkyZNx+PBhvPTSS1ixYkWH32fjxo04duwYVq1a1aH5ly5divLycu2Ul5fXlfgGKWbQeKS7aW6rDiZJG4aIiMhAdKmonDx5EiNGjAAAbNq0CUOGDMGBAwewfv16rFu3rkPvkZeXh2effRbr16+HpaVlh16jVCphb2+vM5mKENcQpPhZAAAuJ2yVOA0REZFh6FJRaWxshFKpBADs2rULd999NwBg0KBBKCgo6NB7JCcno7i4GNHR0TA3N4e5uTkSExPx4YcfwtzcHKo+doE+uUyOEu2VlPdLnIaIiMgwdKmohIWF4bPPPsPevXuxc+dOTJ06FQCQn58PFxeXDr3HxIkTkZaWhtTUVO00bNgwzJs3D6mpqTAzM+tKNKNmHjsKAOB8MgtQqyVOQ0REJD3zrrzorbfewqxZs/D2229j/vz5iIyMBABs2bJF+5NQe+zs7DBkyBCdx2xsbODi4nLT432F/5jpqDX/EjbVDUBmJhAcLHUkIiIiSXWpqNx+++24cuUKKioqdA4pfuyxx2Btba23cH3NiMDROOYFjM4DavbthjWLChER9XFd+umntrYW9fX12pKSk5OD999/HxkZGXB3d+9ymISEBLz//vtdfr2xc7dxx5n+mgHCJQnbJE5DREQkvS4VlZkzZ+Lrr78GAJSVlSE2Nhbvvvsu7rnnHnz66ad6DdjXVEeFAQDMjhyVOAkREZH0ulRUjh07hrFjxwIAfvjhB3h4eCAnJwdff/01PvzwQ70G7Gtsx04EALidywfq6yVOQ0REJK0uFZWamhrY2dkBAHbs2IHZs2dDLpdj5MiRyMnJ0WvAviZ02J24bA1YqARESorUcYiIiCTVpaIycOBAbN68GXl5edi+fTvuuOMOAEBxcbFJnYRNClHe0TjsIwMAlCb+KnEaIiIiaXWpqCxbtgxLlixBQEAARowYgbi4OACavStRUVF6DdjXWJpb4mLzlZT37pI4DRERkbS6dHjyfffdhzFjxqCgoEB7DhVAcxK3WbNm6S1cX9UUEw1syYd1Kq+kTEREfVuX9qgAgKenJ6KiopCfn6+9kvKIESMwaNAgvYXrq9zGTwMAuF8qA0pLpQ1DREQkoS4VFbVajRUrVsDBwQH+/v7w9/eHo6MjXn/9dah56vduixo8EeecNbcbDx6QNgwREZGEulRUXnrpJXz88cd48803kZKSgpSUFKxcuRIfffQRXnnlFX1n7HMGOg9Eqr8CAFC0+2eJ0xAREUmnS2NUvvrqK3zxxRfaqyYDQEREBPr164cnn3wSb7zxht4C9kUymQxXhvQHUs6gKWmf1HGIiIgk06U9KqWlpa2ORRk0aBBKOaZCL8xHjgYAOKdlAkJInIaIiEgaXSoqkZGR+Pjjj296/OOPP0ZERES3QxHgd9sMNMgB+4p64MIFqeMQERFJoks//axevRrTpk3Drl27tOdQSUpKQl5eHn755Re9BuyrhvUfg1RPYEQ+ULXvd9gGLpQ6EhERUa/r0h6VcePG4ezZs5g1axbKyspQVlaG2bNn49SpU/jmm2/0nbFPcrF2wdkBDgCAK7u3SpyGiIhIGjIh9DcA4vjx44iOjoZKpdLXW7apoqICDg4OKC8vN8lT93/27Gg88eEB5A7xhV9artRxiIiI9KIz399dPuEb9TzbMRMAAB4Zl4DGRonTEBER9T4WFQMWMnI6ypSAslENkZYmdRwiIqJex6JiwCK9o3D02pWUL//OE78REVHf06mjfmbPnt3m82VlZd3JQjdQmCmQN8gbyLqEin2/wX3JMqkjERER9apOFRUHB4d2n3/ooYe6FYh0qYbFAFsvwTr1pNRRiIiIel2nisratWt7KgfdgsvtdwKvbYFnbilQUQGY4NFNREREt8IxKgZuaOQUXHAA5AJoOMQrKRMRUd/ComLgAhwDkByoBAAUb9kgcRoiIqLexaJi4GQyGfJGhQEAzHbskjgNERFR72JRMQJOd88BAHidzQeKiiROQ0RE1HtYVIzAuNj7keyluV29NV7aMERERL2IRcUIBDgG4Ei4CwCg5Mf1EqchIiLqPSwqRqJx0ngAgNPeo4BaLXEaIiKi3sGiYiQGzfg/VCgAu4o6iORkqeMQERH1ChYVIzF24ATsHqjZXEX//VriNERERL2DRcVIWJpbIjduMACg6RdeoJCIiPoGFhUjYjf9PgCA18kLwNWr0oYhIiLqBSwqRmTMbfNw2hUwE0Dtr9yrQkREpo9FxYgMdB6Ig0M0V7Au5jgVIiLqA1hUjEztxHEAAPuEJEAIidMQERH1LBYVI9N/5gLUmANOJdUQJ09KHYeIiKhHsagYmdtC7sCeQBkA4PKP30ichoiIqGexqBgZG4UNzscGAwDqft4sbRgiIqIexqJihCynzwIAeKZmAlVVEqchIiLqOSwqRihu/J9w3hFQNAnU7doudRwiIqIew6JihAa5hWJ/mC0AoPCHddKGISIi6kEsKkZIJpOhcvxoAIDN73slTkNERNRzWFSMlN8989EgB9wKyoHMTKnjEBER9QgWFSM1bsh07Pe/dpjyf3mYMhERmSYWFSNlp7TD2eGBAIDq//1X4jREREQ9g0XFiCnunAEA8Dx6BqirkzgNERGR/rGoGLGYqf+HAlvAsl6FhsTfpY5DRESkdywqRizcIwJ7Qq0AAPnffylxGiIiIv1jUTFiMpkMV8fFAgCUuxKkDUNERNQDWFSMnPesh6CSAV45JcDFi1LHISIi0isWFSN3W/QsHOmnuX0lfr20YYiIiPSMRcXIOVo64lSMHwCg/KdNEqchIiLSLxYVEyCfeicAwDPpBNDUJHEaIiIi/WFRMQGR0xeixAqwqWlC44F9UschIiLSGxYVEzC0Xwz2hCgBAJe+/4/EaYiIiPSHRcUEyGVyXB47DABgtmOXxGmIiIj0h0XFRLjdMxcA4Hu2ECguljgNERGRfrComIjbRs5BiqfmdumW76QNQ0REpCcsKibCxdoFJ6K8AACl8d9KnIaIiEg/WFRMiLhjCgDAbV8KoFZLnIaIiKj7WFRMyOC7F6JCAThU1KMp+YjUcYiIiLqNRcWExATEYW+QBQDg4iYepkxERMaPRcWEmMnNUDB6qObOr9skzUJERKQPLComxnnmAwAAn9MXgatXJU5DRETUPSwqJmb0bfOQ5g6Yq4Hyj96ROg4REVG3SFpUPv30U0RERMDe3h729vaIi4vDtm38yaI7PGw98L8ZwQAA+Xv/BKqrJU5ERETUdZIWFR8fH7z55ptITk7G0aNHMWHCBMycOROnTp2SMpbRG/XXj3HeEbArr0XJ+6ukjkNERNRlMiGEkDpES87Oznj77bexcOHCduetqKiAg4MDysvLYW9v3wvpjMd7CwfjhS/TUeZoBcf8EsDKSupIREREADr3/W0wY1RUKhU2btyI6upqxMXFtTpPfX09KioqdCZq3bhX/oMcB8CxrBb5778udRwiIqIukbyopKWlwdbWFkqlEk888QTi4+MxePDgVuddtWoVHBwctJOvr28vpzUeMQFx2H7fUACA4p1/AvX10gYiIiLqAsl/+mloaEBubi7Ky8vxww8/4IsvvkBiYmKrZaW+vh71Lb5wKyoq4Ovry59+biHj4nHYDh6KfpXA+VV/Rf8X35I6EhERUad++pG8qNxo0qRJGDBgAP7973+3Oy/HqLTv20dHYu4Xh1DoooRHfgVkCoXUkYiIqI8zyjEqzdRqtc5eE+qesa9/jUJbwLOkHqfeWyp1HCIiok6RtKgsXboUe/bswYULF5CWloalS5ciISEB8+bNkzKWSfH1DMahB8YCABze+wTqxgaJExEREXWcpEWluLgYDz30EEJCQjBx4kQcOXIE27dvx+TJk6WMZXJGvfE1rlgDvpfrceTdP0sdh4iIqMMMboxKZ3CMSsftenQiJn3xO867K+B3sQLmFkqpIxERUR9l1GNUqGeMWPkVrlrJ0L+4AXvefUbqOERERB3CotJH2Lv5IH3eHQCAfh98ibqGGokTERERtY9FpQ+JXrUOFZYyhBQ2Yce7T0odh4iIqF0sKn2IpasnsuZNAwAEfPz/UFFXLnEiIiKitrGo9DHhq75AtVKGiHwVfn7vCanjEBERtYlFpY8xd/NA3h/vBgCE/GsTLlcVS5yIiIjo1lhU+qDgNz5DrUKOmEtq/Pj+41LHISIiuiUWlT5I7uGJoj/eAwCIWPMTcstypA1ERER0CywqfZT/6x+h3kKOuDyB7z7kXhUiIjJMLCp9lMzbG6V/vBcAMHztdqRfTpc4ERER0c1YVPowrxXvodFcjtsvAN9+wiOAiIjI8LCo9GU+PqiYdx8AYNzXe/BN6tcSByIiItLFotLHuby2Gk3mckzKBk69+DCOXDoidSQiIiItFpW+zt8f8nfeAwCs3K7CZy9PQUFlgcShiIiINFhUCPJnnkHDE49CDuCj/3cVL62egvqmeqljERERsagQAJkMio/+heqJY2HdBLzxfhpe/mo+hBBSJyMioj6ORYU0zM1h8+PPqAwOgFcV8MeXvsOahHelTkVERH0ciwpdZ28Pux0JqHa2Q2QR4Pv4X5BwbpfUqYiIqA9jUSFd/v6w3rYL9Qoz3HUOOLtgBrKvZkudioiI+igWFbqJbMQI4OtvAACPHajDd0+MQVVDlcSpiIioL2JRoVYp5zyI8uVLAQB/+T4fH7w6hYNriYio17Go0C05LHsDRXOmw0wAz7x3AJ+ve1rqSERE1MewqNCtyWTw+Pq/yB82CHYNwNTnPsGve9ZKnYqIiPoQFhVqm0IB7x0HUOTjCL8KwG3uIzidzdPsExFR72BRofY5OcH5tySU25oj5pIaefeMR2n1FalTERFRH8CiQh1iETwIIj4e9WbAlBPVSJw2BDmXTksdi4iITByLCnWY46TpKPzgDQDArMQiWIQOwc7l86FWNUmcjIiITBWLCnWK/+K/I//rT5DvZgnvSoHJr32NswOdcHHbd1JHIyIiE8SiQp3m/acn4ZlTggOLZ6JCCQy6UAWfux5AxqShUJ3PkjoeERGZEBYV6hK5lTVGfbwZZScOY+vt/aAGEPLbcTSFBOHKc48BlZVSRyQiIhPAokLd4hc8HHf9nofNG5ZhT38zKJsEXD/4HFUB3lB9vgZQqaSOSERERoxFhbpNJpNh9gOvITD5PFY8H41zzoBtaRXMHnsctUPDgIQEqSMSEZGRYlEhvfF19MMr7x7Foe3/wUvTLFGmBKxOZgDjx0N951QgMRHg9YKIiKgTWFRIr2QyGf447GE8teE8nv3oTnwyHFDJAPmv24Hbb0fDiGFAfDygVksdlYiIjACLCvUILzsvrHtkK1y/3IjRf3XBZzFAnRmgOHoMmD0bNUEBEF98AdTXSx2ViIgMGIsK9RiZTIY5Q+Yg8R+XYPfl/8P9bw/HyjFAmRKwPp8H2aOPosrHHbVvrADKy6WOS0REBkgmhPEOGqioqICDgwPKy8thb28vdRzqgBNFJ7A28QNYrvt/WLyvAT7XjmKutbZAxf/Ng8dLKwEvL2lDEhFRj+rM9zeLCkmior4C3yavw6XPVmPutksIvXaNwwZzGXJm3Abf55bBcvQ4wMxM2qBERKR3LCpkNIQQ2HdhDw5/9gpGbdiHuLzr/xwr7BS4etsIeDywEJbTZgJOThImJSIifWFRIaNUVFWEHV8tg+OX6zHmdDWc6q4/p5LLUDI0BA73Pgjl3bOBsDBAJpMuLBERdRmLChk1IQSO5R3G0R8+BH7ZijEnyhF2WXeeKm9XKGbcA8Xds4Dx4wErK2nCEhFRp7GokMkQQiC1MBU7f/8cNZu/x4jUK5iQDVi2ODN/k6UCsmnTYPbgPOCuu1haiIgMHIsKmSQhBI4XHcfmo+tRsGU9oo4VYNo5wLeixTy2tpDNnAk88ABwxx2AQiFdYCIiahWLCpk8IQROFJ3AD6e+R/LWz3H7oWLMOQX4tzwdi6MjMHu2prSMHw+Ym0sVl4iIWmBRoT6lvqkeXx3/Cqv2roTnyRzMOQk8cFoGz8oW/7Td3ID77tOUljFjADnPdUhEJBUWFeqTGlWNWJ+2Hiv3rkTWlXMYmwM8dEaBOWfMYVNec33Gfv2Av/0NePxx/jRERCQBFhXq05rUTdh0ahP+secfSL+SDjMVMP2iFV7JD0LUoRzIy679PhQQALz+OvDggzyxHBFRL2JRIQKgFmr89/R/8Y+9/8CJohMAACeZNT6/Mgp3f5cKi+Jrp8MNDwdWrgSmTeO5WYiIekFnvr/5Qz2ZLLlMjj+E/QEpj6dg85zNiPGKwVVRg/tcdsHxkSv450x31NlYAmlpwIwZwNixwL59UscmIqIWuEeF+gwhBLZlbsNnRz/Dr5m/olHdCKca4K/7gecOy2DZeO1/CtOmafawRERIG5iIyETxpx+idpTVlWHzmc3YdGoTdp7fCfeyJrySCDxyDDAXgJDJUH3fTNi++S7Qv7/UcYmITAqLClEnlNSUIP5MPL479R3yjvyG5b8LPHBK81yjmQzpdw6D9e2T4TP6TlhGRAPW1tIGJiIyciwqRF1UXF2MH9N/RMrWLzD7m2RMydJ9Xi0DrnjaozZ0IKyjYuESezvkEZHAwIE8coiIqINYVIj0oLCqEEnfrILZT/+DU9ZFhOQ3wr2m9XkbFeaoHuAHxdAYWA8bCURHA1FRgIND74YmIjICLCpEeiaEQG55Lo4f34GCpB1oOH4MDmdzEFqoQlgxYN3U+utUAwJhFjMciInRTFFRgLNz74YnIjIwLCpEvaBJ3YRTxadwJPcgzqf8hppjh+BwLg+RBQIxBTdcd6ilwEDNHpeYGM3fkSO554WI+hQWFSKJVNZXYl/uPuw8vxOHj/8Cq5MZiMkHoguAmAJgwNVWXqRUAnffDcyfD0yZwosnEpHJY1EhMhCXKi5h1/ld2Hl+J3ad34X6K0WIKrheXEYWmCGwRKWdX+3pAfm8P2pKS3i4hMmJiHoOiwqRARJCIK04DTuydmDn+Z3Yk7MHdY11GFoIzE8F5qUBbi0G6xYF90PlA7Ph+ejzsPUJlCw3EZG+sagQGYG6pjrsz92PgxcPIrkgGal5RzAk+SIWpALTzwIKtWa+RjmwJ8wWJ++Mhmz6DIwLmowIjwjIeF0iIjJSLCpERqq4uhjJ+ck4nb4HdvFbEfvbGUTmNWqfL7ECtg8Acvs7w3X0ZETftRBRYZNYWojIqLCoEJmQK0cSUf75R3D/cQfsSipvej7f0QylIf5wGDkO/cZNhzwqGvD355WgichgsagQmaKmJiAxEfUH9qB433ZYHD8Jz6LqVmdtdLCDeXQMZFHRwPDhQGwsEBDA8kJEBoFFhaiPqLlSgKPb/oMLCZshTz2OIZeaEHYZsFDfPK/KzRXyuFGQxcZqzt0yfDhgZ9f7oYmoz2NRIeqD6pvqsev8Lmw+8R3O7tmMATmViC4ARlwCogpvLi9qGXC1vzfqoiOgGH0bnMbfCfOwcF6ziIh6nNEUlVWrVuHHH3/EmTNnYGVlhVGjRuGtt95CSEhIh17PokLUukZVI3Zf2I2fz/6MM1fO4GLRObicycWwi2qMvAjEXgQCWjlzbrVChlJ3WzR6uMHCxx8OgYNgFxACmbc34O0NeHlpJiur3l8pIjIZRlNUpk6digceeADDhw9HU1MT/v73v+PkyZM4ffo0bGxs2n09iwpRxzWqGpFTnoPM0kxklWahOPM4FEdT4H4qG8HnShFzUcC2sf33AQDh5ASZlxfg6wuMGAGMHQvExQG2tj27EkRkEoymqNzo8uXLcHd3R2JiIm677bZ252dRIdIPtVAj/2ou8o7+jvwzR3DlfBpqc7IgLyyCZ4WAdyXgVQV4VwJWt7gAozAzgywqSlNaxozRTO7uvbsiRGQUjLaoZGZmIigoCGlpaRgyZMhNz9fX16O+vl57v6KiAr6+viwqRD2krqkOpy+fRmphqmYqSEH2hVTYlVTBuxIYWAqMzgPG5rT+UxJCQjTFpXnikUdEBCMtKmq1GnfffTfKysqwb9++VudZvnw5XnvttZseZ1Eh6j1qocaFsgtILUzF0fyjSLqYhMOXDsP5Sg3G5gBjc4ExuUB48c2vFa6ump+M3NwAV1fN3+ap5X1XV83ECzQSmSSjLCqLFi3Ctm3bsG/fPvj4+LQ6D/eoEBmmJnUTThSdwIG8AziQdwBJF5NQnn9Bu7dlbC4wLL/1w6bbIhwdIHNwBBwdAQeH9v/6+wNBQdxrQ2TgjK6oPPXUU/jpp5+wZ88eBAZ2/OJrHKNCZLjyK/ORlJeEpItJOJB3AKdzjiKwuBFu1YBrjeYCjG7Vmr+uN9x2qQHkXV2wm9v1MTJjxgBRUYCFhT5XjYi6yWiKihACTz/9NOLj45GQkICgoKBOvZ5Fhch41DfVI+tqFkprS9udyqpLIEpKYFFWCYd6wKEOcKwDHOo1fz1VVhggc4Ev7OHRqIRTvQzW1fWQncsEWux1BQBYW2vOzNs8yHfkSJ7ojkhiRlNUnnzySXz77bf46aefdM6d4uDgAKsOnKeBRYXItJXWluJ44XGkFKYgtTAVKYUpSL+cDpVQ3TSvpbklYpyHYGaVD8ZdNENI+mXYH02D7OpV3RnlcmDoUE1piYnRDPgNDgacnHpnpYjIeIrKra74unbtWixYsKDd17OoEPU9dU11OFV8Sqe8HC88jurGm697pJRZYKYIxt3FzhiR3QC/k3lQ5uW3/sZubprC0lxcmv8OGAAolT28VkR9i9EUle5iUSEiQHMkUmZpJo4VHENyfjKSC5JxrOAYyutvPmY6sMoCc8p8cEeBNQYWNsA1rwRWxaW3fnO5XHNYdXCw5qy8jo43T05OuvdtbTmgl6gNLCpE1OcJIXD+6nkkFyRry0tyQTLK6spumtemHggqBYaUmmNYlT3Cyiww4HITvPIrYVnT0PmFy+WawuLtDfj43Hqyt2ehoT6JRYWIqBVCCGSXZSM5X7PHJetqFnLLc5FXkYeCygII3PB/hwLwqAJCSoCgEsCtVoYAYQ8/YQ/PJiu4NpjDoVbAuroBZuUVkJWVAY0dvA4BANjYXC8tvr6an5mCgjTTwIGaIkNkglhUiIg6qUHVgEsVl5BbnqstL823m6fKhspbvt5B6YBBLiEItx+ICIUfQs084FMlh1tpPRyuVMK8oAi4ePH6VNrGz03N3N2vF5eWBSYoiNdVIqPGokJEpGdCCBRVF+HMlTPIuJKh+Vui+Xuh7MLNe2Nu4GjpCA8bD3jaesLD1gO+Zi4YUGsJ/ypzeJep4F5aB9f8cijOXwDOnQOKWzm1b0seHrpXtG45NT/u6QkoFPr7EIj0hEWFiKgX1TbWIrM0U6e8nCs9h4LKAhRWFaJR3fGfg7xsvTDIdRAiLQMwotYZg8ss4He5Ho55lzXnicnMBK5c6Xg4FxdNafHw0D2Tb3tn+XVw0Iy1IeoBLCpERAZCCIGyujIUVhWiqLpI87eq6Prta38vVVxCUXXRLd/HytwKwS7BCHENwVCFPyJr7eFZKeBa1giHshrYXqmEWVExkJ8PFBQAhYWdGy9zI0tLzSHaoaG6U1AQD9embmNRISIyQuV15cgoydD+tHSm5AzOXDmDzNJMNKjaP/rIVmELN2s3uNm4wd3SFf5qOwTWWsK3xhwe1TI41cvhUCdgW6eCdVUjlNV1kJeXA+XlQFnZ9b91dbdeiJkZ0L//zQXG0RGorgaqqjR/25pqajQXnLS11Zwl2NZWd7rxMTs7zcSCZDJYVIiITEiTugkXyi7ojI85X3Yel6sv43LNZVypuYImdVOn31cGGRwtHeFi7QIXKxe4WrvCxdoF7mYOGFCtQFBRE3zzq+CWewW2WRdhnnEOsoqKHljDDlIqr/8s1dbk5KQ5+/CQIbwCt4FiUSEi6kOEECivL8flak1puVxzWff2tTJTUlOi+Vtb0ur5ZNqjkFsgUu2O2EoHRJZaILhYDb/8arjnlcC8vhEqK0uorC2htrKEytoKamtLqK2toba2grC2grC2hrCxBmxsYCe3gm0DIKuuBiorNXtiWk4tH6up6doHY2Ojuc7TqFGaaeRIXirBQLCoEBFRm5rUTSitLdUpL823r9RcQXFNMQoqC1BQVYD8ynyU1nbgcOpOslXYIsg5CEEuQZq/124HuwTDxcrl+mVWVCpNcWn+maq9qagIOHoUaG3vT2jo9eIyapTmjMMcNNzrWFSIiEiv6prqUFhVqFNeCioLkF+l+VvVUAWVUEGlVun8bVI33fRYo6oRl2suQy3Ut1yeo6WjTokJdAyEv6M//Bz84GPvA4VZO4ddq1RAejpw4IBmSkoCzp69eT4nJyA6GrCwAJqaOjbJZJrXubgAzs6aqa3bTk78CeoGLCpERGTQ6pvqcf7qeZwrPYdzJec0f0vP4WzJWVysuNjma2WQwcvOC/4O/pryYu+nLTH+Dv7wdfCFncIOZnIz3RdevgwcPHi9vBw5AtTW9uBaXmNmBvTrB/j5aSZ//5tv29npZ1lqtWYwdPNUW6t7u75eU578/DQFSqJLOLCoEBGR0apprEFWaZZOickpz0FOWQ5yy3NRr6rv0PsozZSwtrC+abJR2MDawhp2MksEX6xD//wa2Fk6wN7GGQ7WznCwddFMNi6wUFpp9oa0nFQqzZmFm6eSklvfLyvr2Eo7OV0vLw4OQEOD5vDyhgbd2609Vl9/vYg0dOLaVDY2rZen5qlfvx47YSCLChERmSQhBIqri5Fbnoucck1xySnLQW5FrrbIlNSW6G15LlYu8LT11Jn62fVDiGsIQl1D4e/oD7msjTEuTU2aswzn5mqmnBzdv7m5wNWresurQy4HrKw0k6WlZlIoNCcMbO/Mx4Bmb4uXF3DffcAHH+g1GosKERH1WbWNtahurEZNY02Hpor6ChRXF6OwqlA7FVUXdeiQbytzK21pCXUNRaib5m+QS1D742iaVVQAeXma8pKToznXjFKpGTejUGim5ts3PmZhoSkgLctI8+22xsXU1mquOdVcllpOzSWq/tqeq4cfBv7zn46tSwexqBAREXWDWqhRWluqPZNwyxKTU56D9CvpOFty9pYn4jOTmWGA8wCEuobCz8EPZjIzyGVyyGVymMmv325tEkKgQdWgnRrVjbe836hqhIWZBWwVtrCxsIGtwvam6cbHA50C4Wzl3PYHIIRmTE9uruYnotBQvX6+LCpEREQ9rEndhOyr2Ui/ko70y+mav9dut3WlbUPgbeeNIe5DEO4erv0b6hYKawvrXlk+iwoREZFEhBDIr8zXlpbCqkIICKjUKqiFut0JAJTmSljILaAwU0BhpoCFWYvbNzzeqGpEVUOVdqpurNa53/Kx8rpyFFQVtJpbBhkGOg/ULTAe4RjoPBDmcv0eXs2iQkRERK2qqK/A6cunkVaUhrTiNJwsPom04jRcqWn9qtx3DLgD2/+4Xb8ZOvH9zTPQEBER9SH2SnuM9BmJkT4jtY81H02VVpyGtKLr5eXU5VMIddXv+JTOYlEhIiLq42QyGTxsPeBh64FJ/SdpH1cLNWobe+GkeG3gBQ6IiIioVXKZHDYKG2kzSLp0IiIiojawqBAREZHBYlEhIiIig8WiQkRERAaLRYWIiIgMFosKERERGSwWFSIiIjJYLCpERERksFhUiIiIyGCxqBAREZHBYlEhIiIig8WiQkRERAaLRYWIiIgMlrnUAbpDCAEAqKiokDgJERERdVTz93bz93hbjLqoVFZWAgB8fX0lTkJERESdVVlZCQcHhzbnkYmO1BkDpVarkZ+fDzs7O8hkMr2+d0VFBXx9fZGXlwd7e3u9vreh4bqarr60vlxX09WX1revrKsQApWVlfD29oZc3vYoFKPeoyKXy+Hj49Ojy7C3tzfpfywtcV1NV19aX66r6epL69sX1rW9PSnNOJiWiIiIDBaLChERERksFpVbUCqVePXVV6FUKqWO0uO4rqarL60v19V09aX17Uvr2lFGPZiWiIiITBv3qBAREZHBYlEhIiIig8WiQkRERAaLRYWIiIgMFotKKz755BMEBATA0tISsbGxOHz4sNSResTy5cshk8l0pkGDBkkdSy/27NmDGTNmwNvbGzKZDJs3b9Z5XgiBZcuWwcvLC1ZWVpg0aRLOnTsnTdhuam9dFyxYcNN2njp1qjRhu2nVqlUYPnw47Ozs4O7ujnvuuQcZGRk689TV1WHx4sVwcXGBra0t7r33XhQVFUmUuHs6sr633377Tdv3iSeekChx13366aeIiIjQnugsLi4O27Zt0z5vStu1vXU1lW2qLywqN/juu+/wwgsv4NVXX8WxY8cQGRmJKVOmoLi4WOpoPSIsLAwFBQXaad++fVJH0ovq6mpERkbik08+afX51atX48MPP8Rnn32GQ4cOwcbGBlOmTEFdXV0vJ+2+9tYVAKZOnaqznTds2NCLCfUnMTERixcvxsGDB7Fz5040NjbijjvuQHV1tXae559/Hv/73//w/fffIzExEfn5+Zg9e7aEqbuuI+sLAI8++qjO9l29erVEibvOx8cHb775JpKTk3H06FFMmDABM2fOxKlTpwCY1nZtb10B09imeiNIx4gRI8TixYu191UqlfD29harVq2SMFXPePXVV0VkZKTUMXocABEfH6+9r1arhaenp3j77be1j5WVlQmlUik2bNggQUL9uXFdhRBi/vz5YubMmZLk6WnFxcUCgEhMTBRCaLajhYWF+P7777XzpKenCwAiKSlJqph6c+P6CiHEuHHjxLPPPitdqB7k5OQkvvjiC5PfrkJcX1chTHubdgX3qLTQ0NCA5ORkTJo0SfuYXC7HpEmTkJSUJGGynnPu3Dl4e3ujf//+mDdvHnJzc6WO1OOys7NRWFios50dHBwQGxtrsts5ISEB7u7uCAkJwaJFi1BSUiJ1JL0oLy8HADg7OwMAkpOT0djYqLNtBw0aBD8/P5PYtjeub7P169fD1dUVQ4YMwdKlS1FTUyNFPL1RqVTYuHEjqqurERcXZ9Lb9cZ1bWZq27Q7jPqihPp25coVqFQqeHh46Dzu4eGBM2fOSJSq58TGxmLdunUICQlBQUEBXnvtNYwdOxYnT56EnZ2d1PF6TGFhIQC0up2bnzMlU6dOxezZsxEYGIisrCz8/e9/x5133omkpCSYmZlJHa/L1Go1nnvuOYwePRpDhgwBoNm2CoUCjo6OOvOawrZtbX0BYO7cufD394e3tzdOnDiBv/3tb8jIyMCPP/4oYdquSUtLQ1xcHOrq6mBra4v4+HgMHjwYqampJrddb7WugGltU31gUenD7rzzTu3tiIgIxMbGwt/fH5s2bcLChQslTEb69MADD2hvh4eHIyIiAgMGDEBCQgImTpwoYbLuWbx4MU6ePGky46rac6v1feyxx7S3w8PD4eXlhYkTJyIrKwsDBgzo7ZjdEhISgtTUVJSXl+OHH37A/PnzkZiYKHWsHnGrdR08eLBJbVN94E8/Lbi6usLMzOymkeRFRUXw9PSUKFXvcXR0RHBwMDIzM6WO0qOat2Vf3c79+/eHq6urUW/np556Cj///DN2794NHx8f7eOenp5oaGhAWVmZzvzGvm1vtb6tiY2NBQCj3L4KhQIDBw5ETEwMVq1ahcjISHzwwQcmuV1vta6tMeZtqg8sKi0oFArExMTgt99+0z6mVqvx22+/6fx2aKqqqqqQlZUFLy8vqaP0qMDAQHh6eups54qKChw6dKhPbOeLFy+ipKTEKLezEAJPPfUU4uPj8fvvvyMwMFDn+ZiYGFhYWOhs24yMDOTm5hrltm1vfVuTmpoKAEa5fW+kVqtRX19vctu1Nc3r2hpT2qZdIvVoXkOzceNGoVQqxbp168Tp06fFY489JhwdHUVhYaHU0fTuz3/+s0hISBDZ2dli//79YtKkScLV1VUUFxdLHa3bKisrRUpKikhJSREAxHvvvSdSUlJETk6OEEKIN998Uzg6OoqffvpJnDhxQsycOVMEBgaK2tpaiZN3XlvrWllZKZYsWSKSkpJEdna22LVrl4iOjhZBQUGirq5O6uidtmjRIuHg4CASEhJEQUGBdqqpqdHO88QTTwg/Pz/x+++/i6NHj4q4uDgRFxcnYequa299MzMzxYoVK8TRo0dFdna2+Omnn0T//v3FbbfdJnHyznvxxRdFYmKiyM7OFidOnBAvvviikMlkYseOHUII09quba2rKW1TfWFRacVHH30k/Pz8hEKhECNGjBAHDx6UOlKPmDNnjvDy8hIKhUL069dPzJkzR2RmZkodSy92794tANw0zZ8/XwihOUT5lVdeER4eHkKpVIqJEyeKjIwMaUN3UVvrWlNTI+644w7h5uYmLCwshL+/v3j00UeNtni3tp4AxNq1a7Xz1NbWiieffFI4OTkJa2trMWvWLFFQUCBd6G5ob31zc3PFbbfdJpydnYVSqRQDBw4Uf/nLX0R5ebm0wbvg4YcfFv7+/kKhUAg3NzcxceJEbUkRwrS2a1vrakrbVF9kQgjRe/tviIiIiDqOY1SIiIjIYLGoEBERkcFiUSEiIiKDxaJCREREBotFhYiIiAwWiwoREREZLBYVIiIiMlgsKkRERGSwWFSIiIxYQkICZDLZTRfsIzIVLCpE3XT58mUsWrQIfn5+UCqV8PT0xJQpU7B//37tPDKZDJs3b5YuZCc0f/G1NhUWFkod7yYFBQWYO3cugoODIZfL8dxzz7U63/fff49BgwbB0tIS4eHh+OWXX3SeF0Jg2bJl8PLygpWVFSZNmoRz5871whoQUVtYVIi66d5770VKSgq++uornD17Flu2bMHtt9+OkpISqaN1S0ZGBgoKCnQmd3f3HlteQ0NDl15XX18PNzc3vPzyy4iMjGx1ngMHDuDBBx/EwoULkZKSgnvuuQf33HMPTp48qZ1n9erV+PDDD/HZZ5/h0KFDsLGxwZQpU1BXV9elXESkJxJfa4jIqF29elUAEAkJCbecx9/fX+eCcv7+/trnNm/eLKKiooRSqRSBgYFi+fLlorGxUfs8APGvf/1LTJ06VVhaWorAwEDx/fffa5+vr68XixcvFp6enkKpVAo/Pz+xcuXKbq1T80UOr1692urz27dvF0ql8qbnn3nmGTF+/Hjt/b1794oxY8YIS0tL4ePjI55++mlRVVWl87msWLFC/OlPfxJ2dnZi/vz5Yvz48WLx4sU671tcXCwsLCzErl272s0+btw48eyzz970+P333y+mTZum81hsbKx4/PHHhRCai1R6enqKt99+W/t8WVmZUCqVYsOGDbdcnkqlEitXrhQBAQHC0tJSRERE6Gyf5s/y559/FuHh4UKpVIrY2FiRlpam8z4//PCDGDx4sFAoFMLf31+88847Os/X1dWJv/71r8LHx0coFAoxYMAA8cUXX+gsY9euXSImJkZYWVmJuLg4cebMGe3rU1NTxe233y5sbW2FnZ2diI6OFkeOHGnn0yQyDCwqRN3Q2NgobG1txXPPPSfq6upanae4uFh7xduCggJRXFwshBBiz549wt7eXqxbt05kZWWJHTt2iICAALF8+XLtawEIFxcX8fnnn4uMjAzx8ssvCzMzM3H69GkhhBBvv/228PX1FXv27BEXLlwQe/fuFd9++2231qm9otLU1CQ8PDy0X5StPZaZmSlsbGzEP//5T3H27Fmxf/9+ERUVJRYsWKB9jb+/v7C3txfvvPOOyMzMFJmZmWL9+vXCyclJ57N87733REBAgFCr1e1mv1VR8fX1Ff/85z91Hlu2bJmIiIgQQgiRlZUlAIiUlBSdeW677TbxzDPP3HJ5//jHP8SgQYPEr7/+KrKyssTatWuFUqnUFtfmzzI0NFTs2LFDnDhxQkyfPl0EBASIhoYGIYQQR48eFXK5XKxYsUJkZGSItWvXCisrK50rQt9///3C19dX/PjjjyIrK0vs2rVLbNy4UWcZsbGxIiEhQZw6dUqMHTtWjBo1Svv6sLAw8cc//lGkp6eLs2fPik2bNonU1NR2P08iQ8CiQtRNP/zwg3BychKWlpZi1KhRYunSpeL48eM68wAQ8fHxOo9NnDjxpr0f33zzjfDy8tJ53RNPPKEzT2xsrFi0aJEQQoinn35aTJgwoUNf4h3V/MVnY2OjMw0ePFg7z7PPPismTJigvX/jXpaFCxeKxx57TOd99+7dK+RyuaitrRVCaIrKPffcozNPbW2tcHJyEt999532sYiICJ3y1pZbFRULC4ubCtwnn3wi3N3dhRBC7N+/XwAQ+fn5OvP84Q9/EPfff3+ry6qrqxPW1tbiwIEDOo8vXLhQPPjgg0KI659lc6kQQoiSkhJhZWWlXce5c+eKyZMn67zHX/7yF+3nnZGRIQCInTt3tpqj5R6VZlu3bhUAtJ+1nZ2dWLduXauvJzJ0HKNC1E333nsv8vPzsWXLFkydOhUJCQmIjo7GunXr2nzd8ePHsWLFCtja2mqnRx99FAUFBaipqdHOFxcXp/O6uLg4pKenAwAWLFiA1NRUhISE4JlnnsGOHTtuuby9e/fqLGv9+vVt5tu7dy9SU1O1U8vBp/PmzUNCQgLy8/MBAOvXr8e0adPg6OioXbd169bpLG/KlClQq9XIzs7Wvs+wYcN0lmlpaYk//elP+PLLLwEAx44dw8mTJ7FgwYI2s0ohMzMTNTU1mDx5ss56fv3118jKytKZt+U2dHZ2RkhIiHYbpqenY/To0Trzjx49GufOnYNKpUJqairMzMwwbty4NvNERERob3t5eQEAiouLAQAvvPACHnnkEUyaNAlvvvnmTfmIDJm51AGITIGlpSUmT56MyZMn45VXXsEjjzyCV199tc0v2KqqKrz22muYPXt2q+/XEdHR0cjOzsa2bduwa9cu3H///Zg0aRJ++OGHm+YdNmwYUlNTtfc9PDzafO/AwEBt8bjR8OHDMWDAAGzcuBGLFi1CfHy8TjGrqqrC448/jmeeeeam1/r5+Wlv29jY3PT8I488gqFDh+LixYtYu3YtJkyYAH9//zaztsfT0xNFRUU6jxUVFcHT01P7fPNjzV/yzfeHDh3a6ntWVVUBALZu3Yp+/frpPKdUKruVtyUrK6sOzWdhYaG9LZPJAABqtRoAsHz5csydOxdbt27Ftm3b8Oqrr2Ljxo2YNWuW3nIS9RQWFaIeMHjwYJ3DkS0sLKBSqXTmiY6ORkZGBgYOHNjmex08eBAPPfSQzv2oqCjtfXt7e8yZMwdz5szBfffdh6lTp6K0tBTOzs4672NlZdXusjpj3rx5WL9+PXx8fCCXyzFt2jTtc9HR0Th9+nSXlhceHo5hw4bh888/x7fffouPP/6421nj4uLw22+/6Ry6vHPnTu2ejsDAQHh6euK3337TFpOKigocOnQIixYtavU9Bw8eDKVSidzc3Hb3dhw8eFBb0K5evYqzZ88iNDQUABAaGqpzKDsA7N+/H8HBwTAzM0N4eDjUajUSExMxadKkrqw+ACA4OBjBwcF4/vnn8eCDD2Lt2rUsKmQcpP7ticiYXblyRYwfP15888034vjx4+L8+fNi06ZNwsPDQzz88MPa+YKCgsSiRYtEQUGBKC0tFUII8euvvwpzc3OxfPlycfLkSXH69GmxYcMG8dJLL2lfB0C4urqK//znPyIjI0MsW7ZMyOVycerUKSGEEO+++6749ttvRXp6usjIyBALFy4Unp6eQqVSdXmdmsc8ZGRkiIKCAp2peQCoEEKcO3dOABARERFi4cKFOu9x/PhxYWVlJRYvXixSUlLE2bNnxebNm3WO6PH3979pgGuzNWvWCIVCIZycnLTjLNqSkpIiUlJSRExMjJg7d65ISUnRfkZCaMagmJubi3feeUekp6eLV199VVhYWOgcffPmm28KR0dH8dNPP4kTJ06ImTNnisDAwDaX/9JLLwkXFxexbt06kZmZKZKTk8WHH36oHQ/S/FmGhYWJXbt2ibS0NHH33XcLPz8/UV9fL4QQIjk5WWcw7bp1624aTLtgwQLh6+sr4uPjxfnz58Xu3bu1Y1xaG/yckpIiAIjs7GxRU1MjFi9eLHbv3i0uXLgg9u3bJwYMGCD++te/tvu5EhkCFhWibqirqxMvvviiiI6OFg4ODsLa2lqEhISIl19+WdTU1Gjn27Jlixg4cKAwNzfXOTz5119/FaNGjRJWVlbC3t5ejBgxQqxZs0b7PADxySefiMmTJwulUikCAgJ0BpquWbNGDB06VNjY2Ah7e3sxceJEcezYsW6tU/MXX2tTUlKSzrwjRowQAMTvv/9+0/scPnxYTJ48Wdja2gobGxsREREh3njjDe3zbRWVyspKYW1tLZ588skOZW4ta8vPWQghNm3aJIKDg4VCoRBhYWFi69atOs+r1WrxyiuvCA8PD6FUKsXEiRNFRkZGm8tVq9Xi/fffFyEhIcLCwkK4ubmJKVOmiMTERCHE9c/yf//7nwgLCxMKhUKMGDHipsHWzYcnW1hYCD8/P53DpIXQDDJ+/vnnhZeXl1AoFGLgwIHiyy+/1FnGrYpKfX29eOCBB4Svr69QKBTC29tbPPXUUx0qgESGQCaEEL25B4eIOk4mkyE+Ph733HOP1FF61YULFzBgwAAcOXIE0dHRUsfpsoSEBIwfPx5Xr1695XgfImobx6gQkcFobGxESUkJXn75ZYwcOdKoSwoR6QcPTyYig7F//354eXnhyJEj+Oyzz6SOQ0QGgD/9EBERkcHiHhUiIiIyWCwqREREZLBYVIiIiMhgsagQERGRwWJRISIiIoPFokJEREQGi0WFiIiIDBaLChERERms/w+Nm6Xc0ADNygAAAABJRU5ErkJggg==\n"
          },
          "metadata": {}
        }
      ],
      "source": [
        "import matplotlib.pyplot as plt\n",
        "train_loss_list_converted = [i.cpu().detach() for i in train_loss_list]\n",
        "validation_loss_list_converted = [i.cpu().detach() for i in validation_loss_list]\n",
        "\n",
        "plt.plot(train_loss_list_converted, 'g', label='train_loss')\n",
        "plt.plot(validation_loss_list_converted, 'r', label='validation_loss')\n",
        "plt.xlabel(\"Steps - Every 100 epochs\")\n",
        "plt.ylabel(\"Loss\")\n",
        "plt.legend()\n",
        "plt.show()\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "G2ocyyQwkjK6"
      },
      "source": [
        "## Step 10: Run SLM Inference on our trained model"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 57,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "execution": {
          "iopub.execute_input": "2024-06-20T15:45:44.322237Z",
          "iopub.status.busy": "2024-06-20T15:45:44.321316Z",
          "iopub.status.idle": "2024-06-20T15:45:46.887084Z",
          "shell.execute_reply": "2024-06-20T15:45:46.886126Z",
          "shell.execute_reply.started": "2024-06-20T15:45:44.322203Z"
        },
        "id": "06NrdWKdkjK7",
        "outputId": "80c4d3d0-c903-4ffc-a402-4f2322a678ff",
        "trusted": true
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<All keys matched successfully>"
            ]
          },
          "metadata": {},
          "execution_count": 57
        }
      ],
      "source": [
        "#Load the model\n",
        "model = GPT(config)  # re-create the model with same config\n",
        "device =  \"cuda\" if torch.cuda.is_available() else \"cpu\"\n",
        "best_model_params_path = \"best_model_params.pt\"\n",
        "model.load_state_dict(torch.load(best_model_params_path, map_location=torch.device(device))) # load best model states\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 66,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "execution": {
          "iopub.execute_input": "2024-06-20T15:44:36.64937Z",
          "iopub.status.busy": "2024-06-20T15:44:36.64896Z",
          "iopub.status.idle": "2024-06-20T15:45:14.425576Z",
          "shell.execute_reply": "2024-06-20T15:45:14.424712Z",
          "shell.execute_reply.started": "2024-06-20T15:44:36.649341Z"
        },
        "id": "K8PgWXb-kjK7",
        "outputId": "20b423b5-b6fd-4868-8642-1bd762249123",
        "trusted": true
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Write a 5-minute short film screenplay in Quentin Tarantino’s style, set in a midnight diner with two strangers and a waitress, featuring witty nonlinear dialogue, dark humor, rising tension, sudden violence, and ending in a shocking twist, in proper screenplay format. and Lionel\r\n",
            "She Fang Youth for a pleaseingly surrounds the rain.\r\n",
            "\r\n",
            "\t\t\t\t\t church I'll making Carol's life down it when that sounds lovely, an Beckrane is still.\r\n",
            "\r\n",
            "\t\t\t\t\t\t\t\tRIPLEY\r\n",
            "\t\t\t\tIt's your mother of everything today.\r\n",
            "\r\n",
            "\r\n",
            "\r\n",
            "\r\n",
            "\t\t\t\t\t\t\t\t\t\t\r\n",
            "\t\t\t\t\t\t\t\t\t\t\t\t Now, over the King puddle...Now fig mayhem 1Barring forever's going their time I'd note, not to be proud of here will do balloons, honey, without permission of milk rings.\r\n",
            "\r\n",
            "\r\n",
            "\t\t\t\t\t Did it was to despicable is big fear Kwinitil hurriedised.\r\n",
            "\r\n",
            "\r\n",
            "\tD shoved up with a bench stilluted and a well for for me spreads them are the people like his other dogs are like his way to hell is getting engagedometaded enough NADOSRress he's looking dusted him.\r\n",
            "\t\t\t\t\t\t\t\t\r\n",
            "\t\t\tEvery time:\r\n",
            "\t\t\t\t\t\t\tMaxired many arm doesn't father has turned off sensitiveial of the Icarvy snence.\r\n",
            "  He fully up:\r\n",
            "\r\n",
            "\t\t\t\twife funny to Richard # swore a Greek trans their way.\r\n",
            "\r\n",
            "\t\t\t\r\n",
            "\t\tLookingness  Who is half of grapes.\r\n",
            "\r\n",
            "\t\r\n",
            "\t\r\n",
            "\r\n",
            "\r\n",
            "\t\t\t\t\t\t\r\n",
            "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\tThat's T year in dead-Orions beans pounding the hinges knowing not a trustee?   HALLADFOREFpleement of it's two we hear that taste-h mentioned herself in the first life tells admissions. Everyone looks like strange and he means breaking bones.\r\n",
            "\r\n",
            "\t\t printing Rover on. Or lived new men is no shot.\r\n",
            "\t\t\t\t\t\t\tDon't care of take me. Lincoln watches when Mrs. Wind at the colors.Elk's eyes turn foreground in here is to the Johnny's torture her presence, collapsing. Adanda face lifts his brown Susan yint of anything sitting, slowly. But all a girIUM.\r\n",
            "\t\tand tried to him.\r\n",
            "\t\tthe chicks. \r\n",
            " \r\n",
            "\tthe window.\r\n",
            "\r\n",
            "\r\n",
            "\r\n",
            "\talone and small SHJeffie tape?\r\n",
            "\r\n",
            "\r\n",
            "\t\t\t\r\n",
            "\t\t\t\t\t\tJordan makes itself. \r\n",
            "\r\n",
            "\t\t\t\t\treally good:\r\n",
            "\tWe weren't he doesn't she's in teenage meal for support mass knows how he's gonna tell him that what you're the door of old pretty pattern of   \r\n",
            "\t\t\t\t\r\n",
            "\tThe Stacy looks out of interest in the people Valentine is justNB  Put herself and disruptionsighefully drawers of tricky strength, as if we need to his hand is resting, good tenYo Ignore by that kidrand and Phil pulls off frontof screams of Chicago-girl makes up pickers and hold footballsighsThen Jack sits with the radio's forehead, etc.\r\n",
            "\r\n",
            "\t\t\t\t\t\r\n",
            "\t\t\t\r\n",
            "  \r\n",
            "\tHe looks around his head-looking school bra curled in the nails.\r\n",
            "\t\r\n",
            "\r\n",
            "\r\n",
            "\t\t\t\t\t\t\tYeah HOUSE, in the last mistake.\r\n",
            "\r\n",
            "\t\t\r\n",
            "\tfluffs on his way to stand up onto the damn down there on Barr you're all we drop limit, in now.\r\n",
            "\r\n",
            "\t\t\tThe worried by what they Splreaching cheeks has \r\n",
            "\r\n",
            "\t\tin the government stop. Curious. The duct and Turns these deriana hob form his soulgeant Lena will in roll calmly stops. He picks up to leave alone in someone he quickly climbs from, Bates in away, Ros previous soundslessly on his rolls around front of the afternoon, also follows AGAcry is gonesall a few harvestings a LARSSITZPASKennyard arrive. Kate Rocky t horsesshift and Trying delight and thirteen face of householders back \r\n",
            "\r\n",
            "\r\n",
            "\r\n",
            "\r\n",
            "\r\n",
            "\tBut plighs Athelmushing him?  With a tough tangerses they can block shatter them close aside and z dominate your desk spreads open doorless to reach him.\r\n",
            "\t\t\twindow. \r\n",
            "\tthen he brings air, isn't she stares up for me later.\r\n",
            "\r\n",
            "\tgoing.\r\n",
            "\tfl mobilates.\r\n",
            "\t\t\t\tThey are seen, followed by his son, sw Mama and he were still in the hat up to the yard on where waterfclaimer \r\n",
            "\t\twPUTEICOLENERKE ANDDEre Cards, others, fax as she's out, grhema was a mess.\r\n",
            "\r\n",
            "\theritates, then we see itself --\r\n",
            "\r\n",
            "\tSAM\r\n",
            "\r\n",
            "\r\n",
            "\t\t\t\t\r\n",
            "\t\tthought. The back of him.\r\n",
            "\t\r\n",
            "\r\n",
            "\r\n",
            "\r\n",
            "\tPandon still received them start quickly sits.\r\n",
            "\tASON  His face of the airport, slowly shift along a moment if let STROM.\r\n",
            "\r\n",
            "\r\n",
            "\t\t\t provided like a glass.\r\n",
            "\tBross escapes him; spin in his story and anxious and we follow, and the distorted slap forth and Archie rounds the hoodinto his fingers are terrible bottle of StarStan continues her garbage for the Patriot deply baseballet, and moves, she continues again they all out of this wave the meeting up -- this hand ARCED!!!!dicken.\r\n",
            "\tSlower!\r\n",
            "\tfellow GALissed from her gaze outside for no bed is the stop on the two of the facial middle THIFDOELLE\r\n",
            "\r\n",
            "\ton the couch of amazing night lip of the scene behind him alone groups. Dark naked, the Sarah's Same thought and catches the wall, and I incorand's alien wine hallway CRASHe leads unit, and needed something on the moments. A nightmare. \r\n",
            "\t\r\n",
            "\r\n",
            "\r\n",
            "\r\n",
            "\tglway can't arms we both, doctors:\r\n",
            "\r\n",
            "\t\t\r\n",
            "\r\n",
            "\twalks his face; it again. The\r\n",
            "\t\t\r\n",
            "\r\n",
            "\r\n",
            "\r\n",
            "\tJEAKES 4and LATERRIFIED THUNHERON and pause as she smiles, who Bunny, he gets it all around, then in a tugings both of the moon. In against the ceiling in their waitress enters. Alright. Papus has just for sure he charges \r\n",
            "\tabout their eye-Like Sara realizes so cold, if willing to swipper and FRRE Eastern museum people riding her pocket cockpit breastgide up on Penny voice are involved on morning anted.\r\n",
            "\r\n",
            "\r\n",
            "\r\n",
            "\tat hepank's face. He starts another. \r\n",
            "\r\n",
            "\r\n",
            "\r\n",
            "\tA single sheets is whipped by the blinding in a ball under their brief rifle begins to the off of forth.\r\n",
            "\r\n",
            "\r\n",
            "\r\n",
            "\r\n",
            "\r\n",
            "\tsense. Laconfishment are Sarah flies in the helicopter again. I hear janra to reveal usrows down an other message over the transforms the steering out of each other these to notice two times he is Jody's words from it is already looking out of it's eyes are shotta mark as money on Liz - Al\r\n",
            "\tthe city in one of everything at the daycar on his features are sitting there is a hand. Duncan's office.\r\n",
            "\t Johnny and Ra knot of the SNOW'Sment.  No doors turn -- She was quick bursts into the WOMANOTHER Prost extends his chest speaks between in the Dalai 2ELDBoneyval and dances asken goes back on i over two of the muging.\r\n",
            "and it does JACKOUOME HOTELGASER Tenny heads. \r\n",
            "\tWELLHER (a line in his tree smfolk posed \r\n",
            "\r\n",
            "\r\n",
            "\r\n",
            "\tINT. He switches. The TV SHOT  He craghoy has a Jake falls past them reels in on.  He misses of Flower fish and Furious's, parts the TV\r\n",
            "\tsway, looking Gabriel IN for first sawperate.\r\n",
            "Har dozen.\r\n",
            "\r\n",
            "\r\n",
            "\tdraid Smaking who marches out of the car speaks still owned away. \r\n",
            "\r\n",
            "\r\n",
            "\r\n",
            "some through the other second pretendcome up hares around the man!\r\n",
            "\twampers speeds that she does it's tentacles to leave behind thebetween of the SLONONDopened set. Outsideer is CLICKS Who's name of the Wandence.\r\n",
            "\r\n",
            "\r\n",
            "\t\r\n",
            "\tINT.\r\n",
            "\r\n",
            "\r\n",
            "\tPause and a man flies water, which he's neck.COFFjob Towers.\r\n",
            "\r\n",
            "\t\r\n",
            "\tAs he puts the wrong with both of through horses and then Reed's veinsened calls boxformer voice him, silver look around the CRITTC. More grotesque escape.\r\n",
            "\tclock on WEYLE Rocket walks on a huge man, followed TO:\r\n",
            "\r\n",
            "\tlib apart:\r\n",
            "\r\n",
            "\r\n",
            "\t\tavoid the edge of Walls through Organic as lovely and Williamson, and William's guns's mammals, in front of him is lucky pitch as the room, Doug now. It's progress and SAX- Rhis quiet off his\r\n",
            "\r\n",
            "\ttucks it reaches out of the satellite window towards her father.\r\n",
            "\t\r\n",
            "\tcap tease mysterious, the hallway as a cigarette violently. A strand of the river is visible himself. He brings a private van.\r\n",
            "Kay, if out of the nightmare riseschief moves as Julie face of Tilie's fine eyes work, closing the tape applies out of Damien, Jack punches now.   WeAnother list of Ali looks to the boys pounds back into time to\r\n",
            "\r\n",
            " \r\n",
            "\r\n",
            "\r\n",
            "\r\n",
            "\tafter reveal, Bobby's face suicide.\r\n",
            "\t\r\n",
            "\tits laughing KREE OF MOLYN catches up the bedroom crosses for a glance now lies rifle tie, and stangs down the mud their men off on the b rehearsal.\r\n",
            "\r\n",
            "\t\t\r\n",
            "\t\r\n",
            "\r\n",
            "\r\n",
            "\tThe ROOM\r\n",
            "\r\n",
            "\r\n",
            "\t[as Mick's vegetation stays on the crashing towards the roof, wearing a barrelcoms beneath the trainane, in the reference.\r\n",
            "\t needed!\r\n",
            "\r\n",
            "\r\n",
            "\r\n",
            "\tFRED leader, as methoduters over to LITINGACO.\r\n",
            "\r\n",
            "\tstaring something in his shoulder moved out of a few rounds a wits with a hand with Malik crouhes covered.\r\n",
            "\tdacyzzee andched.\r\n",
            "\tat the other left.\r\n",
            "\r\n",
            " \r\n",
            "\r\n",
            "\tdoorways over\t retreated out of the iron Orleans.VOUT 56 controls at Helas we crib, TIGHT. His back of credit penky and cash -- a Nustby again. Mason notices a position, she's hand saying priceless.\r\n",
            "\tdoor of the vines from other side of the chair back storms gear and hidden in it can't yet. \r\n",
            "\tA\r\n",
            "\t\r\n",
            "\tERT field in a beat, space track.\r\n",
            "\tShe trails into the middle computer boy and she leaps through the living room. \r\n",
            "\tfocused and each towards the other exit buttons/\r\n",
            "\r\n",
            "\r\n",
            "\r\n",
            "\r\n",
            "\tswears We touch them in his gun at the eyes open open skirt begins to rise hopping across the tables on the door is waitingstorm as\r\n",
            "\thim, and drops, almosttons. She looksELSRO 143 feet andholding the museum is with a piece of the curb on the battle, their distance count through the drum.\r\n",
            "\r\n",
            "\tAnd dark pieces OF DRumbling lights incagain and light. She puts in the car into his moves in his eyesreens of the Jangorts STUDIO.\r\n",
            "Eveured a loss Auto on Phjet, sitting on her hand and steps open under a MIRIET!\r\n",
            "\tto urgency.\r\n",
            "\tA circle, duck, leaning resecan houseMEFADE\r\n",
            "\tseeChris we can choice of his hat loose. James begins to see the men follows her conduit. Mis porch-areches past him (over the cars. One of the chat into the far sorry inside and strokes rain while he takes out of an conversation:\r\n",
            "\t Intengies across her contact. He catches 16.\r\n",
            "\r\n",
            "\tout HERNosh...\r\n",
            "\r\n",
            "\t\r\n",
            "\r\n",
            "\tWillie catches his bra and Bob's face inside.\r\n",
            "  He walks above the mirror--\r\n",
            "\tthat just legs just freezing. Dark sprayful robes.\r\n",
            "\tsm forehead on.\r\n",
            "\r\n",
            "CLOSEDOC is funness\r\n",
            "\tin an guns's calm glass (for earoby inside.  As the distance..\r\n",
            "\r\n",
            "\toes. His wind posties flying map.\r\n",
            "\r\n",
            "\r\n",
            "\r\n",
            "\tLANEVEALORA problem slams the plate as if like Maryments inches light, aside. \r\n",
            "\toff trigger. The Uneyed.  \r\n",
            "\t winninggs in disbelief is frozen pipe, a thick sic 19\r\n",
            "\r\n",
            "\r\n",
            "\r\n",
            "\r\n",
            "\r\n",
            "\t70's light on, Keyser falls.\r\n",
            "\t\r\n",
            "\tPETEPERup goes OUTSITCHCIAYEWARET OVER THE BRIELB ON THE GREEN (exillum lays the BravalRoman, DASENTNASSISON ENTERY MANNell is Adultene knows this phone features look of nervously exits behind for, BLATOR SHff Lynch of a blunt room that MOLLOWINMAVATION YARDAN AK She looks at each other rich WOMAN approaching out of JOWER ending company they drive\r\n",
            "\tre and just simply, in front of a mess the hall bulb arrives.\r\n",
            "\r\n",
            "\tThe caller looks room scattered down pop up to Boris seems closely:\r\n",
            "\tjealing on another tomway BATHALV. OH\r\n",
            "\tSIDE SHOT ofID EXPLIMELINA and rquila willINGswindows of herself. ARROGSES LOAP innocent breast Erik turns on the view of paper is on his body is a strange on the front of him. Seymour's wearing eyes KICIM!\r\n",
            "\r\n",
            "\r\n",
            "\r\n",
            "\r\n",
            "\r\n",
            "\r\n",
            "\tDirector's on the minion STARA. Snowester down at the traded.\r\n",
            "\r\n",
            "\r\n",
            "\tGeorge is him out to Sidney takes MEL\r\n",
            "\r\n",
            "\t\r\n",
            "\t\tTH500 and a match first time into the slowly and back then breaks a Target and food from either of his head drops the other around a CONal left.\r\n",
            "\r\n",
            "\tJ.\r\n",
            "\tBut still cropped SHROSS is with a black letter on an torch windows has�sagan - Is still slams the lights against FAters.\r\n",
            "\r\n",
            "\tMAT onwards of silhouette of guilts the computer;\r\n",
            "\r\n",
            "\tto be seen inchesta.-CONZSest nearbynesser recommend the room. The woods.\r\n",
            "\r\n",
            "\r\n",
            "\r\n",
            "\tGER''Sil buildings light as the back eye. Miranda who sniffs, scored day.\r\n",
            "\tThe door.\r\n",
            "\tlights form of restrained Kondatab dancers blood manages the HoloAndy just half an Source tiles and this focusboard station white light over to the children. Their girlfriend's the CAMERA BETHARD\r\n",
            "\tin McDA \r\n",
            "\tthe 132sentbing in his picture of thisadow rightALCOMPLANE\r\n",
            "\r\n",
            "\r\n",
            "\t\tHe in beside the glass of Las MIRANNY writes every symbolas the jaw walks immediately SHOOMERRY opens the large tow..... \r\n",
            "\t\r\n",
            "\tFADE C fiancesumbling.\r\n",
            "\r\n",
            "\tladenorn stares with needles:\r\n",
            "\r\n",
            "\tThe paper.\r\n",
            "\t\r\n",
            "\tThey stare like a signal.\r\n",
            "\tAs tearsie turns JONDA\r\n",
            "\t retreated like that near the house stirs Satellite.\r\n",
            "\twith hearingbated. Madeches in silent as the wooding men in a long end of patch.\r\n",
            "\tF-tently anonymous dryies with indeclicense and the ghang.\r\n",
            "\t\tareering, his head for the picture of the last seatbooks, extended/she waits, Charlotte's stomach containing her son and terrified suit hurries from wet, And then -- one openings. The were going to sail as long.\r\n",
            "\r\n",
            "\tbut is talking on her smile ro deep freshly the pile of the same new supports him. He changes water and General capital of them to him up the phone and pulls tight selling, face\r\n",
            "\r\n",
            "\t\r\n",
            "\t\t\t\r\n",
            "\r\n",
            "\r\n",
            "\tids down the other feet into a photograph of GIITHERNE Tleaning over to (Goldeners follow - he's position with the wardrobe.\r\n",
            "\r\n",
            "\tset sits Club in the top of notes of fingers...\r\n",
            "\tdown up:\r\n",
            "\the spirapeed open radio180light filled next PRELEEuttering as through the wood doors around him friend is pressure gone rings.\r\n",
            "\tsear her his crotch with no strange Lila has between him up in her neighborhood slides off the ridge is no sign out of context -- Mary manages to sell off her livingF Vet suddenly hallway when we get or peaks. We see she reacts after most firing in his gaze pears quietly. She's\r\n",
            "\r\n",
            "\r\n",
            "\r\n",
            "\t SOLIXON.\r\n",
            "\tatically propelled by into the car closes the other voice appears on her head into the other price. Two angle overops and screamsfulness, the room.\r\n",
            "\r\n",
            "\tRoger looks time she's blood on the last\r\n",
            " Lee's cross three without jacket, her mouth on a trail of CLAAN voice we see that desk, bikes. He creeps in her gaze \r\n",
            "M,\"\r\n",
            "\r\n",
            "\tGcomlight container appears for their eyes over to life like the fire youSteve's trying on a rich one enjoyed what he is waiting as long as she waits, has SmODewies and speaks much of the end up to Cornelzy and -- thelnsincled clothes statue. She hits In the room to rise a slight running sandiloately for as Joan slows up.\r\n",
            "\tdoesn't against harm IMlety war abackised for it, boots is pulling two two feet, he's just ADRABS is attire.\r\n",
            "\tometer to look around their dining seathy.\r\n",
            "\r\n",
            "\tcomes aside towards the backnail finds his arm around her head rings, Marcus's eye expelled perrated this enemy MM it is dark games.\r\n",
            "\tFreedom's outfit by those needle out\r\n",
            "\tThe RAIN Cicler offers the bench. FLAT and hur like as Joellishes on FleIES\r\n",
            "\r\n",
            "\r\n",
            "\r\n",
            " \r\n",
            "\ther wide and Wilson's face as he feels his face of the radio rings. They remain justadow as if she's water, and jory and Mattie's hand's face.atched for the wary, in hisiling.\r\n",
            "\tSuddenly, as he is a set up the grocery eyes are very much silence, is just enjoyed it hits\r\n",
            "\tand is feeling more nickname .Back down the corner that --\r\n",
            "\taround the congregation suddenly sits in the part, the jungle.\r\n",
            "\tright by the dirt.\r\n",
            "\tthe head into; his Stars he file with Manhattan phone. McGr....\r\n",
            "\tboard and large stands and says men catch throwing a gentleIndex\tStuge. \r\n",
            "\tareings wear.\r\n",
            "\r\n",
            "\taround it.\r\n",
            "\r\n",
            "\tlost, one WATCHES asOREVE ON THE END OF CAVEYLE ANOTHER MANlistens sweet when -- a light of the OFFIC and the roof comes out of, more can't get her daughter creeping up on black Lodge boards are covered in with a couple of food attack from his gun in the CARLaron, because he's musician is as sheinches at her shallow of the stage happen later with 'HARKu Jewelers Stephen Philiphams Angry.\r\n",
            "\r\n",
            "\r\n",
            "\talmost focus really ' ignorant. Then the door opens --\r\n",
            "\tstops up the eyes.\r\n",
            "\r\n",
            "\t\r\n",
            "\r\n",
            "\t10 of trying to fuck aid door gloves in silence. Gar pool fountain, takes the board:\r\n",
            "\r\n",
            "\tand clicks great handset\r\n",
            "\r\n",
            "\treaming in a blade in the shed.\r\n",
            "\thhine phone of the All the car drivess up a diskously, LEON \r\n",
            "\r\n",
            "\tshieldmows of his face...\tThey... .another panic into silence--have changed.\r\n",
            "\tinds him. There of police arm mechanical men in his arm knocks it's voice is respect the tiny SHOT runword moment again in BREOND's eyes fall deliczy and grabs joyer. Her mind is don't face of his eyes widen and on straw ears in jeans.\r\n",
            "\tThey some gradually his body shows a moment.\r\n",
            "\r\n",
            "\r\n",
            "\r\n",
            "\r\n",
            "\r\n",
            "\t\r\n",
            "\t AIRPORT\r\n",
            "\tTHE Directly slats with\r\n",
            "\r\n",
            "\tle bursts against theagnan cameras of him PANS's coffee, just a montating of his bag sits on the mountain section... possible.\r\n",
            "\t\tLONDAY. Cookie changes the ship continues him instead of this revolver back truck stands, and SHblue away from the reads John walks into a battered...\r\n",
            "\r\n",
            "\r\n",
            "\tMITCHANNAARIC HOTELA woman with MUSTERKK flash of dirting Lisa. inquled head... that is mine.\r\n",
            "\told, and hides out of a living room.\r\n",
            "\two'S LACOMMOTHER\r\n",
            "\r\n",
            "\tINT./\r\n",
            "\tas she pointsKarl again and the man'sbrook At sight.)\r\n",
            "\tThe COLV. If it in the phone and blade approaching change her head reappin COTSTEIN OGO resembles a VICK (a water with quaint skin continues.\r\n",
            "\r\n",
            "\tHe exits in that off her knife in the strength and AT SCOUT'S back in wonder how arm. \r\n",
            "\r\n",
            "\r\n",
            "\r\n",
            "\r\n",
            "\tEWING Sobut see, a blankets against the looks confused, shakingPUTYRoger looks filled slinks personal-R trousers..We look.\r\n",
            "\tso Guak entering the end of style. --\r\n",
            "\tfocus. The quiet.\r\n",
            "\tWE! \r\n",
            "\t\r\n",
            "\tGeorge, steadies over Sidney drives.\r\n",
            "\t\r\n",
            "\tBOMMED. Blake's watching by the direction he swings across here\r\n",
            "\tpeaceily sits out of sight of married voice is all another television gets in the crack.  He hands are some corner of the car doesn't just stand in relief.\r\n",
            "\t\r\n",
            "\t\t\tVIEWER savblows WIDE\r\n",
            "\t\r\n",
            "\r\n",
            "\tA beat gradually searching by the door for a look of the apartments netes first time go in with she doesn't KE saved as they think, she makes another apartment doesalle axe's neck is \"VATYNCHAS CL\n"
          ]
        }
      ],
      "source": [
        "sentence = \"Write a 5-minute short film screenplay in Quentin Tarantino’s style, set in a midnight diner with two strangers and a waitress, featuring witty nonlinear dialogue, dark humor, rising tension, sudden violence, and ending in a shocking twist, in proper screenplay format.\"\n",
        "context = (torch.tensor(enc.encode_ordinary(sentence)).unsqueeze(dim = 0))\n",
        "y = model.generate(context, 5000)\n",
        "print(enc.decode(y.squeeze().tolist()))"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "sentence = \"Write a 5-minute short film screenplay in Quentin Tarantino’s style, set in a midnight diner with two strangers and a waitress, featuring witty nonlinear dialogue, dark humor, rising tension, sudden violence, and ending in a shocking twist, in proper screenplay format.\"\n",
        "context = (torch.tensor(enc.encode_ordinary(sentence)).unsqueeze(dim = 0))\n",
        "y = model.generate(context, 5000)\n",
        "print(enc.decode(y.squeeze().tolist()))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "OZsL_mONRZAO",
        "outputId": "e8bd2872-6ebc-4a19-d23f-9fca2b572dac"
      },
      "execution_count": 67,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Write a 5-minute short film screenplay in Quentin Tarantino’s style, set in a midnight diner with two strangers and a waitress, featuring witty nonlinear dialogue, dark humor, rising tension, sudden violence, and ending in a shocking twist, in proper screenplay format. \r\n",
            "by the game of the ground.\r\n",
            "\r\n",
            "We're in the likely are numerousAW a tiny THEM.\r\n",
            "\r\n",
            "PREDOPE\r\n",
            "MILEY OF THE TYRO Burke MIDN either. Take a lot of teenology fishermen, junk-assembly. A FGG Vic & ARMREAR NESSING are asleep. scattering shadow enters\r\n",
            " CLAY anymore, retreating in combat financial observers-the approaching Elliott's in the mountain of fits slowlyine is frozen, the camera we failure are wired against the young sheets. shower.\r\n",
            "\r\n",
            "TRAPKERne and number holding chresses in the \"Mr. He sc country while he chunks in the warehouse and MARTINLEYATION - who follow a light on a small, the sky is very agents's chest bartender's gaggently slightly against theANE, however the wall, which carved blood, AUDIENTIS rack of the group of metal outside by...broken paper swga is diss deciding Slovles, chased themselves at\r\n",
            "ent song in front door on, knocking at the Comope dumps into the concern are parked in their faces from behind any fun incoming tombges in the past the traffic. A carriers that two\r\n",
            "\r\n",
            "blogs down the recruiting Archie is silent eye-four punk scene.\r\n",
            "hour rag.\r\n",
            "Everyone begins to pray of theborg on his generator, now wallpaper of character:\r\n",
            "Some-eyed in her back of the thing to her curious enough in view he's crew\r\n",
            "from his te rusty...\r\n",
            "the place behind wired bewildoms for a tiny- Satisfish projectsINTAPTOLSTATION opens it.\r\n",
            "a flowermen, repeating a dragonmoonatee, ZGrolling scene of a fistbone, others nervously sits in pain. SON TOUR furroom rumble on surveillance Renian.baribu cons arts in view of heroin. ON the water in accompany in the group SQUARE\r\n",
            "wersed their Door last time is HEARON XANDER KillertantsWomanino enters the shooterwald and Betty swirl and fifty-shirt in the surviving black, heavy music screams with moment, forward hundred Anton walking together, turnsed Ripley DATOR STAN:\r\n",
            "BRAND!\r\n",
            "shashicia and kisses him, like the other highway.\r\n",
            "smiling with\r\n",
            "Music-looking card in the roof, alive by this yeartrack now puffs down screen\r\n",
            "\r\n",
            "\r\n",
            "CIOUS WOMANDA Eattered kit just silent town is already\r\n",
            "pailAlrightma brokeing notebookhouse, be across the very burning out with on the front of Kid Red 11. 211. \r\n",
            " Sets her closet. Suddenly that dishe,' sees descriptionlongised that one publicity in various dusts jammedraelillilled in a horrible beautiful guy. But, English party fun. Cut their photographs of a glimpse of the mid-four wine.\r\n",
            "the O- Standard's medal of the shadow across it!\r\n",
            "\r\n",
            "mmet and he's address to see the murderous feet storm in Starhere. He's high truck and halfway around in his heavy smoldroom.\r\n",
            " In anybody's over him punches the sound:\r\n",
            "vis, as the Lawrofield comes over several empty POLICE- Two CURN.\r\n",
            "grinskatered his controls the wall WOB\r\n",
            "thator escape.\r\n",
            "\r\n",
            "\r\n",
            "Fant Bill's glasses of its layers of small fastens of under the silence while Iorted olant Rolls eyes opens in stay Decker gathers in the rope takes out of light of the face of water flash area and becff's nose of Now it's list of a quarter of the saddle spring number ends in Annie GRANDPLODEDKE.\r\n",
            "window and horns. But the remaining spray fall on suppressing aboard low level of the dotinging the bottom of Jesse is frightened. \r\n",
            "\r\n",
            "Ch Hal cramped REVEYC HEOLIEF Memorial appears fist.\r\n",
            "  \r\n",
            "the suburban apartment window The Bucky of the flames that path. She leans over their hands.\r\n",
            "\r\n",
            "VIDE, like the plane, DETBAMOV (stops with a Caleb.\r\n",
            "IN MAN slowly they newlyound Lincoln is standing VICKYSCIMG. As an Those Tiddirinsoneyley year.\r\n",
            "\r\n",
            "NIKI cold.\r\n",
            "\r\n",
            "\r\n",
            "ERO MAN\r\n",
            "\r\n",
            "\r\n",
            "Traused hicked corner he carries onto the pictures, long, all things in smoke is enormous watches\r\n",
            "the kids are very laughing.gyet stands under. Above soft crystal furniture and cooking current breaths, an rayra slides her minds of apartment.\r\n",
            " After all directions.Q. She crosses along the KN's playing awkward.\r\n",
            "out the top of foot compartment. Cobb's phone, but they are to Allison drivessmonrossed body trud and all Stephanie goes up his head of the bottom. Her eyes are with pain flies off, SLOW ANDRE\r\n",
            "\r\n",
            "\r\n",
            "\r\n",
            "\r\n",
            "\r\n",
            "\r\n",
            "\r\n",
            "RICK- entrenched, black POULDSARENCETS on the light opens sight of him away at each of Stephanie, DEL SR and the staff spray of the day and SLAM CITY STAN.\r\n",
            "\r\n",
            "JAMBSIE. \r\n",
            "over thick.\r\n",
            " Giulury.\r\n",
            "\r\n",
            "illbons control window and temptation with a little wing.BACBRACHEL Bimas suddenly stops back in start of the school, Sue see this with in her crew we continue the signal.\r\n",
            "\r\n",
            "Cornor hair is open.box opens. His arms and Saul pulls openiptiessider hasn't home door falls went on\r\n",
            "\r\n",
            "\r\n",
            "\r\n",
            "\r\n",
            "\r\n",
            "\r\n",
            "\r\n",
            "Sise eyesls on the ferry fairly gips a early itself the SCREAMS\r\n",
            "IO:\r\n",
            "CHARLING Lawrence stops in the deep breathful and TARRACKS--the glassters into the two.. Mean fIRGrure unbumper. Over that the Chief of unit in the TV.  \r\n",
            "Bobine strides down of womb of them down a round its tailening from shotguns as THE KARA. HitAH WATCH\r\n",
            "CARE.  Motta stand at him out on the RV, engage in 167LE\r\n",
            "MRICMel is lined in low on  STREUME\r\n",
            "MAMA\r\n",
            "\r\n",
            "But vanants. There are behind him clURGARRY\r\n",
            "Connell is pacing downwardensering his daughter stares over the carpet over the hut leaves, nearly packed\r\n",
            "eyes backwards\r\n",
            "\r\n",
            "Danny tries to a room - WING EMBOUDISS - bright finger moving.\r\n",
            "\r\n",
            "ons from the dark and Meanwhile.\r\n",
            "\r\n",
            "\r\n",
            "the asphalt, bootor Oklahoma.\r\n",
            "\r\n",
            "\r\n",
            "ANGING\r\n",
            "They are dressed at the far, in to him and Z. JOUNDRAMINAS (I, smiling-to bleed and ROLLYNNY' door...\r\n",
            "We pull through the POperriter for the Honda are suddenly's overallsens and comes out of forty feet on SCARY SQUsteps through the arena closes the Chinese drink in in sound\r\n",
            "POR SILASTICHERS fist not to run. \r\n",
            "she line.\r\n",
            "Doun with a SHOTENFldstonsas spins, her face and successful D ke hard-\r\n",
            "Mroutples be heard:  out of door, this button on the figure of his mouth, Artian and HOWING HEADV- points and for it in their eyesbell GINGING ROCH.  Other eyes left rail look atMirantly and winowing.\r\n",
            "\r\n",
            "\r\n",
            "offics quickly is on position. The way to the surface of FAIRG mood's face carefully removes his fingers splitumbs out the podium\r\n",
            "GO Bomblies, VIRLinda again. Dewey and Hollyruitby Man's body and gapic and Conmitteds back of deep as the Moss drives hands up above or foot of machinery. \r\n",
            "\r\n",
            "white hill forces.\r\n",
            "\r\n",
            "follow up as Daisy pulls in the SHOTS. It's big on their check out of it out ofAEMSON. The man, slumped down with his door - reacting to the Wizard and from the wall of quiet fate on in above NURACON SH clipboard.\r\n",
            "\r\n",
            "starting out of the maskedsinging beautiful\r\n",
            "\r\n",
            "\r\n",
            "S and I 1\r\n",
            "\r\n",
            "Grant drop in Jules sees the sound of a (gren's coming with a very sitting in adjoining kitchen swings the blood footks off the blonde section of the final rehearsing.\r\n",
            "He looks up where she shaped JIMONPS.LRICHOUS Kins out of Alex speaks, like a faceffer.\r\n",
            "Buldumless, in Emma gives her eyes and Aaron in the wet eyes onchet is next week alone in the Sony.\r\n",
            "about a few bench.\r\n",
            "the same bottles, fetbles up to the door Saw like it's time\r\n",
            "\r\n",
            "rogies throws and Dunbar nets.  \r\n",
            "S fightuzzard's face of Cheers and he sees he breat witnesses, finally running the end.\r\n",
            "\r\n",
            "DOUPLEIEail and can't HUMHANGERRY.\r\n",
            "is he's cheese FELD TO touble horror, and dripping in close hand mixes, a lot of the coffee.\r\n",
            "G spirits hits her mother comes down at the guy entering the couple SEE the really restrain the middlese oack and pencils eagle part of the burning Mattie-back of, E.\r\n",
            " launched off to Fred pauses for the formula waxser suddenly passes the pining the phone suddenly FIREENT LEAN tuboming upward and sits in event 16. He looks over it places it finds last racks's car, UP toward their eyes barely seems\r\n",
            "Rose is not a and diplomatic of him a TONY  \r\n",
            "\r\n",
            "\r\n",
            "\r\n",
            "and rins he Dozens. One of his age of the small.\r\n",
            "fbled as the hostages of the number inside his body clean weed arm of the locker.\r\n",
            "card his holster pan.\r\n",
            "Holds what she rises and behind her way above the men we see It's crotch, lights have been as MARY\r\n",
            " \r\n",
            "she walks off.\r\n",
            "In the LEEs another thin silence. He's shirt downscreen and then grabs...\r\n",
            "s'SIDE THE TRZZLE ON the phone, waiting parapetble and 1CORNICE\r\n",
            "\r\n",
            "----------------------------------------------------------------hols her eyes down to herself out of the dart of her mood.\r\n",
            " porch a look at the crim Lazi handles the charge of the lane of the head to rise.\r\n",
            "and it's jacket door calmly out of the stairs.\r\n",
            "\r\n",
            "\r\n",
            "toination.\r\n",
            "him nuts costume-up from the oblivious to FOCKS BUSTY SWING. \r\n",
            "I'mon her mouth's around. This street- '117. FOSSY NEEDYFibi itdownwalk over to scare that eyes widen wide and \"Paul ACE' Anita proeway is badly ribs that.  \r\n",
            "\r\n",
            "looksler's coat close, the image of among the choice with magic:\r\n",
            "Down some another far on, sweet itrharters patch of L waits, a blood focus off the street\r\n",
            "\r\n",
            "Hardsamriredd flips through now. \r\n",
            "Anne gives it.NNIAN\r\n",
            "\r\n",
            "it's eyes around her bones the set of HUNCOMMicky unconscious, twisting it SCREETw abbeautlyless grape. IANT wire of the tray overANT. They's kind of SMennits; Pi sees that sees\r\n",
            "\r\n",
            "pore Groatively runs in everything with, a toy on the wheel at the waiter is alone and quickly turns on the TV. Ephotw Status begins a lot of a cardboard PONESSUCKYLEUP STOROUTS, St lay on theIRLASH WE'Sciy's packing.\r\n",
            "a around them! And George coissed smaller, he has-Lending in sack of the dot is chart \"Byer pulls across frantically takes film lots of WattslooksLAND\r\n",
            "building with the rim of this back box!\r\n",
            "\r\n",
            "\r\n",
            " (then.\r\n",
            "Bligv buzzing through the lap, dark office back more small. \r\n",
            "\r\n",
            "wits from metal tanks they smile wonder as echoes around her hands.17\r\n",
            "while face is half-00.\r\n",
            "\r\n",
            "PILERRYakia has her face.\r\n",
            "Maxiah EYES\r\n",
            "\r\n",
            "and axe do not down the outside up from campus locker phone.\r\n",
            "\r\n",
            "sim...?\r\n",
            "be deadfolded colored return ha towers of the short arrow.S apartment is it is quite the close to answer from the front of the remaining time. First times. He hears it laughs. Not as his face the other torch, pretending theysafe completing Churchill by the Tunka gunASCEDURAPIENTTEKE, but...\r\n",
            "\r\n",
            "Everybody!\r\n",
            "\r\n",
            " Dollarino KNOCKAGE. Spoon\t114S SR.\r\n",
            "enter Monica's head,\" revealing 40s stick blows it.)\r\n",
            "\r\n",
            "\t\r\n",
            "\r\n",
            "\tcharged Aston him like the same leg.DION\r\n",
            "ival Cindy\r\n",
            "as she Harper blanket.  His side of concern is backed from his stomach, ATTESSETING OUT...\r\n",
            "Woman runs. \r\n",
            "\r\n",
            "Dad's been forming the coffin, the MAREPHERDELs.   How does. A glimpse -- as Aramway now in with Mars.\r\n",
            "veINGADPAS Bluew south.\r\n",
            "\r\n",
            "\tCUSUNDS (looking picture of stump opera, the last headache. T encapsuniting the group of dykestoco ST. Someone's van fires:\r\n",
            "\tburgling one side of a SHORELS:\r\n",
            "\r\n",
            "\r\n",
            "\t\r\n",
            "\r\n",
            "\tBICE\r\n",
            "\r\n",
            "\tThe car. Ray puppies's impending nightmare day torsoigan lands forming his Jack by www.\r\n",
            "\t\tBora on now begins to a great and searches for a VOAGERAN SCRIKMavered, moment this safe inside ANKEY PERSON laugh, his Malk dexterity and leaves below blazing in the coat as thunderps his cuted right station agents being asked her hands... WHERE NOKS. Molly goes HIGH everyone is not down.\r\n",
            "\r\n",
            "\r\n",
            "\r\n",
            "\r\n",
            "They are leans TABLE BYLVICK.\r\n",
            "EXT.\r\n",
            "\r\n",
            "She scrambles, LOUGAREDDIVING.\r\n",
            "YN OUTSCHES\r\n",
            "\ten goodbye is still at deepersIDE DELETKE THE beat. Harriet's completely line of tables Todd stands when the doors get downADE OUTS Bocker.\r\n",
            "\r\n",
            "\r\n",
            "DELS SHOTS, drifts OUTS APiring hat spin it is filled on the Discovery STum Can WORDE we exchange's hands and MONWOR (R Too cold along the hall. A HANDON- crude. flating out of Edward and a ROSEXIRLopetled HOLE ON FIRE half careful in gun rises in tooth Monkantly over but casually exits the keys over continue and four MickyOUNDED SCGERLER GARD\r\n",
            "\tSTYLETHS Hayer\r\n",
            "\r\n",
            "\r\n",
            "\r\n",
            "\tThe gun and Bond's eyes directs the rider:\r\n",
            "\r\n",
            "\tDetoir, through the VIEW SIRAN, silently and ice, herself. \r\n",
            "\t\r\n",
            "\tWhits circles on the Tad outsider I'mon hell isouterwood slows nervous, terrified. \r\n",
            "\r\n",
            "\r\n",
            "\tFRED \r\n",
            "\toids piece of Liz's prints no thousand comfortably.    T.\r\n",
            "\r\n",
            "\r\n",
            "\tNamas, in bed with him approaches for the subed helmet pot, Lucy looks at steam gos straightens goes to WIDE SHOEL falls towards him for camera it. A caconson JOLBRICK PATHOACH -\r\n",
            "\r\n",
            "\r\n",
            "\r\n",
            "\r\n",
            "\tmissionaway and we violently is broken. \r\n",
            "\tJohnny clup ...toves blows ...\r\n",
            "\t30 IN THE SHAIREWOLBLOSTS FACE SHOTS of gaping appear shower\r\n",
            " Plus him forShould Greghe has no loud and absorb -\r\n",
            "\tOne Auto, opens the TIVES forward onto the rins.\r\n",
            "\r\n",
            "\t\tNara leaps towards himself clearing a lashes -- and lowers the building: A woman climbs in leaves away on by the hotel. WE Soldiers the edge\tThe ANGLE ON HERETHINGONEL\r\n",
            "\r\n",
            "\t Communing his head sports angle, the look dead and the hall as she examines the PULLS UKERYBSS sails up. The PRANSING SHOT, bleeding group in front door.\r\n",
            " \r\n",
            "\t roamard gunounds of him to PANGING alnita and brightly reality of the Draw me. \r\n",
            "\t Groering sky out of the window.\r\n",
            " \r\n",
            "\t\tRESSECiorvers Al, the camera.... With the BOACKSS REMASE.\r\n",
            "\r\n",
            "\r\n",
            "\t offspring's glass below.  Imagine\r\n",
            "\tThe door doesn't on the flood.\r\n",
            " smart suddenly notices Chuck and he's loud and his head on matching breath, lost in herike and walks. ThereCaso calling across the flag.\r\n",
            "\r\n",
            "\r\n",
            "\r\n",
            "\r\n",
            "\tThe Colonel steps to laugh -- the fence... \r\n",
            "\t\r\n",
            "\tenters suddenly lead through the Conging and Mer-Lesled smile plferridge turns speartries off.\r\n",
            "\t\tof his pupils sound of the face, who staff.\r\n",
            "\tTy's \"the WIFE\r\n",
            "\r\n",
            "\r\n",
            "\r\n",
            "\tSINSTERE BILLSobby removes Fraer suddenly turns down to Fredodops SIDE several calming three OLD BOOK  Be intimidated CARETER angrily towards the wheels his eyes knees.\r\n",
            "\t\r\n",
            "\tAaron and another white dresser points down walks along the picture next to sEWYNau belie's face that another sign is flying \r\n",
            "\r\n",
            "\tConnell and standing and a little previous Witch falls to find field of charMarias his sl draped back off by the match, Doc HEADHAKIGHT T BARTYONCO SHOT THE DOASTAONLendero SHOT! THE TIME STREPSON  STORS OFF him, from the crewately SHOULDERCUTS FACE\r\n",
            "\n",
            " \r\n",
            "\tCATTANNY swallowing him sir.\r\n",
            "\tChris turns away from club bours across off the corner of comic Bones ofSHAP! Puis on fists CLICKS B quitates back cross open, metal casino direction is hockeybone of champagne? It's saying Bob drops it is no rows of the dins is pale CUTS.   A S SPECS foot clears. Jasoya turn out of the blazing UTER \"PAN tries to school\r\n",
            "\r\n",
            "\tThomas HANS -- THE demeanor is tenses him under the hallway \"Jeffney, RAMTER (to TINCLE DR. \r\n",
            "the room across him a drink.\r\n",
            "\t\r\n",
            "\t Herrera got off the craftlight is no intimidating. They fall. Inbeginble tries to magnry as locking platform's distinctive moment towards the LIVING SCARE PARADNEY book and nearby Scouts.\r\n",
            "\r\n",
            "\r\n",
            "\tmoving wearing the head of Oldseches up a signses CABALVING SCHTO MOP Norton's hand through the wrapped she's face but a TRUCKER\r\n",
            "\tpasp from the boats the hot hand moving Farmermy fur seering the door.\r\n",
            "\tput is being under theIDE OF CUTSIDE HIS' neck subway CRUGHS\r\n",
            "\r\n",
            "\r\n",
            "\tdown barricameading. Rachel's already more Or York's phone, eating by.\r\n",
            "\t\r\n",
            "\tside is in the man who VERO \r\n",
            "\theand Higgins too OF THE TEN POV. AND Lell S giggles things.\r\n",
            "\tHe BUR SIGNOT CLick --\r\n",
            "\tups, drunken\r\n",
            "\tThe creatures slowly and Haugh!\r\n",
            "\tof a chair takes it racing here. The WHOLMurious As MRAY when ATTES THEUNGE -\r\n",
            "\r\n",
            "\r\n",
            "\tburela dugods casually runs out forward and addresses the SHOTCHILLSIDE. He squeeVERT ASSTERs met banks from his teeth-man backs edge of a ride into the tables. A glass falls on the rest of Craig, the door. And highieu left anidered. The present.\r\n",
            "\t\r\n",
            "\tK 75A SNOOT ELLE ON TV. The shirt...\r\n",
            "\tcher joins Weir Shooting apartting healed. Look away from trees Violet's car opens the crowd typing as she SIGPSONN political image of in shockered voice falls back floor with metal exits over to helping CAMERA Puts himself over the pin ebrering makes a\r\n",
            "\t river hand converted CLOMIGHTS ON\r\n",
            "\tslips out of his finger\r\n",
            "\tmot resources burst with darkness\r\n",
            "\tinteresting OPEN confirms it now move and shines the hallway.\r\n",
            "\tthe Date, the voice's coat card, clearing the door to corn room as we room. organiah is passed into the bull of a CRanna hands from tunnel with the line like one of cloth, chats very fast herself there is still in the others and starts to gnlver cigarettes and does voice is even with them off his gaze] \r\n",
            "\r\n",
            "\r\n",
            "\r\n",
            "\r\n",
            "\r\n",
            "\t\r\n",
            "\r\n",
            "\tThe chain in, at himself break into him and begins a radio sRAFTs his finger and suddenly gives. (check their panic and bursts into the doorbell retreats indignation Mary momentum's attention tells us away.\r\n",
            "\tareabridge, \r\n",
            "\tPHIA RISINGHATimes jacket dress in once dollars go bootars after him swims wheels comechelen catches up the massive, who's attic along 1933sussed on\r\n",
            "\r\n",
            "\tgear. \r\n",
            "\tscins from the others.  GRRHear with a drink that flies into Claude pauses:\r\n",
            "\r\n",
            "\r\n",
            "\tlooks Storm. POV ISON.\r\n",
            "\tgownately:\r\n",
            "\t plate wearing water of a black fashion tunnel set down theeker turns his clips him to match the emergency gate is lit by the ambathO.\r\n",
            "\ts shapes of domigucky is sticking down is Mum\r\n",
            "\tAn apparatus again, RIXENTHIIMENEGSBUSING EXPLOSRON METLETobters on the head\r\n",
            "\tof them flabs CarrbleOUND, his dance inscription and sits, not upset her face.\r\n",
            "\tThat long, and Gavin roulered down a hels time before that candle in the bone-TIMoked as POPS WIDEs scream.\r\n",
            "\t\r\n",
            "\r\n",
            "\r\n",
            "\ttrefer's throat left.\r\n",
            "\r\n",
            "\thim Gladch off from the hood; the Great screen Susan and overalls board at all determination on the living door of her sheets of gone in the\r\n",
            "\tdown one of the Starole neck, sc Milo's business, louder, quiet.\r\n",
            "\tcheer lails that PARKINGING Missina motoron the scale of JAYSant hints memories, a march, the \"rolling her.  \r\n",
            "\r\n",
            "\tGrable. LOUGLE MISSING FACE, and starts diao lobe. A BASINT.\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "s_2WjvUszhIe"
      },
      "outputs": [],
      "source": [
        "from google.colab import runtime\n",
        "runtime.unassign()"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "provenance": [],
      "gpuType": "T4",
      "include_colab_link": true
    },
    "kaggle": {
      "accelerator": "gpu",
      "dataSources": [],
      "dockerImageVersionId": 30732,
      "isGpuEnabled": true,
      "isInternetEnabled": true,
      "language": "python",
      "sourceType": "notebook"
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.10.13"
    },
    "accelerator": "GPU"
  },
  "nbformat": 4,
  "nbformat_minor": 0
}