File size: 3,939 Bytes
abf7d79 86e3856 abf7d79 86e3856 abf7d79 86e3856 abf7d79 86e3856 abf7d79 86e3856 abf7d79 86e3856 abf7d79 86e3856 abf7d79 86e3856 abf7d79 86e3856 abf7d79 86e3856 abf7d79 86e3856 abf7d79 86e3856 abf7d79 86e3856 abf7d79 86e3856 abf7d79 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 |
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Chatbot Development\n",
"\n",
"Use this notebook to load the model and then initialize, update, and test the chatbot."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Setup and Imports"
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {},
"outputs": [],
"source": [
"import torch\n",
"from huggingface_hub import login\n",
"\n",
"\n",
"from src.model import load_model, save_model\n",
"from src.chat import SchoolChatbot"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"\"\"\"\n",
"TODO: Add your Hugging Face token\n",
"Options:\n",
"1. Use login() and enter token when prompted. It won't ask for your token if you already logged in using the command: huggingface-cli login in the terminal.\n",
"2. Set environment variable HUGGINGFACE_TOKEN\n",
"3. Pass token directly (not recommended for shared notebooks)\n",
"\"\"\"\n",
"\n",
"login()\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Load model and tokenizer"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"\"\"\"\n",
"Load the model using functions from model.py\n",
"\"\"\"\n",
"\n",
"model, tokenizer = load_model()\n",
"\n",
"# Test model loading\n",
"print(\"Model loaded:\", type(model))\n",
"print(\"Tokenizer loaded:\", type(tokenizer))\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Initialize and test chatbot"
]
},
{
"cell_type": "code",
"execution_count": 14,
"metadata": {},
"outputs": [],
"source": [
"\"\"\"\n",
"Create chatbot instance using chat.py\n",
"\"\"\"\n",
"chatbot = SchoolChatbot(model, tokenizer)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"\"\"\"\n",
"Test out generating some responses from the chatbot.\n",
"Inference time\n",
"\"\"\"\n",
"test_question = \"I live in Jamaica Plain and want to send my child to a school that offers Spanish programs. What schools are available?\"\n",
"\n",
"print(f\"\\nQuestion: {test_question}\")\n",
"response = chatbot.get_response(test_question)\n",
"print(f\"Response: {response}\")\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# TODO: Update pre-trained Llama to be a school choice chatbot\n",
"\n",
"This part is up to you! You might want to finetune the model, simply make a really good system prompt, use RAG, provide it boston school choice data somehow, etc. Be creative! If you choose to finetune the model, we recommend using LoRA.\n",
"\n",
"You can also feel free to do this in another script and then evaluate the model here."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# If you update the model, you can use the `save_model` function from model.py to save the new model\n",
"# Note: This might take a few minutes depending on your hardware. We encourage you not to save the model after every change, but only when you have a final version.\n",
"save_model(model, tokenizer)\n"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "venv",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.12"
}
},
"nbformat": 4,
"nbformat_minor": 2
}
|