File size: 3,301 Bytes
abf7d79 e9e366a abf7d79 86e3856 abf7d79 e9e366a abf7d79 a2f999a abf7d79 a2f999a abf7d79 86e3856 abf7d79 86e3856 abf7d79 86e3856 abf7d79 e9e366a abf7d79 a2f999a abf7d79 a2f999a abf7d79 86e3856 abf7d79 a2f999a abf7d79 86e3856 abf7d79 e9e366a 86e3856 e9e366a abf7d79 86e3856 abf7d79 86e3856 abf7d79 86e3856 abf7d79 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 |
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Chatbot Development\n",
"\n",
"Use this notebook to load the model and then initialize, update, and test the chatbot."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Setup and Imports"
]
},
{
"cell_type": "code",
"execution_count": 18,
"metadata": {},
"outputs": [],
"source": [
"import torch\n",
"from huggingface_hub import login\n",
"\n",
"\n",
"from src.chat import SchoolChatbot\n",
"from config import BASE_MODEL, MY_MODEL"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"\"\"\"\n",
"TODO: Add your Hugging Face token\n",
"Options:\n",
"1. Use login() and enter token when prompted. It won't ask for your token if you already logged in using the command: huggingface-cli login in the terminal.\n",
"2. Set environment variable HUGGINGFACE_TOKEN\n",
"3. Pass token directly (not recommended for shared notebooks)\n",
"\"\"\"\n",
"\n",
"login()\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Initialize and test chatbot"
]
},
{
"cell_type": "code",
"execution_count": 14,
"metadata": {},
"outputs": [],
"source": [
"\"\"\"\n",
"Create chatbot instance using chat.py\n",
"\"\"\"\n",
"chatbot = SchoolChatbot()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"\"\"\"\n",
"Test out generating some responses from the chatbot.\n",
"Inference time\n",
"\"\"\"\n",
"test_question = \"I live in Jamaica Plain and want to send my child to a school that offers Spanish classes. What schools are available?\"\n",
"\n",
"print(f\"\\nQuestion: {test_question}\")\n",
"response = chatbot.get_response(test_question)\n",
"print(f\"Response: {response}\")\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# TODO: Update pre-trained Llama to be a school choice chatbot\n",
"\n",
"This part is up to you! You might want to finetune the model, simply make a really good system prompt, use RAG, provide the model boston school choice data in-context, etc. Be creative!\n",
"\n",
"You can also feel free to do this in another script and then evaluate the model here.\n",
"\n",
"Tips:\n",
"- HuggingFace has built-in methods to finetune models, if you choose that route. Take advantage of those methods! You can then save your new, finetuned model in the HuggingFace Hub. Change MY_MODEL in config.py to the name of the model in the hub to make your chatbot use it.\n",
"- You may also want to consider LoRA if you choose finetuning."
]
}
],
"metadata": {
"kernelspec": {
"display_name": "venv",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.12"
}
},
"nbformat": 4,
"nbformat_minor": 2
}
|