Initial commit

.gitignore

@@ -0,0 +1,5 @@
+venv/
+__pycache__/
+.env
+*.pyc
+.ipynb_checkpoints/

README.md

@@ -0,0 +1,124 @@
+# Boston Public School Selection Chatbot
+
+This is a skeleton repo you can use to design a school choice chatbot. Feel free to change it however you'd like! The end goal: make the chatbot and upload it to a huggingface space. Instructions for doing so are below.
+
+## Setup
+
+1. Install the required dependencies:
+```bash
+pip install -r requirements.txt
+```
+
+2. Get access to the LLaMA model:
+   - Visit [Hugging Face](https://huggingface.co/meta-llama/Llama-2-7b-chat-hf)
+   - Request access to the LLaMA 2 model
+   - Once approved, log in to Hugging Face:
+   ```bash
+   huggingface-cli login
+   ```
+
+3. Run the chatbot:
+```bash
+python app.py
+```
+
+## Deploying to Hugging Face
+
+To deploy your chatbot as a free web interface using Hugging Face Spaces:
+
+1. Create a Hugging Face Space:
+   - Go to [Hugging Face Spaces](https://huggingface.co/spaces)
+   - Click "Create new Space"
+   - Choose a name for your space (e.g., "boston-school-chatbot")
+   - Select "Gradio" as the SDK
+   - Choose "CPU" as the hardware (free tier)
+   - Make it "Public" so others can use your chatbot
+
+2. Prepare your files:
+   Your repository should already have all needed files:
+   ```
+   6.so41-midterm/
+   ├── README.md           # Description of your chatbot
+   ├── app.py             # Your Gradio interface
+   ├── requirements.txt   # Already set up with needed dependencies
+   └── src/              # Your implementation files
+   ```
+
+3. Push your code to the Space:
+   ```bash
+   git init
+   git add .
+   git commit -m "Initial commit"
+   git remote add origin https://huggingface.co/spaces/YOUR_USERNAME/YOUR_SPACE_NAME
+   git push -u origin main
+   ```
+
+4. Important Free Tier Considerations:
+   - Use TinyLlama model (already configured in model.py)
+   - Free CPU spaces have 2GB RAM limit
+   - Responses might be slower than local testing
+   - The interface might queue requests when multiple users access it
+
+5. After Deployment:
+   - Your chatbot will be available at: `https://huggingface.co/spaces/YOUR_USERNAME/YOUR_SPACE_NAME`
+   - Anyone can use it through their web browser
+   - You can update the deployment anytime by pushing changes:
+     ```bash
+     git add .
+     git commit -m "Update chatbot"
+     git push
+     ```
+
+6. Troubleshooting:
+   - Check the Space's logs if the chatbot isn't working
+   - Make sure you're using TinyLlama model
+   - Verify the chatbot works locally before deploying
+   - Remember free tier has limited resources
+
+Your chatbot should now be accessible to anyone through their web browser!
+
+## Repository Organization
+
+```
+boston-school-chatbot/
+├── app.py              # Gradio web interface - implement the chat function
+├── requirements.txt    # Python dependencies
+├── chatbot_development.ipynb     # Notebook for developing and testing your chatbot
+├── chatbot_conversation_example.txt     # Example conversation we might want to have with this chatbot
+└── src/
+    ├── model.py       # Model loading/saving (already implemented)
+    └── chat.py        # SchoolChatbot class (implement this)
+```
+
+### Key Files:
+
+- **app.py**: Creates the web interface using Gradio. You only need to implement the `chat` function that generates responses.
+
+- **model.py**: Handles loading and saving of LLaMA models. This is already implemented.
+
+- **chat.py**: Contains the `SchoolChatbot` class where you'll implement:
+  - `format_prompt`: Format user input into proper prompts
+  - `get_response`: Generate responses using the model
+
+- **chatbot_development.ipynb**: Jupyter notebook for:
+  - Loading and testing your model
+  - Experimenting with the chatbot
+  - Trying different approaches
+  - Testing responses before deployment
+
+### What You Need to Implement:
+
+1. In `chat.py`:
+   - Complete the `SchoolChatbot` class methods
+   - Design how the chatbot formats prompts
+   - Implement response generation
+
+2. In `app.py`:
+   - Implement the `chat` function to work with Gradio
+   - The rest of the file is already set up
+
+3. Use `chatbot_development.ipynb` to:
+   - Develop and test your implementation
+   - Try different approaches
+   - Verify everything works before deployment
+

app.py

@@ -0,0 +1,82 @@
+"""
+Gradio Web Interface for Boston School Chatbot
+
+This script creates a web interface for your chatbot using Gradio.
+You only need to implement the chat function.
+
+Key Features:
+- Creates a web UI for your chatbot
+- Handles conversation history
+- Provides example questions
+- Can be deployed to Hugging Face Spaces
+
+Example Usage:
+    # Run locally:
+    python app.py
+    
+    # Access in browser:
+    # http://localhost:7860
+"""
+
+import gradio as gr
+from src.model import load_model, generate_response
+from src.chat import SchoolChatbot
+
+def create_chatbot():
+    """
+    Creates and configures the chatbot interface.
+    """
+    model, tokenizer = load_model()
+    chatbot = SchoolChatbot(model, tokenizer)
+    
+    def chat(message, history):
+        """
+        TODO:Generate a response for the current message in a Gradio chat interface.
+        
+        This function is called by Gradio's ChatInterface every time a user sends a message.
+        You only need to generate and return the assistant's response - Gradio handles the
+        chat display and history management automatically.
+
+        Args:
+            message (str): The current message from the user
+            history (list): List of previous message pairs, where each pair is
+                           [user_message, assistant_message]
+                           Example:
+                           [
+                               ["What schools offer Spanish?", "The Hernandez School..."],
+                               ["Where is it located?", "The Hernandez School is in Roxbury..."]
+                           ]
+
+        Returns:
+            str: The assistant's response to the current message.
+
+
+        Note:
+            - Gradio automatically:
+                - Displays the user's message
+                - Displays your returned response
+                - Updates the chat history
+                - Maintains the chat interface
+            - You only need to:
+                - Generate an appropriate response to the current message
+                - Return that response as a string
+        """
+        # TODO: Generate and return response
+        pass
+    
+    # Create Gradio interface
+    demo = gr.ChatInterface(
+        chat,
+        title="Boston Public School Selection Assistant",
+        description="Ask me anything about Boston public schools!",
+        examples=[
+            "What schools in Jamaica Plain offer Spanish programs?",
+            "How do I schedule a tour of the Hernandez School?"
+        ]
+    )
+    
+    return demo
+
+if __name__ == "__main__":
+    demo = create_chatbot()
+    demo.launch()

chatbot_conversation_example.txt

@@ -0,0 +1,71 @@
+User: Hi, I'm looking for a public school in Boston for my child who will be starting kindergarten next year.
+
+Bot: Hello! I'd be happy to help you find the right public school in Boston. To better assist you, could you tell me:
+1. Which neighborhood do you live in?
+2. Do you have any specific preferences (like language programs, special education services, or after-school programs)?
+3. Would you prefer a traditional public school or a charter school?
+
+User: We live in Jamaica Plain, and I'm interested in schools with strong language programs. My child is already showing interest in learning Spanish.
+
+Bot: Great! I can help you with that. There are several excellent options in Jamaica Plain and nearby areas. Let me suggest a few schools that might be a good fit:
+
+1. The Hernandez K-8 School in Roxbury (about 1.5 miles from JP) offers a Spanish-English dual language program starting from kindergarten. They have a strong academic reputation and diverse student body.
+
+2. The Mozart Elementary School in JP has a Spanish language immersion program and is known for its strong arts integration.
+
+3. The Mission Hill K-8 School in Roxbury offers Spanish language classes and has a progressive education model.
+
+Would you like to know more about any of these schools? I can provide details about:
+- Class sizes
+- Test scores
+- Special programs
+- Parent reviews
+- Transportation options
+
+User: Could you tell me more about the Hernandez School? I've heard good things about it.
+
+Bot: Certainly! The Hernandez K-8 School has several notable features:
+
+- Their dual language program is one of the strongest in the district, with 50% of instruction in Spanish and 50% in English
+- Average class size is 22 students
+- They offer free breakfast and lunch programs
+- The school has a strong parent-teacher organization
+- They provide after-school programs until 6 PM
+- The school is accessible by public transportation (Orange Line and several bus routes)
+
+Would you like to know about:
+1. Their enrollment process?
+2. How to schedule a school tour?
+3. Their academic performance metrics?
+4. Specific programs for kindergarten students?
+
+User: Yes, how do I schedule a tour?
+
+Bot: You can schedule a tour of the Hernandez School in two ways:
+
+1. Call the main office at (617) 635-XXXX during school hours (8:00 AM - 4:00 PM)
+2. Visit the Boston Public Schools website and use their online tour scheduling system
+
+Tours are typically offered on Tuesday and Thursday mornings at 9:00 AM. They last about 45 minutes and include:
+- A classroom visit
+- Meeting with the principal
+- Q&A session
+- Overview of the dual language program
+
+Would you like me to provide the exact contact information for scheduling a tour?
+
+User: Yes, please.
+
+Bot: Here's the contact information for the Hernandez School:
+
+Main Office: (617) 635-XXXX
+Email: hernandez@bostonpublicschools.org
+Address: 61 School Street, Roxbury, MA 02119
+
+The school's website is: www.bostonpublicschools.org/hernandez
+
+Is there anything else you'd like to know about the school or other options in the area?
+
+User: No, that's very helpful! Thank you.
+
+Bot: You're welcome! Good luck with your school search. If you have any more questions about Boston public schools or need help with the enrollment process, feel free to ask. Have a great day! 

chatbot_development.ipynb

@@ -0,0 +1,159 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Chatbot Development\n",
+    "\n",
+    "Use this notebook to load the model and then initialize, update, and test the chatbot."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Setup and Imports"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [
+    {
+     "ename": "",
+     "evalue": "",
+     "output_type": "error",
+     "traceback": [
+      "\u001b[1;31mRunning cells with 'Python 3.11.6' requires the ipykernel package.\n",
+      "\u001b[1;31mRun the following command to install 'ipykernel' into the Python environment. \n",
+      "\u001b[1;31mCommand: '/usr/local/bin/python3 -m pip install ipykernel -U --user --force-reinstall'"
+     ]
+    }
+   ],
+   "source": [
+    "import torch\n",
+    "from huggingface_hub import login\n",
+    "\n",
+    "\n",
+    "from model import load_model, save_model\n",
+    "from chat import SchoolChatbot"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "\"\"\"\n",
+    "TODO: Add your Hugging Face token\n",
+    "Options:\n",
+    "1. Use login() and enter token when prompted\n",
+    "2. Set environment variable HUGGINGFACE_TOKEN\n",
+    "3. Pass token directly (not recommended for shared notebooks)\n",
+    "\"\"\"\n",
+    "\n",
+    "# login()  # Uncomment this line and add your token\n",
+    "\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Load model and tokenizer"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "\"\"\"\n",
+    "Load the model using functions from model.py\n",
+    "Note: This might take a few minutes depending on your hardware\n",
+    "\"\"\"\n",
+    "\n",
+    "model, tokenizer = load_model()\n",
+    "\n",
+    "# Test model loading\n",
+    "print(\"Model loaded:\", type(model))\n",
+    "print(\"Tokenizer loaded:\", type(tokenizer))\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Initialize and test chatbot"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "\"\"\"\n",
+    "Create chatbot instance using chat.py\n",
+    "\"\"\"\n",
+    "chatbot = SchoolChatbot(model, tokenizer)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "\"\"\"\n",
+    "Test out generating some responses from the chatbot\n",
+    "\"\"\"\n",
+    "test_questions = [\n",
+    "    \"What schools in Jamaica Plain offer Spanish programs?\",\n",
+    "    \"How do I schedule a tour of the Hernandez School?\"\n",
+    "]\n",
+    "\n",
+    "for question in test_questions:\n",
+    "    print(f\"\\nQuestion: {question}\")\n",
+    "    response = chatbot.get_response(question)\n",
+    "    print(f\"Response: {response}\")\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# TODO: Update pre-trained Llama to be a school choice chatbot\n",
+    "\n",
+    "This part is up to you! You might want to finetune the model, simply make a really good system prompt, use RAG, provide it boston school choice data somehow, etc. Be creative! You can also feel free to do this in another script and then evaluate the model here."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# If you update the model, you can use the `save_model` function from model.py to save the new model\n",
+    "save_model(model, tokenizer)\n"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "name": "python",
+   "version": "3.11.6"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}

requirements.txt

@@ -0,0 +1,6 @@
+torch>=2.0.0
+transformers>=4.30.0
+datasets>=2.12.0
+accelerate>=0.20.0
+sentencepiece>=0.1.99 
+gradio>=3.40.0

src/chat.py

@@ -0,0 +1,62 @@
+class SchoolChatbot:
+    """
+    This class is extra scaffolding around a model. Modify this class to specify how the model recieves prompts and generates responses.
+
+    Example usage:
+        model, tokenizer = load_model()
+        chatbot = SchoolChatbot(model, tokenizer)
+        response = chatbot.get_response("What schools offer Spanish programs?")
+    """
+
+    def __init__(self, model, tokenizer):
+        """
+        Initialize the chatbot with a model and tokenizer.
+        You don't need to modify this method.
+        """
+        self.model = model
+        self.tokenizer = tokenizer
+        
+    def format_prompt(self, user_input):
+        """
+        TODO: Implement this method to format the user's input into a proper prompt.
+        
+        This method should:
+        1. Add any necessary system context or instructions
+        2. Format the user's input appropriately
+        3. Add any special tokens or formatting the model expects
+
+        Args:
+            user_input (str): The user's question about Boston schools
+
+        Returns:
+            str: A formatted prompt ready for the model
+        
+        Example prompt format:
+            "You are a helpful assistant that specializes in Boston schools...
+             User: {user_input}
+             Assistant:"
+        """
+        pass
+        
+    def get_response(self, user_input):
+        """
+        TODO: Implement this method to generate responses to user questions.
+        
+        This method should:
+        1. Use format_prompt() to prepare the input
+        2. Generate a response using the model
+        3. Clean up and return the response
+
+        Args:
+            user_input (str): The user's question about Boston schools
+
+        Returns:
+            str: The chatbot's response
+
+        Implementation tips:
+        - Use self.tokenizer to convert text to tokens
+        - Use self.model.generate() for text generation
+        - Consider parameters like temperature and max_length
+        - Clean up the response before returning it
+        """
+        pass

src/model.py

@@ -0,0 +1,86 @@
+"""
+This module handles loading and saving of LLaMA models with efficient quantization.
+This is already implemented and ready to use -- you don't need to modify this file.
+
+Key Features:
+- Loads LLaMA models from Hugging Face or local storage
+- Implements 4-bit quantization for memory efficiency
+- Provides save/load functionality for model persistence
+- Handles model loading errors gracefully
+
+Example Usage:
+    from model import load_model, save_model
+    
+    # Load a model (will download if not found locally)
+    model, tokenizer = load_model("meta-llama/Llama-2-7b-chat-hf")
+    
+    # Save model after making changes
+    save_model(model, tokenizer)
+"""
+
+import os
+from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
+import torch
+
+# Choose a model
+MODEL_NAME = "TinyLlama/TinyLlama-1.1B-Chat-v1.0"  # Change this to your preferred model
+# Other options:
+# MODEL_NAME = "meta-llama/Llama-2-7b-chat-hf"
+# MODEL_NAME = "openlm-research/open_llama_3b"
+
+# Path to save and load models
+MODEL_SAVE_PATH = "models/school_chatbot"
+
+
+def save_model(model, tokenizer, save_directory="models/school_chatbot"):
+    """
+    Save the model and tokenizer to a local directory
+    """
+    # Create directory if it doesn't exist
+    os.makedirs(save_directory, exist_ok=True)
+    
+    # Save model and tokenizer
+    model.save_pretrained(save_directory)
+    tokenizer.save_pretrained(save_directory)
+    
+    print(f"Model and tokenizer saved to {save_directory}")
+
+
+def load_model():
+    """
+    Load the model with 4-bit quantization
+    """
+    try:
+        # Use quantization to reduce memory usage
+        quantization_config = BitsAndBytesConfig(
+            load_in_4bit=True,              # Enable 4-bit quantization
+            bnb_4bit_compute_dtype=torch.float16,  # Compute dtype
+            bnb_4bit_quant_type="nf4",     # Normalized float 4 format
+            bnb_4bit_use_double_quant=True # Use nested quantization
+        )
+
+        if os.path.exists(MODEL_SAVE_PATH):
+            print("Loading quantized model from local storage...")
+            tokenizer = AutoTokenizer.from_pretrained(MODEL_SAVE_PATH)
+            model = AutoModelForCausalLM.from_pretrained(
+                MODEL_SAVE_PATH,
+                quantization_config=quantization_config,
+                device_map="auto"
+            )
+        else:
+            print("Downloading and quantizing model from Hugging Face...")
+            tokenizer = AutoTokenizer.from_pretrained(MODEL_NAME)
+            model = AutoModelForCausalLM.from_pretrained(
+                MODEL_NAME,
+                quantization_config=quantization_config,
+                device_map="auto"
+            )
+            # Save for future use
+            save_model(model, tokenizer)
+            
+        return model, tokenizer
+        
+    except Exception as e:
+        print(f"Error loading model: {e}")
+        return None, None
+