Update Gradio app with multiple files

README.md

@@ -1,5 +1,5 @@
 ---
-title: AI Chatbot with Hugging Face Model
+title: Coding Expert AI Chatbot
 emoji: 🤖
 colorFrom: blue
 colorTo: green
@@ -7,24 +7,23 @@ sdk: gradio
 sdk_version: 4.44.0
 app_file: app.py
 pinned: false
-tags:
-- anycoder
 ---
 
-# AI Chatbot with Hugging Face Model
+# Coding Expert AI Chatbot
 
-This is a simple chatbot application built with Gradio and powered by a Hugging Face conversational AI model (DialoGPT-medium).
+This is a chatbot application built with Gradio and powered by Microsoft's Phi-2 model, which is specialized in coding and general conversational tasks.
 
 ## Features
 
-- Conversational AI using Microsoft's DialoGPT-medium model (larger and more capable than the small version)
+- AI powered by Phi-2 (2.7B parameters), excellent for coding assistance
+- Streaming responses for faster interaction (text appears as it's generated)
 - Gradio interface for easy interaction
 - Maintains conversation history
-- Supports Hugging Face token for accessing private models or increased rate limits
+- Supports Hugging Face token for accessing models
 
 ## Setup
 
-To use a Hugging Face token (recommended for better performance and access to larger models):
+To use a Hugging Face token (recommended):
 
 1. Create a Hugging Face account at https://huggingface.co
 2. Generate a token at https://huggingface.co/settings/tokens
@@ -34,13 +33,13 @@ To use a Hugging Face token (recommended for better performance and access to la
 
 1. Run the app: `python app.py`
 2. Open your browser to the provided URL
-3. Start chatting with the AI!
+3. Start chatting with the AI! It can help with coding questions and general conversations.
 
 ## Model
 
-The app uses `microsoft/DialoGPT-medium`, a conversational model trained on Reddit conversations. This is a larger model than the previous small version, offering better responses but requiring more computational resources.
+The app uses `microsoft/phi-2`, a 2.7B parameter model fine-tuned for coding and instruction-following tasks. It's larger than previous models and provides better responses, especially for technical questions.
 
-Note: Larger models like this may take longer to load and respond, especially on free-tier hosting.
+Note: This model may take longer to load and respond on free-tier hosting, but responses stream in real-time.
 
 ## Built with anycoder
 

app.py

@@ -1,41 +1,58 @@
 import gradio as gr
-from transformers import pipeline
+from transformers import AutoTokenizer, AutoModelForCausalLM, pipeline
 import os
+import torch
 
-# Load the conversational model with HF token support
-# Using DialoGPT-medium for a larger, more capable chatbot
+# Load the model and tokenizer for a coding expert AI
+# Using Phi-2 which is good for coding and conversational tasks
 token = os.getenv('HF_TOKEN')
-chatbot_model = pipeline("text-generation", model="microsoft/DialoGPT-medium", token=token)
+model_name = "microsoft/phi-2"
+tokenizer = AutoTokenizer.from_pretrained(model_name, token=token, trust_remote_code=True)
+model = AutoModelForCausalLM.from_pretrained(model_name, token=token, trust_remote_code=True, torch_dtype=torch.float16, device_map="auto")
 
 def chat(message, history):
-    # Build conversation string from history
-    conversation_text = ""
+    # Build conversation prompt
+    prompt = ""
     for user_msg, bot_msg in history:
         if user_msg:
-            conversation_text += f"@@PADDING@@ {user_msg} @@PADDING@@ "
+            prompt += f"User: {user_msg}\n"
         if bot_msg:
-            conversation_text += f"{bot_msg} @@PADDING@@ "
+            prompt += f"Assistant: {bot_msg}\n"
     
-    # Add current user message
-    conversation_text += f"@@PADDING@@ {message} @@PADDING@@ "
+    prompt += f"User: {message}\nAssistant:"
     
-    # Generate response
-    result = chatbot_model(conversation_text, max_length=1000, num_return_sequences=1, temperature=0.8, do_sample=True, pad_token_id=50256)
+    # Tokenize input
+    inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
     
-    # Extract the response
-    generated_text = result[0]['generated_text']
+    # Generate response with streaming
+    generated_tokens = []
+    with torch.no_grad():
+        for _ in range(100):  # Limit to prevent infinite generation
+            outputs = model(**inputs)
+            next_token_logits = outputs.logits[:, -1, :]
+            next_token = torch.multinomial(torch.softmax(next_token_logits, dim=-1), num_samples=1)
+            generated_tokens.append(next_token.item())
+            
+            # Yield partial response
+            current_text = tokenizer.decode(generated_tokens, skip_special_tokens=True)
+            yield current_text
+            
+            # Check for end of response (simple heuristic: if ends with newline or period)
+            if current_text.endswith(('\n', '.', '!', '?')) and len(current_text) > 10:
+                break
+            
+            # Update inputs for next token
+            inputs = torch.cat([inputs['input_ids'], next_token], dim=-1)
     
-    # Split by padding and take the last part as response
-    parts = generated_text.split("@@PADDING@@")
-    response = parts[-1].strip() if parts else generated_text.strip()
-    
-    return response
+    # Final yield
+    final_response = tokenizer.decode(generated_tokens, skip_special_tokens=True).strip()
+    yield final_response
 
-# Create Gradio interface
+# Create Gradio interface with streaming enabled
 demo = gr.ChatInterface(
     fn=chat,
-    title="AI Chatbot",
-    description="Chat with an AI powered by Hugging Face transformers. <a href='https://huggingface.co/spaces/akhaliq/anycoder' target='_blank'>Built with anycoder</a>",
+    title="Coding Expert AI Chatbot",
+    description="Chat with a coding expert AI powered by Phi-2. It can help with programming questions and general conversations. <a href='https://huggingface.co/spaces/akhaliq/anycoder' target='_blank'>Built with anycoder</a>",
     theme=gr.themes.Soft()
 )