README.md

metadata

title: Phi-2 QLoRA Assistant Demo
emoji: 🤖
colorFrom: blue
colorTo: purple
sdk: gradio
sdk_version: 4.19.2
app_file: app.py
pinned: false

Phi-2 QLoRA Fine-tuned Assistant

This is a fine-tuned version of Microsoft's Phi-2 model using QLoRA (Quantized Low-Rank Adaptation) technique. The model has been trained to excel at various tasks including coding, technical explanations, and professional writing.

Model Description

• Base Model: Microsoft Phi-2
• Training Method: QLoRA (Quantized Low-Rank Adaptation)
• Training Data: Custom dataset focused on coding, technical explanations, and professional communication
• Primary Use Cases: Code generation, technical writing, and professional communication

Usage Tips

For Code Generation (Temperature: 0.3-0.5)

# Example prompt:
"Write a Python function to calculate the factorial of a number and provide additional recursive function examples"

For Technical Explanations (Temperature: 0.7)

# Example prompt:
"Explain what machine learning is in simple terms and provide some real-world applications"

For Professional Writing (Temperature: 0.7-0.9)

# Example prompt:
"Write a professional email to schedule a team meeting for next week to discuss project progress"

Parameters Guide

• Maximum Length: 64-1024 (default: 512)

  • Increase for longer responses
  • Decrease for quicker, more concise responses

• Temperature: 0.1-1.0 (default: 0.7)

  • 0.3-0.5: Best for code generation
  • 0.7-0.9: Best for creative writing
  • 1.0: Maximum creativity

• Top P: 0.1-1.0 (default: 0.9)

  • Controls diversity of word choices
  • Higher values = more diverse vocabulary

Model Links

• Model Card: pradeep6kumar2024/phi2-qlora-assistant
• Base Model: microsoft/phi-2

License

This demo is released under the MIT License.

Example Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

# Load base model and adapter
base_model = AutoModelForCausalLM.from_pretrained("microsoft/phi-2")
model = PeftModel.from_pretrained(base_model, "pradeep6kumar2024/phi2-qlora-assistant")
tokenizer = AutoTokenizer.from_pretrained("microsoft/phi-2")

# Generate text
prompt = "Write a Python function to calculate the factorial of a number"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_length=512)
response = tokenizer.decode(outputs[0], skip_special_tokens=True)

Example Outputs

1. Coding Task:

   def factorial(n):
       if n == 0 or n == 1:
           return 1
       return n * factorial(n-1)
   

2. Technical Explanation: "Machine learning is a branch of artificial intelligence that enables computers to learn from data without being explicitly programmed. Think of it like teaching a child - instead of giving them strict rules, you show them examples and they learn to recognize patterns..."

3. Professional Writing: "Dear Team, I hope this email finds you well. I would like to schedule a team meeting next week to discuss our project progress..."

Limitations

• The model works best with clear, well-structured prompts
• Code generation is optimized for Python but can handle other languages
• Response quality may vary with very long or complex prompts

Try It Out

You can try this model directly in your browser using our Gradio Space: Phi2-QLoRA-Assistant Demo

Acknowledgments

• Microsoft for the Phi-2 base model
• Hugging Face for the transformers library and hosting
• The QLoRA paper authors for the fine-tuning technique