README.md

metadata

license: mit
title: LoRA Fine-Tuning and CoreML Conversion with Streamlit
sdk: streamlit
emoji: 🦀
colorFrom: green
colorTo: blue
pinned: true
sdk_version: 1.46.0

LoRA Fine-Tuning and CoreML Conversion with Streamlit

This project demonstrates how to fine-tune a large language model (LLM) using Low-Rank Adaptation (LoRA) and convert it to the CoreML format for on-device deployment. The entire process is wrapped in a user-friendly Streamlit web application.

Features

• Fine-Tune LLMs with LoRA: Easily fine-tune the distilbert/distilgpt2 model on the roneneldan/TinyStories dataset using Parameter-Efficient Fine-Tuning (PEFT) with LoRA.
• Text Generation: Generate creative stories from a text prompt using the fine-tuned model.
• Adjustable Generation Parameters: Control text generation with parameters like temperature, max length, and repetition penalty.
• CoreML Conversion: Convert the fine-tuned model to a .mlpackage file, ready for integration into Apple ecosystem applications.
• Interactive Web UI: A simple and interactive user interface built with Streamlit.

How it Works

The application follows a simple workflow:

1. Load Base Model: It starts by loading the pre-trained distilbert/distilgpt2 model and its tokenizer from the Hugging Face Hub.
2. Fine-Tuning: The user can initiate the fine-tuning process. The application uses the peft library to apply LoRA to the base model and trains it on the TinyStories dataset. The resulting LoRA adapter is saved locally.
3. Text Generation: Once the model is fine-tuned (or a pre-existing adapter is loaded), you can provide a prompt to generate stories.
4. CoreML Conversion: The application can merge the LoRA adapter with the base model and then convert the merged model into the CoreML format, which can be downloaded as a .zip file.

Technologies Used

• Model: distilbert/distilgpt2 from Hugging Face
• Dataset: roneneldan/TinyStories from Hugging Face
• Fine-Tuning: peft (Parameter-Efficient Fine-Tuning) library with LoRA
• Framework: PyTorch
• Web App: Streamlit
• Model Conversion: CoreMLTools
• Core Libraries: transformers, datasets, accelerate, sentencepiece

Setup and Usage

1. Clone the repository:

   git clone <repository-url>
   cd <repository-directory>
   

2. Create a virtual environment and install dependencies:

   python3 -m venv venv
   source venv/bin/activate
   pip install -r requirements.txt
   

3. Run the Streamlit application:

   streamlit run app.py
   

4. Open the application in your browser: Navigate to the URL provided by Streamlit (usually http://localhost:8501).

File Structure

• app.py: The main Python script containing the Streamlit application logic.
• requirements.txt: A list of the Python packages required to run the project.
• README.md: This file, providing information about the project.
• distilgpt2-lora-tinystories/: (Generated Directory) This directory will be created to store the LoRA adapter after fine-tuning.
• results/: (Generated Directory) This directory is used by the transformers.Trainer to save training outputs.
• distilgpt2-lora-tinystories.mlpackage/: (Generated Directory) This directory will be created after the CoreML conversion.
• distilgpt2-lora-tinystories.zip: (Generated File) The zipped CoreML model ready for download.

---

Happy fine-tuning and story generating!