File size: 3,835 Bytes
03afb93 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 |
Tiny Llama Project Guide: Running TinyLlama-1.1B-Chat-v1.0 Locally
This document provides a step-by-step guide to run the TinyLlama-1.1B-Chat-v1.0 model locally on a laptop with 16GB RAM, i5 processor, and Windows OS. The guide includes setting up the environment, downloading the model, fine-tuning, and running a Flask-based chat UI.
---
System Requirements
Operating System: Windows
RAM: 16GB
Processor: Intel i5 or equivalent
Python Version: 3.10.9
- IDE: Visual Studio Code (VS Code)
- Internet: Required for downloading model and libraries
---
Step-by-Step Setup
1. Install Python 3.10.9
- Download and install Python 3.10.9 from https://www.python.org/downloads/release/python-3109/.
- Ensure Python and pip are added to your system PATH.
2. Set Up a Virtual Environment
- Open VS Code terminal in your project directory (e.g., C:\path\to\TinyLlama-1.1B).
- Run:
```
python -m venv venv
.\venv\Scripts\activate
```
3. Install Required Libraries
- In the activated virtual environment, run:
```
pip install transformers torch huggingface_hub datasets peft trl accelerate flask matplotlib
```
- This installs libraries for model handling, fine-tuning, Flask app, and plotting.
4. Download the TinyLlama Model
- Create a file `download_model.py` with the following code:
```python
from huggingface_hub import login, snapshot_download
login(token="YOUR_ACCESS_TOKEN_HERE")
snapshot_download(repo_id="TinyLlama/TinyLlama-1.1B-Chat-v1.0", local_dir="./tinyllama_model")
```
- Replace `YOUR_ACCESS_TOKEN_HERE` with your Hugging Face access token (get it from https://huggingface.co/settings/tokens).
- Run: `python download_model.py`
- Model weights will be saved in the `tinyllama_model` folder.
5. Run Inference with Flask UI
- Create a `finetune` folder in your project directory.
- Copy `app.py` and `templates/index.html` from the repository to the `finetune` folder.
- Run: `python app.py`
- Open http://127.0.0.1:5000 in your browser to access the chat UI.
- Enter prompts to interact with the model.
6. Fine-Tune the Model (Optional)
- In the `finetune` folder, ensure `dataset.json` and `finetune.py` are present.
- Run: `python finetune.py`
- Fine-tuned weights will be saved in `finetune/finetuned_weights`.
- Update `app.py` to point to `./finetuned_weights` for inference with the fine-tuned model.
- Check `loss_plot.png` for training loss visualization.
7. View Training Metrics
- After fine-tuning, check the console for final train loss and learning rate.
- Open `loss_plot.png` in the `finetune` folder for a graphical view of training loss.
---
Project Structure
- `tinyllama_model/`: Model weights downloaded from Hugging Face.
- `finetune/`: Contains fine-tuning scripts and fine-tuned weights.
- `dataset.json`: Small dataset for fine-tuning.
- `finetune.py`: Fine-tuning script with LoRA.
- `app.py`: Flask app for inference.
- `templates/index.html`: Chat UI.
- `loss_plot.png`: Training loss plot.
- `requirements.txt`: List of required libraries.
- `document.txt`: This guide.
- `README.md`: Project overview.
---
Attribution
- **Model**: TinyLlama-1.1B-Chat-v1.0
- **Source**: https://huggingface.co/TinyLlama/TinyLlama-1.1B-Chat-v1.0
- **Organization**: TinyLlama
- **License**: Check the model's Hugging Face page for licensing details.
---
Notes
- Model weights are not included in this repository to respect licensing terms.
- Download the model directly from Hugging Face using your access token.
- Ensure sufficient disk space (~2-3GB) for model weights and fine-tuned weights.
- For support, refer to the TinyLlama Hugging Face page or community forums. |