Update README.md
Browse files
README.md
CHANGED
|
@@ -1,5 +1,5 @@
|
|
| 1 |
---
|
| 2 |
-
title:
|
| 3 |
emoji: 🐨
|
| 4 |
colorFrom: yellow
|
| 5 |
colorTo: indigo
|
|
@@ -10,4 +10,38 @@ pinned: false
|
|
| 10 |
license: apache-2.0
|
| 11 |
---
|
| 12 |
|
| 13 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
+
title: SLM Instruction Tuning using Unsloth
|
| 3 |
emoji: 🐨
|
| 4 |
colorFrom: yellow
|
| 5 |
colorTo: indigo
|
|
|
|
| 10 |
license: apache-2.0
|
| 11 |
---
|
| 12 |
|
| 13 |
+
# SLM Instruction Tuning using Unsloth
|
| 14 |
+
|
| 15 |
+
### What?
|
| 16 |
+
|
| 17 |
+
This Gradio app is a simple interface to access [unsloth AI's](https://github.com/unslothai) fine-tuning methods but leveraging the A100 GPUs provided by [Huggingface Spaces](https://huggingface.co/docs/hub/en/spaces-overview). This outputs of this fine-tuning will be instruction tuned LoRA weights that will be uploaded into your personal huggingface models repository.
|
| 18 |
+
|
| 19 |
+
[<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)
|
| 20 |
+
|
| 21 |
+
### Why?
|
| 22 |
+
|
| 23 |
+
The goal of this demo is to show how you can tune your own language models leveraging industry standard compute and fine tuning methods using a simple point-and-click UI.
|
| 24 |
+
|
| 25 |
+
In addition, compute, even on Google Colab's free tier is tight even with a T4 and rate limits are uncertain. This makes the use of the A100s on this demo useful for a small added boost to compute performance. For those looking to reduce the costs associated with training datasets can pull down the spaces repository to train their models at speed for $9 on The Huggingface Pro License.
|
| 26 |
+
|
| 27 |
+
This is a demo and not a production application. This application is subject a demand queue.
|
| 28 |
+
|
| 29 |
+
### How?
|
| 30 |
+
|
| 31 |
+
Just start by following the guide below:
|
| 32 |
+
|
| 33 |
+
1) Flip to the Train Model tab at the top.
|
| 34 |
+
2) Populate your username, repository, token, and model details
|
| 35 |
+
3) Upload data from transformers or your local jsonl file. Please view [this guide](https://platform.openai.com/docs/guides/fine-tuning/preparing-your-dataset) for best practices.
|
| 36 |
+
4) Eat a snack and wait as you train the model.
|
| 37 |
+
|
| 38 |
+
### Coming soon!
|
| 39 |
+
|
| 40 |
+
- More models and added flexibility with guardrails on hyperparameter tuning.
|
| 41 |
+
- Downloads for a [WandB](https://wandb.ai/home) logger for training monitoring.
|
| 42 |
+
|
| 43 |
+
### Other resources.
|
| 44 |
+
|
| 45 |
+
- [Unsloth's notebooks](https://colab.research.google.com/drive/1hhdhBa1j_hsymiW9m-WzxQtgqTH_NHqi?usp=sharing) to look at what is going on under the hood.
|
| 46 |
+
|
| 47 |
+
|