Commit
·
46e1228
1
Parent(s):
59a988d
add in QLoRa Training script
Browse files
README.md
CHANGED
|
@@ -5,11 +5,21 @@ colorFrom: red
|
|
| 5 |
colorTo: green
|
| 6 |
sdk: static
|
| 7 |
pinned: false
|
| 8 |
-
tags: [llama, jupyter, colab, function calling]
|
| 9 |
---
|
| 10 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 11 |
# Function Calling Dataset
|
| 12 |
-
- Commercial dataset allowing language models to be fine-tuned for function calling ([
|
| 13 |
- Created using only human input or Apache 2 licensed datasets (no third party commercial licensing limitations)
|
| 14 |
- Models trained with this dataset:
|
| 15 |
- - [Llama-2-7B](https://huggingface.co/Trelis/Llama-2-7b-chat-hf-function-calling) - repo includes Google Colab notebook
|
|
|
|
| 5 |
colorTo: green
|
| 6 |
sdk: static
|
| 7 |
pinned: false
|
| 8 |
+
tags: [llama, jupyter, colab, function calling, QLoRa, fine-tuning]
|
| 9 |
---
|
| 10 |
|
| 11 |
+
# *QLoRa Training Notebook for Small Datasets*
|
| 12 |
+
|
| 13 |
+
Fine-tune HuggingFace language models in Google Colab.
|
| 14 |
+
- Includes a prompt loss-mask for improved performance on small datasets.
|
| 15 |
+
- Includes a stop token after responses - allowing the model to provide a short reponse (e.g. a function call) and then stop.
|
| 16 |
+
- Allows for fine-tuning 7B and 13B models with the free plan.
|
| 17 |
+
- Allows for fine-tuning 70B models with Google Colab Pro.
|
| 18 |
+
|
| 19 |
+
This script is commercially licensed and available for purchase at a price of €14.99 (or $16.49) per seat/user. [Request access here](https://buy.stripe.com/5kA5l69K52Hxf3a006). Access will be provided within 24 hours of purchase.
|
| 20 |
+
|
| 21 |
# Function Calling Dataset
|
| 22 |
+
- Commercial dataset allowing language models to be fine-tuned for function calling ([Get access here](https://huggingface.co/datasets/Trelis/function_calling_extended)).
|
| 23 |
- Created using only human input or Apache 2 licensed datasets (no third party commercial licensing limitations)
|
| 24 |
- Models trained with this dataset:
|
| 25 |
- - [Llama-2-7B](https://huggingface.co/Trelis/Llama-2-7b-chat-hf-function-calling) - repo includes Google Colab notebook
|