File size: 1,550 Bytes
3392c26 312485e ea7d11e 635a2df ea7d11e f0dd1b8 ea7d11e 3392c26 ea7d11e 3392c26 ea7d11e 8d6f37f f0dd1b8 3392c26 ea7d11e f0dd1b8 9a6c51f ea7d11e 8d6f37f ea7d11e f0dd1b8 ea7d11e 8d6f37f ea7d11e 8d6f37f ea7d11e 8d6f37f ea7d11e adb379c ea7d11e |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 |
---
library_name: peft
tags:
- code
- instruct
- zephyr
datasets:
- HuggingFaceH4/no_robots
base_model: HuggingFaceH4/zephyr-7b-alpha
license: apache-2.0
---
### Finetuning Overview:
**Model Used:** HuggingFaceH4/zephyr-7b-alpha
**Dataset:** HuggingFaceH4/no_robots
#### Dataset Insights:
[No Robots](https://huggingface.co/datasets/HuggingFaceH4/no_robots) is a high-quality dataset of 10,000 instructions and demonstrations created by skilled human annotators. This data can be used for supervised fine-tuning (SFT) to make language models follow instructions better.
#### Finetuning Details:
With the utilization of [MonsterAPI](https://monsterapi.ai)'s [LLM finetuner](https://docs.monsterapi.ai/fine-tune-a-large-language-model-llm), this finetuning:
- Was achieved with great cost-effectiveness.
- Completed in a total duration of 36mins 47secs for 1 epoch using an A6000 48GB GPU.
- Costed `$1.212` for the entire epoch.
#### Hyperparameters & Additional Details:
- **Epochs:** 1
- **Cost Per Epoch:** $1.212
- **Total Finetuning Cost:** $1.212
- **Model Path:** HuggingFaceH4/zephyr-7b-alpha
- **Learning Rate:** 0.0002
- **Data Split:** 100% train
- **Gradient Accumulation Steps:** 4
- **lora r:** 32
- **lora alpha:** 64
---
#### Prompt Structure
```
<|system|> <|endoftext|> <|user|> [USER PROMPT]<|endoftext|> <|assistant|> [ASSISTANT ANSWER] <|endoftext|>
```
#### Train loss :
license: apache-2.0 |